2024-02-26T04:33:58.818 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-26T04:33:58.818 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376 --name vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi --verbose --owner scheduled_vshankar@teuthology --description fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/direct_io}} -- /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376/orig.config.yaml 2024-02-26T04:33:58.871 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-26T04:33:58.963 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376 branch: wip-vshankar-testing-20240216.060239 description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/direct_io}} email: vshankar@redhat.com first_in_suite: false job_id: '7573376' kernel: client: sha1: distro flavor: default kdb: true sha1: e861cfd5f0314cb9ccffd908cd661b60f9f09ce1 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240216.060239 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 25296a8f176256aea5ed8b5850b25167dce0ae33 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 25296a8f176256aea5ed8b5850b25167dce0ae33 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240216.060239 sha1: bc0b84a094b787eb15ce0eff5f7a342217d95e39 owner: scheduled_vshankar@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9861 sha1: 25296a8f176256aea5ed8b5850b25167dce0ae33 sleep_before_teardown: 0 subset: 914/1024 suite: fs:workload suite_branch: wip-vshankar-testing-20240216.060239 suite_path: /home/teuthworker/src/github.com_vshankar_ceph_bc0b84a094b787eb15ce0eff5f7a342217d95e39/qa suite_relpath: qa suite_repo: https://github.com/vshankar/ceph.git suite_sha1: bc0b84a094b787eb15ce0eff5f7a342217d95e39 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBE0248Lkbm9nd8IorZk9W5/tVxMKneLUXTFgFtnIyoDsvbZN+X4FsywsLHE9eMiv/vEsUlUjppN2yM/rWKzMBE= smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI1FvUUWLV6SY0EJCg0XhSRZuVm3ltLE4Mk/1+rauJq+nj2S/YKmIOQWxo/3vlJWh1MlDKZLLkhZZr0CPBQXVvc= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAzMIylPe644StE4DN8EqmhqurfOL15oHbwkuNjcE0WfcivbJH6NbPUMoNOEQK7pOFksjD1OXmaOZMPSPCRk/6I= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-26_04:25:56 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-26T04:33:58.963 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_vshankar_ceph_bc0b84a094b787eb15ce0eff5f7a342217d95e39/qa; will attempt to use it 2024-02-26T04:33:58.964 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_vshankar_ceph_bc0b84a094b787eb15ce0eff5f7a342217d95e39/qa/tasks 2024-02-26T04:33:58.964 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-26T04:33:58.965 INFO:teuthology.task.internal:Checking packages... 2024-02-26T04:33:58.986 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '25296a8f176256aea5ed8b5850b25167dce0ae33' 2024-02-26T04:33:58.986 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-26T04:33:58.986 INFO:teuthology.packaging:ref: None 2024-02-26T04:33:58.986 INFO:teuthology.packaging:tag: None 2024-02-26T04:33:58.986 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240216.060239 2024-02-26T04:33:58.986 INFO:teuthology.packaging:sha1: 25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T04:33:58.987 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240216.060239 2024-02-26T04:33:59.128 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1514.g25296a8f 2024-02-26T04:33:59.129 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-26T04:33:59.139 INFO:teuthology.task.internal:no buildpackages task found 2024-02-26T04:33:59.139 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-26T04:33:59.161 INFO:teuthology.task.internal:Saving configuration 2024-02-26T04:33:59.180 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-26T04:33:59.191 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T04:33:59.216 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 04:26:44.043620', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBE0248Lkbm9nd8IorZk9W5/tVxMKneLUXTFgFtnIyoDsvbZN+X4FsywsLHE9eMiv/vEsUlUjppN2yM/rWKzMBE='} 2024-02-26T04:33:59.236 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 04:26:44.045440', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI1FvUUWLV6SY0EJCg0XhSRZuVm3ltLE4Mk/1+rauJq+nj2S/YKmIOQWxo/3vlJWh1MlDKZLLkhZZr0CPBQXVvc='} 2024-02-26T04:33:59.259 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 04:26:44.046581', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAzMIylPe644StE4DN8EqmhqurfOL15oHbwkuNjcE0WfcivbJH6NbPUMoNOEQK7pOFksjD1OXmaOZMPSPCRk/6I='} 2024-02-26T04:33:59.259 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-26T04:33:59.265 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-02-26T04:33:59.265 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-02-26T04:33:59.266 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-02-26T04:33:59.266 INFO:teuthology.run_tasks:Running task console_log... 2024-02-26T04:33:59.371 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9a3958af70>, signals=[15]) 2024-02-26T04:33:59.371 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-26T04:33:59.380 INFO:teuthology.task.internal:Opening connections... 2024-02-26T04:33:59.380 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T04:33:59.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T04:33:59.486 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2024-02-26T04:33:59.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T04:33:59.573 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2024-02-26T04:33:59.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T04:33:59.660 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-26T04:33:59.667 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2024-02-26T04:33:59.686 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2024-02-26T04:33:59.686 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2024-02-26T04:33:59.744 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2024-02-26T04:33:59.744 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="9" 2024-02-26T04:33:59.744 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2024-02-26T04:33:59.744 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2024-02-26T04:33:59.744 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="9" 2024-02-26T04:33:59.744 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el9" 2024-02-26T04:33:59.744 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-26T04:33:59.744 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2024-02-26T04:33:59.744 INFO:teuthology.orchestra.run.smithi060.stdout:LOGO="fedora-logo-icon" 2024-02-26T04:33:59.744 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-26T04:33:59.744 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2024-02-26T04:33:59.745 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-26T04:33:59.745 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-26T04:33:59.745 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-26T04:33:59.745 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2024-02-26T04:33:59.776 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2024-02-26T04:33:59.794 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2024-02-26T04:33:59.795 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2024-02-26T04:33:59.851 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="CentOS Stream" 2024-02-26T04:33:59.852 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="9" 2024-02-26T04:33:59.852 INFO:teuthology.orchestra.run.smithi156.stdout:ID="centos" 2024-02-26T04:33:59.852 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="rhel fedora" 2024-02-26T04:33:59.852 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="9" 2024-02-26T04:33:59.852 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el9" 2024-02-26T04:33:59.852 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-26T04:33:59.852 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2024-02-26T04:33:59.852 INFO:teuthology.orchestra.run.smithi156.stdout:LOGO="fedora-logo-icon" 2024-02-26T04:33:59.852 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-26T04:33:59.852 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://centos.org/" 2024-02-26T04:33:59.852 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-26T04:33:59.852 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-26T04:33:59.852 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-26T04:33:59.853 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2024-02-26T04:33:59.876 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-02-26T04:33:59.894 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-02-26T04:33:59.894 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-02-26T04:33:59.951 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-02-26T04:33:59.952 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="9" 2024-02-26T04:33:59.952 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-02-26T04:33:59.952 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-02-26T04:33:59.952 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="9" 2024-02-26T04:33:59.952 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el9" 2024-02-26T04:33:59.952 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-26T04:33:59.952 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-02-26T04:33:59.952 INFO:teuthology.orchestra.run.smithi191.stdout:LOGO="fedora-logo-icon" 2024-02-26T04:33:59.952 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-26T04:33:59.952 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-02-26T04:33:59.952 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-26T04:33:59.952 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-26T04:33:59.952 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-26T04:33:59.953 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-02-26T04:33:59.976 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-26T04:33:59.987 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-26T04:33:59.995 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-26T04:33:59.996 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2024-02-26T04:33:59.998 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2024-02-26T04:34:00.000 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2024-02-26T04:34:00.017 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-26T04:34:00.025 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-26T04:34:00.025 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2024-02-26T04:34:00.055 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2024-02-26T04:34:00.057 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2024-02-26T04:34:00.175 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-26T04:34:00.245 INFO:teuthology.run_tasks:Running task kernel... 2024-02-26T04:34:00.260 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'e861cfd5f0314cb9ccffd908cd661b60f9f09ce1'} 2024-02-26T04:34:00.261 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-02-26T04:34:00.261 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-02-26T04:34:00.261 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-26T04:34:00.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T04:34:00.279 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-02-26T04:34:00.337 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-361.el9.x86_64 2024-02-26T04:34:00.337 INFO:teuthology.task.kernel:Running kernel on smithi060: 5.14.0-361.el9.x86_64 2024-02-26T04:34:00.337 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum install -y kernel 2024-02-26T04:34:02.807 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - AppStream 9.9 MB/s | 8.1 MB 00:00 2024-02-26T04:34:04.799 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.5 MB 00:00 2024-02-26T04:34:05.739 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-02-26T04:34:07.841 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-02-26T04:34:16.483 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 6.1 MB 00:04 2024-02-26T04:34:18.983 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 15 kB 00:01 2024-02-26T04:34:21.260 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 20 MB 00:02 2024-02-26T04:34:26.402 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-02-26T04:34:28.765 INFO:teuthology.orchestra.run.smithi060.stderr:Modular dependency problems: 2024-02-26T04:34:28.765 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-02-26T04:34:28.766 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 1: conflicting requests 2024-02-26T04:34:28.766 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.766 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 2: conflicting requests 2024-02-26T04:34:28.766 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.766 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 3: conflicting requests 2024-02-26T04:34:28.766 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.766 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 4: conflicting requests 2024-02-26T04:34:28.766 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.766 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 5: conflicting requests 2024-02-26T04:34:28.766 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.766 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 6: conflicting requests 2024-02-26T04:34:28.766 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.766 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 7: conflicting requests 2024-02-26T04:34:28.767 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.767 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 8: conflicting requests 2024-02-26T04:34:28.767 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.767 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 9: conflicting requests 2024-02-26T04:34:28.767 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.767 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 10: conflicting requests 2024-02-26T04:34:28.767 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.767 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 11: conflicting requests 2024-02-26T04:34:28.767 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.767 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 12: conflicting requests 2024-02-26T04:34:28.767 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.767 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 13: conflicting requests 2024-02-26T04:34:28.768 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.768 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 14: conflicting requests 2024-02-26T04:34:28.768 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.768 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 15: conflicting requests 2024-02-26T04:34:28.768 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.768 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 16: conflicting requests 2024-02-26T04:34:28.768 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.768 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 17: conflicting requests 2024-02-26T04:34:28.768 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.768 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 18: conflicting requests 2024-02-26T04:34:28.768 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.768 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 19: conflicting requests 2024-02-26T04:34:28.768 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.768 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 20: conflicting requests 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 21: conflicting requests 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 22: conflicting requests 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 23: conflicting requests 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 24: conflicting requests 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 25: conflicting requests 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 26: conflicting requests 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 27: conflicting requests 2024-02-26T04:34:28.769 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 28: conflicting requests 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 29: conflicting requests 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 30: conflicting requests 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 31: conflicting requests 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 32: conflicting requests 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 33: conflicting requests 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 34: conflicting requests 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.770 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 35: conflicting requests 2024-02-26T04:34:28.771 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.771 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 36: conflicting requests 2024-02-26T04:34:28.771 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.771 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.771 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.771 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T04:34:28.771 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.771 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.771 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.771 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.771 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 38: conflicting requests 2024-02-26T04:34:28.771 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.771 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.771 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.772 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.772 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.772 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.772 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T04:34:28.772 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.772 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.772 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.772 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.772 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T04:34:28.772 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.772 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.772 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.772 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.772 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.773 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T04:34:28.774 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.774 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.774 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T04:34:28.774 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T04:34:29.167 INFO:teuthology.orchestra.run.smithi060.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-26T04:34:29.167 INFO:teuthology.orchestra.run.smithi060.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-26T04:34:29.266 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-02-26T04:34:29.279 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-02-26T04:34:29.279 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2024-02-26T04:34:29.279 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-02-26T04:34:29.279 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-02-26T04:34:29.279 INFO:teuthology.orchestra.run.smithi060.stdout: kernel x86_64 5.14.0-419.el9 baseos 6.0 M 2024-02-26T04:34:29.279 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-02-26T04:34:29.279 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-core x86_64 5.14.0-419.el9 baseos 21 M 2024-02-26T04:34:29.279 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules x86_64 5.14.0-419.el9 baseos 39 M 2024-02-26T04:34:29.279 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-core x86_64 5.14.0-419.el9 baseos 33 M 2024-02-26T04:34:29.279 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:34:29.279 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-02-26T04:34:29.279 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-02-26T04:34:29.279 INFO:teuthology.orchestra.run.smithi060.stdout:Install 4 Packages 2024-02-26T04:34:29.280 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:34:29.280 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 99 M 2024-02-26T04:34:29.280 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 122 M 2024-02-26T04:34:29.280 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-02-26T04:34:30.236 INFO:teuthology.orchestra.run.smithi060.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 8.8 MB/s | 6.0 MB 00:00 2024-02-26T04:35:00.309 INFO:teuthology.orchestra.run.smithi060.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 694 kB/s | 21 MB 00:30 2024-02-26T04:35:08.892 INFO:teuthology.orchestra.run.smithi060.stdout:(3/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 1.0 MB/s | 39 MB 00:39 2024-02-26T04:35:15.350 INFO:teuthology.orchestra.run.smithi060.stdout:(4/4): kernel-modules-core-5.14.0-419.el9.x86_6 757 kB/s | 33 MB 00:45 2024-02-26T04:35:15.350 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-02-26T04:35:15.350 INFO:teuthology.orchestra.run.smithi060.stdout:Total 2.2 MB/s | 99 MB 00:46 2024-02-26T04:35:16.268 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-02-26T04:35:16.713 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-02-26T04:35:16.713 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-02-26T04:35:18.082 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-02-26T04:35:18.083 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-02-26T04:35:20.933 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-02-26T04:35:21.602 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-26T04:35:21.727 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-26T04:35:24.653 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-26T04:35:24.839 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-26T04:35:29.944 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-26T04:35:30.076 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-26T04:35:35.217 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-26T04:36:06.068 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-26T04:36:06.068 INFO:teuthology.orchestra.run.smithi060.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-26T04:36:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:36:06.157 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-26T04:36:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-26T04:36:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-26T04:36:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-26T04:36:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-26T04:36:07.209 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-26T04:36:07.209 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:36:07.209 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-02-26T04:36:07.210 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-26T04:36:07.210 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-26T04:36:07.210 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-26T04:36:07.210 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-26T04:36:07.210 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:36:07.210 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-02-26T04:36:07.882 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q kernel | sort -rV | head -n 1 2024-02-26T04:36:08.026 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-26T04:36:08.026 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-26T04:36:08.027 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-26T04:36:08.027 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-26T04:36:08.027 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-26T04:36:08.027 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-02-26T04:36:08.027 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi060.front.sepia.ceph.com, path=None, version=distro) 2024-02-26T04:36:08.027 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q kernel | sort -rV | head -n 1 2024-02-26T04:36:08.152 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-26T04:36:08.153 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-26T04:36:08.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -qi grub2-tools 2024-02-26T04:36:08.199 INFO:teuthology.orchestra.run.smithi060.stdout:Name : grub2-tools 2024-02-26T04:36:08.199 INFO:teuthology.orchestra.run.smithi060.stdout:Epoch : 1 2024-02-26T04:36:08.199 INFO:teuthology.orchestra.run.smithi060.stdout:Version : 2.06 2024-02-26T04:36:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:Release : 68.el9 2024-02-26T04:36:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: x86_64 2024-02-26T04:36:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-26T04:36:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:Group : Unspecified 2024-02-26T04:36:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:Size : 8264254 2024-02-26T04:36:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:License : GPLv3+ 2024-02-26T04:36:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-26T04:36:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-26T04:36:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-26T04:36:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-26T04:36:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:Packager : builder@centos.org 2024-02-26T04:36:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:Vendor : CentOS 2024-02-26T04:36:08.200 INFO:teuthology.orchestra.run.smithi060.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-26T04:36:08.201 INFO:teuthology.orchestra.run.smithi060.stdout:Summary : Support tools for GRUB. 2024-02-26T04:36:08.201 INFO:teuthology.orchestra.run.smithi060.stdout:Description : 2024-02-26T04:36:08.201 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:36:08.201 INFO:teuthology.orchestra.run.smithi060.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-26T04:36:08.201 INFO:teuthology.orchestra.run.smithi060.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-26T04:36:08.201 INFO:teuthology.orchestra.run.smithi060.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-26T04:36:08.201 INFO:teuthology.orchestra.run.smithi060.stdout:hardware devices. 2024-02-26T04:36:08.201 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:36:08.201 INFO:teuthology.orchestra.run.smithi060.stdout:This subpackage provides tools for support of all platforms. 2024-02-26T04:36:08.202 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-26T04:36:08.202 INFO:teuthology.task.kernel:Updating grub on smithi060 to boot 5.14.0-419.el9.x86_64 2024-02-26T04:36:08.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-26T04:36:08.787 INFO:teuthology.orchestra.run.smithi060.stderr:Generating grub configuration file ... 2024-02-26T04:36:10.349 INFO:teuthology.orchestra.run.smithi060.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-26T04:36:10.389 INFO:teuthology.orchestra.run.smithi060.stderr:done 2024-02-26T04:36:10.391 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-02-26T04:36:10.406 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.ozCDW40jyH 2024-02-26T04:36:10.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ozCDW40jyH 2024-02-26T04:36:10.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.ozCDW40jyH 2024-02-26T04:36:10.620 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.ozCDW40jyH is 6KB 2024-02-26T04:36:10.668 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.ozCDW40jyH 2024-02-26T04:36:10.683 DEBUG:teuthology.orchestra.run.smithi060:> sudo /bin/ls /boot/loader/entries || true 2024-02-26T04:36:10.751 INFO:teuthology.orchestra.run.smithi060.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-26T04:36:10.751 INFO:teuthology.orchestra.run.smithi060.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-26T04:36:10.751 INFO:teuthology.orchestra.run.smithi060.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-26T04:36:10.751 INFO:teuthology.orchestra.run.smithi060.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-26T04:36:10.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-26T04:36:10.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo shutdown -r now 2024-02-26T04:36:40.916 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-02-26T04:36:40.917 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-02-26T04:36:40.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T04:36:59.324 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.60 2024-02-26T04:37:08.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-02-26T04:37:08.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T04:38:08.330 DEBUG:teuthology.orchestra.remote:timed out 2024-02-26T04:38:20.332 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-02-26T04:38:20.333 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T04:38:20.718 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-02-26T04:38:21.102 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-02-26T04:38:21.103 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-419.el9.x86_64"... 2024-02-26T04:38:21.103 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-02-26T04:38:21.119 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-419.el9.x86_64 2024-02-26T04:38:21.119 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-26T04:38:21.119 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-26T04:38:21.119 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-26T04:38:22.120 DEBUG:teuthology.parallel:result is None 2024-02-26T04:38:22.120 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-26T04:38:22.128 INFO:teuthology.task.internal:Creating test directory... 2024-02-26T04:38:22.128 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-26T04:38:22.130 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-26T04:38:22.133 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-26T04:38:22.151 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-26T04:38:22.157 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-26T04:38:22.166 INFO:teuthology.task.internal:Creating archive directory... 2024-02-26T04:38:22.166 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-26T04:38:22.187 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-26T04:38:22.191 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-26T04:38:22.314 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-26T04:38:22.323 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-26T04:38:22.323 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-26T04:38:22.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T04:38:22.339 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-26T04:38:22.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T04:38:22.356 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-26T04:38:22.358 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-26T04:38:22.382 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-26T04:38:22.436 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T04:38:22.440 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T04:38:22.451 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T04:38:22.459 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T04:38:22.459 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T04:38:22.492 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T04:38:22.494 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-26T04:38:22.501 INFO:teuthology.task.internal:Configuring sudo... 2024-02-26T04:38:22.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-26T04:38:22.503 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-26T04:38:22.506 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-26T04:38:22.535 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-26T04:38:22.546 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-26T04:38:22.546 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-26T04:38:22.575 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-26T04:38:22.578 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-26T04:38:22.593 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T04:38:22.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T04:38:22.721 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T04:38:22.806 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T04:38:22.848 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:38:22.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-26T04:38:22.917 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T04:38:22.959 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T04:38:23.014 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T04:38:23.099 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T04:38:23.143 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-26T04:38:23.143 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-26T04:38:23.213 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T04:38:23.258 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T04:38:23.312 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T04:38:23.394 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T04:38:23.440 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-26T04:38:23.440 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-26T04:38:23.508 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2024-02-26T04:38:23.511 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2024-02-26T04:38:23.513 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2024-02-26T04:38:23.561 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T04:38:23.569 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T04:38:23.591 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T04:38:24.038 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-26T04:38:24.046 INFO:teuthology.task.internal:Starting timer... 2024-02-26T04:38:24.046 INFO:teuthology.run_tasks:Running task pcp... 2024-02-26T04:38:24.056 INFO:teuthology.run_tasks:Running task selinux... 2024-02-26T04:38:24.064 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-26T04:38:24.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo service auditd rotate 2024-02-26T04:38:24.121 INFO:teuthology.orchestra.run.smithi060.stdout:Rotating logs: 2024-02-26T04:38:24.123 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2024-02-26T04:38:24.177 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2024-02-26T04:38:24.179 DEBUG:teuthology.orchestra.run.smithi191:> sudo service auditd rotate 2024-02-26T04:38:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:Rotating logs: 2024-02-26T04:38:24.267 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-26T04:38:24.267 DEBUG:teuthology.orchestra.run.smithi060:> /usr/sbin/getenforce 2024-02-26T04:38:24.290 INFO:teuthology.orchestra.run.smithi060.stdout:Permissive 2024-02-26T04:38:24.290 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2024-02-26T04:38:24.312 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2024-02-26T04:38:24.312 DEBUG:teuthology.orchestra.run.smithi191:> /usr/sbin/getenforce 2024-02-26T04:38:24.333 INFO:teuthology.orchestra.run.smithi191.stdout:Permissive 2024-02-26T04:38:24.333 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi060.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi156.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi191.front.sepia.ceph.com': 'permissive'} 2024-02-26T04:38:24.333 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T04:38:24.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T04:38:24.362 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T04:38:24.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T04:38:24.392 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T04:38:24.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T04:38:24.420 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-26T04:38:24.420 DEBUG:teuthology.orchestra.run.smithi060:> sudo /usr/sbin/setenforce permissive 2024-02-26T04:38:24.450 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2024-02-26T04:38:24.489 DEBUG:teuthology.orchestra.run.smithi191:> sudo /usr/sbin/setenforce permissive 2024-02-26T04:38:24.533 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-26T04:38:24.543 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-26T04:38:24.548 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-26T04:38:24.668 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-26T04:38:24.683 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-26T04:38:24.695 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi060.front.sepia.ceph.com,smithi156.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-26T04:43:31.862 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi060.front.sepia.ceph.com'), Remote(name='ubuntu@smithi156.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2024-02-26T04:43:31.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-02-26T04:43:31.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T04:43:31.959 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-02-26T04:43:32.035 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-02-26T04:43:32.035 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-02-26T04:43:32.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T04:43:32.141 DEBUG:teuthology.orchestra.run.smithi156:> true 2024-02-26T04:43:32.215 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-02-26T04:43:32.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-26T04:43:32.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T04:43:32.302 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-02-26T04:43:32.371 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-26T04:43:32.371 INFO:teuthology.run_tasks:Running task clock... 2024-02-26T04:43:32.395 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-26T04:43:32.395 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-26T04:43:32.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T04:43:32.398 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-26T04:43:32.398 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T04:43:32.401 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-26T04:43:32.401 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T04:43:32.429 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-26T04:43:32.432 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-26T04:43:32.442 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-26T04:43:32.447 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-26T04:43:32.450 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-26T04:43:32.458 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-26T04:43:32.477 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: ntpd: command not found 2024-02-26T04:43:32.480 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2024-02-26T04:43:32.487 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: ntpd: command not found 2024-02-26T04:43:32.491 INFO:teuthology.orchestra.run.smithi060.stdout:506 Cannot talk to daemon 2024-02-26T04:43:32.494 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2024-02-26T04:43:32.501 INFO:teuthology.orchestra.run.smithi191.stdout:506 Cannot talk to daemon 2024-02-26T04:43:32.508 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-26T04:43:32.511 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-26T04:43:32.517 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-26T04:43:32.525 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-26T04:43:32.528 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-26T04:43:32.533 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-26T04:43:32.572 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: ntpq: command not found 2024-02-26T04:43:32.575 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T04:43:32.576 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2024-02-26T04:43:32.576 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T04:43:32.576 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T04:43:32.576 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T04:43:32.576 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T04:43:32.576 INFO:teuthology.orchestra.run.smithi191.stderr:bash: line 1: ntpq: command not found 2024-02-26T04:43:32.577 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-02-26T04:43:32.579 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T04:43:32.579 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-02-26T04:43:32.579 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T04:43:32.579 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T04:43:32.580 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T04:43:32.580 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T04:43:32.581 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T04:43:32.581 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-02-26T04:43:32.581 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T04:43:32.581 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T04:43:32.582 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T04:43:32.582 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T04:43:32.582 INFO:teuthology.run_tasks:Running task install... 2024-02-26T04:43:32.590 DEBUG:teuthology.task.install:project ceph 2024-02-26T04:43:32.590 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33'}} 2024-02-26T04:43:32.590 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33'} 2024-02-26T04:43:32.590 INFO:teuthology.task.install:Using flavor: default 2024-02-26T04:43:32.602 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-26T04:43:32.603 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-02-26T04:43:32.603 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'tag': None, 'wait_for_package': False} 2024-02-26T04:43:32.603 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T04:43:32.606 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'tag': None, 'wait_for_package': False} 2024-02-26T04:43:32.606 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T04:43:32.607 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'tag': None, 'wait_for_package': False} 2024-02-26T04:43:32.607 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T04:43:32.755 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/ 2024-02-26T04:43:32.755 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1514.g25296a8f 2024-02-26T04:43:32.768 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/ 2024-02-26T04:43:32.768 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1514.g25296a8f 2024-02-26T04:43:32.890 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-26T04:43:32.891 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-26T04:43:32.891 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-26T04:43:32.898 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/ 2024-02-26T04:43:32.898 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1514.g25296a8f 2024-02-26T04:43:32.918 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-26T04:43:32.919 DEBUG:teuthology.orchestra.run.smithi191:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/25296a8f176256aea5ed8b5850b25167dce0ae33/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-26T04:43:32.987 DEBUG:teuthology.orchestra.run.smithi191:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-26T04:43:33.010 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-26T04:43:33.011 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-26T04:43:33.011 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-26T04:43:33.040 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-26T04:43:33.041 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/25296a8f176256aea5ed8b5850b25167dce0ae33/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-26T04:43:33.066 DEBUG:teuthology.orchestra.run.smithi191:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-26T04:43:33.109 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-26T04:43:33.133 INFO:teuthology.orchestra.run.smithi191.stdout:check_obsoletes = 1 2024-02-26T04:43:33.134 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum clean all 2024-02-26T04:43:33.137 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-26T04:43:33.137 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:43:33.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-26T04:43:33.166 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-26T04:43:33.166 DEBUG:teuthology.orchestra.run.smithi060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/25296a8f176256aea5ed8b5850b25167dce0ae33/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-26T04:43:33.188 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-26T04:43:33.236 DEBUG:teuthology.orchestra.run.smithi060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-26T04:43:33.255 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2024-02-26T04:43:33.258 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2024-02-26T04:43:33.317 DEBUG:teuthology.orchestra.run.smithi060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-26T04:43:33.369 INFO:teuthology.orchestra.run.smithi191.stdout:74 files removed 2024-02-26T04:43:33.384 INFO:teuthology.orchestra.run.smithi060.stdout:check_obsoletes = 1 2024-02-26T04:43:33.386 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum clean all 2024-02-26T04:43:33.397 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-26T04:43:33.494 INFO:teuthology.orchestra.run.smithi156.stdout:74 files removed 2024-02-26T04:43:33.522 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-26T04:43:33.633 INFO:teuthology.orchestra.run.smithi060.stdout:74 files removed 2024-02-26T04:43:33.660 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-26T04:43:34.303 INFO:teuthology.orchestra.run.smithi191.stdout:ceph packages for x86_64 133 kB/s | 82 kB 00:00 2024-02-26T04:43:34.524 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 115 kB/s | 82 kB 00:00 2024-02-26T04:43:34.654 INFO:teuthology.orchestra.run.smithi060.stdout:ceph packages for x86_64 122 kB/s | 82 kB 00:00 2024-02-26T04:43:34.712 INFO:teuthology.orchestra.run.smithi191.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-02-26T04:43:34.841 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 36 kB/s | 10 kB 00:00 2024-02-26T04:43:35.146 INFO:teuthology.orchestra.run.smithi191.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-02-26T04:43:35.196 INFO:teuthology.orchestra.run.smithi060.stdout:ceph noarch packages 20 kB/s | 10 kB 00:00 2024-02-26T04:43:35.275 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-02-26T04:43:35.813 INFO:teuthology.orchestra.run.smithi060.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2024-02-26T04:43:35.904 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-9 - AppStream 11 MB/s | 8.1 MB 00:00 2024-02-26T04:43:36.137 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS-9 - AppStream 9.8 MB/s | 8.1 MB 00:00 2024-02-26T04:43:36.574 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - AppStream 11 MB/s | 8.1 MB 00:00 2024-02-26T04:43:37.776 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.5 MB 00:00 2024-02-26T04:43:37.987 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.5 MB 00:00 2024-02-26T04:43:38.391 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.5 MB 00:00 2024-02-26T04:43:38.902 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-02-26T04:43:39.625 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.0 MB 00:01 2024-02-26T04:43:41.681 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.0 MB 00:02 2024-02-26T04:43:46.326 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - AppStream 3.4 MB/s | 19 MB 00:05 2024-02-26T04:43:47.032 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 19 MB 00:04 2024-02-26T04:43:53.275 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.1 MB 00:02 2024-02-26T04:43:56.125 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 9.3 kB/s | 15 kB 00:01 2024-02-26T04:43:58.043 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-02-26T04:44:02.317 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - CRB 502 kB/s | 6.1 MB 00:12 2024-02-26T04:44:03.327 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-02-26T04:44:04.026 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 15 kB 00:00 2024-02-26T04:44:06.045 INFO:teuthology.orchestra.run.smithi060.stderr:Modular dependency problems: 2024-02-26T04:44:06.045 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-02-26T04:44:06.045 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 1: conflicting requests 2024-02-26T04:44:06.045 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.045 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 2: conflicting requests 2024-02-26T04:44:06.045 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.045 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 3: conflicting requests 2024-02-26T04:44:06.045 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.046 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 4: conflicting requests 2024-02-26T04:44:06.046 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.046 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 5: conflicting requests 2024-02-26T04:44:06.046 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.046 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 6: conflicting requests 2024-02-26T04:44:06.046 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.046 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 7: conflicting requests 2024-02-26T04:44:06.046 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.046 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 8: conflicting requests 2024-02-26T04:44:06.046 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.046 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 9: conflicting requests 2024-02-26T04:44:06.046 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.047 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 10: conflicting requests 2024-02-26T04:44:06.047 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.047 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 11: conflicting requests 2024-02-26T04:44:06.047 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.047 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 12: conflicting requests 2024-02-26T04:44:06.047 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.047 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 13: conflicting requests 2024-02-26T04:44:06.047 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.047 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 14: conflicting requests 2024-02-26T04:44:06.047 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.047 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 15: conflicting requests 2024-02-26T04:44:06.047 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.047 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 16: conflicting requests 2024-02-26T04:44:06.048 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.048 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 17: conflicting requests 2024-02-26T04:44:06.048 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.048 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 18: conflicting requests 2024-02-26T04:44:06.048 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.048 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 19: conflicting requests 2024-02-26T04:44:06.048 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.048 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 20: conflicting requests 2024-02-26T04:44:06.048 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.048 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 21: conflicting requests 2024-02-26T04:44:06.048 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.048 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 22: conflicting requests 2024-02-26T04:44:06.049 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.049 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 23: conflicting requests 2024-02-26T04:44:06.049 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.049 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 24: conflicting requests 2024-02-26T04:44:06.049 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.049 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 25: conflicting requests 2024-02-26T04:44:06.049 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.049 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 26: conflicting requests 2024-02-26T04:44:06.049 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.049 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 27: conflicting requests 2024-02-26T04:44:06.049 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.049 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 28: conflicting requests 2024-02-26T04:44:06.050 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.050 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 29: conflicting requests 2024-02-26T04:44:06.050 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.050 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 30: conflicting requests 2024-02-26T04:44:06.050 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.050 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 31: conflicting requests 2024-02-26T04:44:06.050 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.050 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 32: conflicting requests 2024-02-26T04:44:06.050 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.050 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 33: conflicting requests 2024-02-26T04:44:06.050 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.050 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 34: conflicting requests 2024-02-26T04:44:06.051 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.051 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 35: conflicting requests 2024-02-26T04:44:06.051 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.051 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 36: conflicting requests 2024-02-26T04:44:06.051 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.051 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.051 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.051 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T04:44:06.051 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.051 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.051 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.051 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.051 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 38: conflicting requests 2024-02-26T04:44:06.052 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.052 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.052 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.052 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.052 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.052 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.052 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T04:44:06.052 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.052 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.052 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.052 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T04:44:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T04:44:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T04:44:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.053 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T04:44:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T04:44:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.054 INFO:teuthology.orchestra.run.smithi060.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T04:44:06.541 INFO:teuthology.orchestra.run.smithi060.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T04:44:06.542 INFO:teuthology.orchestra.run.smithi060.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T04:44:06.555 INFO:teuthology.orchestra.run.smithi060.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-26T04:44:06.556 INFO:teuthology.orchestra.run.smithi060.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-26T04:44:06.557 INFO:teuthology.orchestra.run.smithi060.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-26T04:44:06.559 INFO:teuthology.orchestra.run.smithi060.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-26T04:44:06.560 INFO:teuthology.orchestra.run.smithi060.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-26T04:44:06.563 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-26T04:44:06.564 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-26T04:44:06.663 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-02-26T04:44:06.675 INFO:teuthology.orchestra.run.smithi060.stdout:========================================================================================= 2024-02-26T04:44:06.675 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-02-26T04:44:06.675 INFO:teuthology.orchestra.run.smithi060.stdout:========================================================================================= 2024-02-26T04:44:06.675 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-02-26T04:44:06.675 INFO:teuthology.orchestra.run.smithi060.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-26T04:44:06.675 INFO:teuthology.orchestra.run.smithi060.stdout: ceph x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 6.6 k 2024-02-26T04:44:06.675 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.3 M 2024-02-26T04:44:06.676 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 886 k 2024-02-26T04:44:06.676 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 145 k 2024-02-26T04:44:06.676 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 1.5 M 2024-02-26T04:44:06.676 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 138 k 2024-02-26T04:44:06.676 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 1.8 M 2024-02-26T04:44:06.676 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 7.4 M 2024-02-26T04:44:06.676 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 52 k 2024-02-26T04:44:06.676 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 9.7 M 2024-02-26T04:44:06.676 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 47 M 2024-02-26T04:44:06.676 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 264 k 2024-02-26T04:44:06.676 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 748 k 2024-02-26T04:44:06.676 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 217 k 2024-02-26T04:44:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-shell x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 27 k 2024-02-26T04:44:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 18 k 2024-02-26T04:44:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-26T04:44:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-26T04:44:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-26T04:44:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 31 k 2024-02-26T04:44:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 743 k 2024-02-26T04:44:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 127 k 2024-02-26T04:44:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-26T04:44:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-26T04:44:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-26T04:44:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-26T04:44:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-26T04:44:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-26T04:44:06.678 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 161 k 2024-02-26T04:44:06.678 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 320 k 2024-02-26T04:44:06.678 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 298 k 2024-02-26T04:44:06.678 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 99 k 2024-02-26T04:44:06.678 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 85 k 2024-02-26T04:44:06.678 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T04:44:06.678 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 171 k 2024-02-26T04:44:06.678 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-26T04:44:06.678 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2024-02-26T04:44:06.678 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-26T04:44:06.678 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-26T04:44:06.678 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-26T04:44:06.679 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-26T04:44:06.679 INFO:teuthology.orchestra.run.smithi060.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-26T04:44:06.679 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-26T04:44:06.679 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-26T04:44:06.679 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-26T04:44:06.679 INFO:teuthology.orchestra.run.smithi060.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-26T04:44:06.679 INFO:teuthology.orchestra.run.smithi060.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-26T04:44:06.679 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.4 M 2024-02-26T04:44:06.679 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T04:44:06.679 INFO:teuthology.orchestra.run.smithi060.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-26T04:44:06.679 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-26T04:44:06.679 INFO:teuthology.orchestra.run.smithi060.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-26T04:44:06.679 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-26T04:44:06.680 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-02-26T04:44:06.680 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-26T04:44:06.680 INFO:teuthology.orchestra.run.smithi060.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-26T04:44:06.680 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-26T04:44:06.680 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 20 M 2024-02-26T04:44:06.680 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 31 k 2024-02-26T04:44:06.680 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 2.1 M 2024-02-26T04:44:06.680 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 247 k 2024-02-26T04:44:06.680 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 4.7 M 2024-02-26T04:44:06.680 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 17 M 2024-02-26T04:44:06.680 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 15 k 2024-02-26T04:44:06.680 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 25 k 2024-02-26T04:44:06.681 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-26T04:44:06.681 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-26T04:44:06.681 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-26T04:44:06.681 INFO:teuthology.orchestra.run.smithi060.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-26T04:44:06.681 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-26T04:44:06.681 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-26T04:44:06.681 INFO:teuthology.orchestra.run.smithi060.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-26T04:44:06.681 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-26T04:44:06.681 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-26T04:44:06.681 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 163 k 2024-02-26T04:44:06.681 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-26T04:44:06.681 INFO:teuthology.orchestra.run.smithi060.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-26T04:44:06.681 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-26T04:44:06.682 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-26T04:44:06.682 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-26T04:44:06.682 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 498 k 2024-02-26T04:44:06.682 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-26T04:44:06.682 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.1 M 2024-02-26T04:44:06.682 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-26T04:44:06.682 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-26T04:44:06.682 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-26T04:44:06.682 INFO:teuthology.orchestra.run.smithi060.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-26T04:44:06.682 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-26T04:44:06.682 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-26T04:44:06.682 INFO:teuthology.orchestra.run.smithi060.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-26T04:44:06.682 INFO:teuthology.orchestra.run.smithi060.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-26T04:44:06.683 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-26T04:44:06.683 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-26T04:44:06.683 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-26T04:44:06.683 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-26T04:44:06.683 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-26T04:44:06.683 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-26T04:44:06.683 INFO:teuthology.orchestra.run.smithi060.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-26T04:44:06.683 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-26T04:44:06.683 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-26T04:44:06.683 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-26T04:44:06.683 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 45 k 2024-02-26T04:44:06.683 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 131 k 2024-02-26T04:44:06.683 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-26T04:44:06.683 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-26T04:44:06.684 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-26T04:44:06.684 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-26T04:44:06.684 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-26T04:44:06.684 INFO:teuthology.orchestra.run.smithi060.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-26T04:44:06.684 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-26T04:44:06.684 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-26T04:44:06.684 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-26T04:44:06.684 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-26T04:44:06.684 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-26T04:44:06.684 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-26T04:44:06.684 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-26T04:44:06.684 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-26T04:44:06.684 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-26T04:44:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-26T04:44:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-26T04:44:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-26T04:44:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-26T04:44:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-26T04:44:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-26T04:44:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-26T04:44:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-26T04:44:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-26T04:44:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-26T04:44:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-26T04:44:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-26T04:44:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-26T04:44:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-26T04:44:06.686 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-26T04:44:06.686 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-26T04:44:06.686 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-26T04:44:06.686 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-26T04:44:06.686 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-26T04:44:06.686 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-26T04:44:06.686 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-26T04:44:06.686 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-26T04:44:06.686 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-26T04:44:06.686 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-26T04:44:06.686 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-26T04:44:06.686 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-26T04:44:06.686 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-26T04:44:06.687 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-26T04:44:06.688 INFO:teuthology.orchestra.run.smithi060.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-26T04:44:06.688 INFO:teuthology.orchestra.run.smithi060.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-26T04:44:06.688 INFO:teuthology.orchestra.run.smithi060.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-26T04:44:06.688 INFO:teuthology.orchestra.run.smithi060.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-26T04:44:06.688 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-26T04:44:06.688 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-26T04:44:06.688 INFO:teuthology.orchestra.run.smithi060.stdout:Installing weak dependencies: 2024-02-26T04:44:06.688 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-26T04:44:06.688 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:44:06.688 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-02-26T04:44:06.688 INFO:teuthology.orchestra.run.smithi060.stdout:========================================================================================= 2024-02-26T04:44:06.688 INFO:teuthology.orchestra.run.smithi060.stdout:Install 150 Packages 2024-02-26T04:44:06.688 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 16 Packages 2024-02-26T04:44:06.688 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:44:06.689 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 221 M 2024-02-26T04:44:06.689 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-02-26T04:44:08.036 INFO:teuthology.orchestra.run.smithi060.stdout:(1/166): ceph-19.0.0-1514.g25296a8f.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-02-26T04:44:08.662 INFO:teuthology.orchestra.run.smithi060.stdout:(2/166): ceph-fuse-19.0.0-1514.g25296a8f.el9.x8 1.4 MB/s | 886 kB 00:00 2024-02-26T04:44:08.803 INFO:teuthology.orchestra.run.smithi060.stdout:(3/166): ceph-immutable-object-cache-19.0.0-151 1.0 MB/s | 145 kB 00:00 2024-02-26T04:44:09.366 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - AppStream 667 kB/s | 19 MB 00:29 2024-02-26T04:44:09.520 INFO:teuthology.orchestra.run.smithi060.stdout:(4/166): ceph-mds-19.0.0-1514.g25296a8f.el9.x86 3.0 MB/s | 2.1 MB 00:00 2024-02-26T04:44:09.820 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 20 MB 00:05 2024-02-26T04:44:10.012 INFO:teuthology.orchestra.run.smithi060.stdout:(5/166): ceph-mgr-19.0.0-1514.g25296a8f.el9.x86 3.0 MB/s | 1.5 MB 00:00 2024-02-26T04:44:10.488 INFO:teuthology.orchestra.run.smithi060.stdout:(6/166): ceph-base-19.0.0-1514.g25296a8f.el9.x8 1.9 MB/s | 5.3 MB 00:02 2024-02-26T04:44:10.996 INFO:teuthology.orchestra.run.smithi060.stdout:(7/166): ceph-mon-19.0.0-1514.g25296a8f.el9.x86 4.8 MB/s | 4.7 MB 00:00 2024-02-26T04:44:12.571 INFO:teuthology.orchestra.run.smithi060.stdout:(8/166): ceph-radosgw-19.0.0-1514.g25296a8f.el9 6.2 MB/s | 9.7 MB 00:01 2024-02-26T04:44:12.671 INFO:teuthology.orchestra.run.smithi060.stdout:(9/166): ceph-selinux-19.0.0-1514.g25296a8f.el9 252 kB/s | 25 kB 00:00 2024-02-26T04:44:14.566 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.1 MB 00:01 2024-02-26T04:44:14.794 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-02-26T04:44:15.622 INFO:teuthology.orchestra.run.smithi060.stdout:(10/166): ceph-common-19.0.0-1514.g25296a8f.el9 2.5 MB/s | 20 MB 00:07 2024-02-26T04:44:15.755 INFO:teuthology.orchestra.run.smithi060.stdout:(11/166): cephfs-mirror-19.0.0-1514.g25296a8f.e 1.6 MB/s | 217 kB 00:00 2024-02-26T04:44:15.839 INFO:teuthology.orchestra.run.smithi060.stdout:(12/166): cephfs-shell-19.0.0-1514.g25296a8f.el 325 kB/s | 27 kB 00:00 2024-02-26T04:44:16.047 INFO:teuthology.orchestra.run.smithi060.stdout:(13/166): ceph-osd-19.0.0-1514.g25296a8f.el9.x8 3.0 MB/s | 17 MB 00:05 2024-02-26T04:44:16.073 INFO:teuthology.orchestra.run.smithi060.stdout:(14/166): libcephfs-devel-19.0.0-1514.g25296a8f 133 kB/s | 31 kB 00:00 2024-02-26T04:44:16.256 INFO:teuthology.orchestra.run.smithi060.stdout:(15/166): libcephsqlite-19.0.0-1514.g25296a8f.e 892 kB/s | 163 kB 00:00 2024-02-26T04:44:16.259 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 15 kB 00:00 2024-02-26T04:44:16.281 INFO:teuthology.orchestra.run.smithi060.stdout:(16/166): libcephfs2-19.0.0-1514.g25296a8f.el9. 3.1 MB/s | 743 kB 00:00 2024-02-26T04:44:16.332 INFO:teuthology.orchestra.run.smithi060.stdout:(17/166): librados-devel-19.0.0-1514.g25296a8f. 1.6 MB/s | 127 kB 00:00 2024-02-26T04:44:16.423 INFO:teuthology.orchestra.run.smithi060.stdout:(18/166): libradosstriper1-19.0.0-1514.g25296a8 3.4 MB/s | 498 kB 00:00 2024-02-26T04:44:16.499 INFO:teuthology.orchestra.run.smithi060.stdout:(19/166): python3-ceph-argparse-19.0.0-1514.g25 602 kB/s | 45 kB 00:00 2024-02-26T04:44:16.674 INFO:teuthology.orchestra.run.smithi060.stdout:(20/166): python3-ceph-common-19.0.0-1514.g2529 750 kB/s | 131 kB 00:00 2024-02-26T04:44:16.808 INFO:teuthology.orchestra.run.smithi060.stdout:(21/166): python3-cephfs-19.0.0-1514.g25296a8f. 1.2 MB/s | 161 kB 00:00 2024-02-26T04:44:16.941 INFO:teuthology.orchestra.run.smithi060.stdout:(22/166): python3-rados-19.0.0-1514.g25296a8f.e 2.3 MB/s | 320 kB 00:00 2024-02-26T04:44:17.092 INFO:teuthology.orchestra.run.smithi060.stdout:(23/166): python3-rbd-19.0.0-1514.g25296a8f.el9 1.9 MB/s | 298 kB 00:00 2024-02-26T04:44:17.384 INFO:teuthology.orchestra.run.smithi191.stderr:Modular dependency problems: 2024-02-26T04:44:17.384 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-02-26T04:44:17.384 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 1: conflicting requests 2024-02-26T04:44:17.384 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.385 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 2: conflicting requests 2024-02-26T04:44:17.385 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.385 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 3: conflicting requests 2024-02-26T04:44:17.385 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.385 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 4: conflicting requests 2024-02-26T04:44:17.385 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.385 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 5: conflicting requests 2024-02-26T04:44:17.385 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.385 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 6: conflicting requests 2024-02-26T04:44:17.385 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.385 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 7: conflicting requests 2024-02-26T04:44:17.385 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.385 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 8: conflicting requests 2024-02-26T04:44:17.386 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.386 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 9: conflicting requests 2024-02-26T04:44:17.386 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.386 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 10: conflicting requests 2024-02-26T04:44:17.386 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.386 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 11: conflicting requests 2024-02-26T04:44:17.386 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.386 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 12: conflicting requests 2024-02-26T04:44:17.386 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.386 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 13: conflicting requests 2024-02-26T04:44:17.386 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.386 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 14: conflicting requests 2024-02-26T04:44:17.386 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.386 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 15: conflicting requests 2024-02-26T04:44:17.387 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.387 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 16: conflicting requests 2024-02-26T04:44:17.387 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.387 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 17: conflicting requests 2024-02-26T04:44:17.387 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.387 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 18: conflicting requests 2024-02-26T04:44:17.387 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.387 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 19: conflicting requests 2024-02-26T04:44:17.387 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.387 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 20: conflicting requests 2024-02-26T04:44:17.387 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.387 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 21: conflicting requests 2024-02-26T04:44:17.387 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.387 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 22: conflicting requests 2024-02-26T04:44:17.388 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.388 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 23: conflicting requests 2024-02-26T04:44:17.388 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.388 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 24: conflicting requests 2024-02-26T04:44:17.388 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.388 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 25: conflicting requests 2024-02-26T04:44:17.388 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.388 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 26: conflicting requests 2024-02-26T04:44:17.388 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.388 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 27: conflicting requests 2024-02-26T04:44:17.388 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.389 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 28: conflicting requests 2024-02-26T04:44:17.389 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.389 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 29: conflicting requests 2024-02-26T04:44:17.389 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.389 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 30: conflicting requests 2024-02-26T04:44:17.389 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.389 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 31: conflicting requests 2024-02-26T04:44:17.389 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.389 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 32: conflicting requests 2024-02-26T04:44:17.390 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.390 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 33: conflicting requests 2024-02-26T04:44:17.390 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.390 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 34: conflicting requests 2024-02-26T04:44:17.390 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.390 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 35: conflicting requests 2024-02-26T04:44:17.390 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.390 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 36: conflicting requests 2024-02-26T04:44:17.390 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.390 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.390 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.390 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T04:44:17.391 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.391 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.391 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.391 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.391 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 38: conflicting requests 2024-02-26T04:44:17.391 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.391 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.391 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.391 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.392 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.392 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.392 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T04:44:17.392 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.392 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.392 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.392 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.392 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T04:44:17.392 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.392 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.392 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.392 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.392 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T04:44:17.393 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.393 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.393 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.393 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.393 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T04:44:17.393 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.393 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.393 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.393 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.393 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T04:44:17.393 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.393 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.393 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.393 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.394 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T04:44:17.394 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.394 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.394 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.394 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T04:44:17.542 INFO:teuthology.orchestra.run.smithi060.stdout:(24/166): python3-rgw-19.0.0-1514.g25296a8f.el9 221 kB/s | 99 kB 00:00 2024-02-26T04:44:17.651 INFO:teuthology.orchestra.run.smithi060.stdout:(25/166): rbd-fuse-19.0.0-1514.g25296a8f.el9.x8 788 kB/s | 85 kB 00:00 2024-02-26T04:44:17.734 INFO:teuthology.orchestra.run.smithi060.stdout:(26/166): librgw2-19.0.0-1514.g25296a8f.el9.x86 3.6 MB/s | 5.1 MB 00:01 2024-02-26T04:44:17.766 INFO:teuthology.orchestra.run.smithi191.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T04:44:17.767 INFO:teuthology.orchestra.run.smithi191.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T04:44:17.780 INFO:teuthology.orchestra.run.smithi191.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-26T04:44:17.781 INFO:teuthology.orchestra.run.smithi191.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-26T04:44:17.782 INFO:teuthology.orchestra.run.smithi191.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-26T04:44:17.783 INFO:teuthology.orchestra.run.smithi191.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-26T04:44:17.785 INFO:teuthology.orchestra.run.smithi191.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-26T04:44:17.788 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-26T04:44:17.789 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-26T04:44:17.826 INFO:teuthology.orchestra.run.smithi060.stdout:(27/166): rbd-nbd-19.0.0-1514.g25296a8f.el9.x86 1.8 MB/s | 171 kB 00:00 2024-02-26T04:44:17.877 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-26T04:44:17.894 INFO:teuthology.orchestra.run.smithi191.stdout:========================================================================================= 2024-02-26T04:44:17.894 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-02-26T04:44:17.894 INFO:teuthology.orchestra.run.smithi191.stdout:========================================================================================= 2024-02-26T04:44:17.895 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-26T04:44:17.895 INFO:teuthology.orchestra.run.smithi191.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-26T04:44:17.895 INFO:teuthology.orchestra.run.smithi191.stdout: ceph x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 6.6 k 2024-02-26T04:44:17.895 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.3 M 2024-02-26T04:44:17.895 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 886 k 2024-02-26T04:44:17.895 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 145 k 2024-02-26T04:44:17.895 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 1.5 M 2024-02-26T04:44:17.895 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 138 k 2024-02-26T04:44:17.895 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 1.8 M 2024-02-26T04:44:17.895 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 7.4 M 2024-02-26T04:44:17.895 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 52 k 2024-02-26T04:44:17.896 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 9.7 M 2024-02-26T04:44:17.896 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 47 M 2024-02-26T04:44:17.896 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 264 k 2024-02-26T04:44:17.896 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 748 k 2024-02-26T04:44:17.896 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 217 k 2024-02-26T04:44:17.896 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-shell x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 27 k 2024-02-26T04:44:17.896 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 18 k 2024-02-26T04:44:17.896 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-26T04:44:17.896 INFO:teuthology.orchestra.run.smithi191.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-26T04:44:17.896 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-26T04:44:17.896 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 31 k 2024-02-26T04:44:17.897 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 743 k 2024-02-26T04:44:17.897 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 127 k 2024-02-26T04:44:17.897 INFO:teuthology.orchestra.run.smithi191.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-26T04:44:17.897 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-26T04:44:17.897 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-26T04:44:17.897 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-26T04:44:17.897 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-26T04:44:17.897 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-26T04:44:17.897 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 161 k 2024-02-26T04:44:17.897 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 320 k 2024-02-26T04:44:17.898 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 298 k 2024-02-26T04:44:17.898 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 99 k 2024-02-26T04:44:17.898 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 85 k 2024-02-26T04:44:17.898 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T04:44:17.898 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 171 k 2024-02-26T04:44:17.898 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-26T04:44:17.898 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrading: 2024-02-26T04:44:17.898 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-26T04:44:17.898 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-26T04:44:17.898 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-26T04:44:17.898 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-26T04:44:17.899 INFO:teuthology.orchestra.run.smithi191.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-26T04:44:17.899 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-26T04:44:17.899 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-26T04:44:17.899 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-26T04:44:17.899 INFO:teuthology.orchestra.run.smithi191.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-26T04:44:17.899 INFO:teuthology.orchestra.run.smithi191.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-26T04:44:17.899 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.4 M 2024-02-26T04:44:17.899 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T04:44:17.899 INFO:teuthology.orchestra.run.smithi191.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-26T04:44:17.899 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-26T04:44:17.899 INFO:teuthology.orchestra.run.smithi191.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-26T04:44:17.900 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-26T04:44:17.900 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-02-26T04:44:17.900 INFO:teuthology.orchestra.run.smithi191.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-26T04:44:17.900 INFO:teuthology.orchestra.run.smithi191.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-26T04:44:17.900 INFO:teuthology.orchestra.run.smithi191.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-26T04:44:17.900 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 20 M 2024-02-26T04:44:17.900 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 31 k 2024-02-26T04:44:17.900 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 2.1 M 2024-02-26T04:44:17.900 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 247 k 2024-02-26T04:44:17.900 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 4.7 M 2024-02-26T04:44:17.900 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 17 M 2024-02-26T04:44:17.901 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 15 k 2024-02-26T04:44:17.901 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 25 k 2024-02-26T04:44:17.901 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-26T04:44:17.901 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-26T04:44:17.901 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-26T04:44:17.901 INFO:teuthology.orchestra.run.smithi191.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-26T04:44:17.901 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-26T04:44:17.901 INFO:teuthology.orchestra.run.smithi191.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-26T04:44:17.901 INFO:teuthology.orchestra.run.smithi191.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-26T04:44:17.901 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-26T04:44:17.902 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-26T04:44:17.902 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 163 k 2024-02-26T04:44:17.902 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-26T04:44:17.902 INFO:teuthology.orchestra.run.smithi191.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-26T04:44:17.902 INFO:teuthology.orchestra.run.smithi191.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-26T04:44:17.902 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-26T04:44:17.902 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-26T04:44:17.902 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 498 k 2024-02-26T04:44:17.902 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-26T04:44:17.902 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.1 M 2024-02-26T04:44:17.902 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-26T04:44:17.902 INFO:teuthology.orchestra.run.smithi191.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-26T04:44:17.903 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-26T04:44:17.903 INFO:teuthology.orchestra.run.smithi191.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-26T04:44:17.903 INFO:teuthology.orchestra.run.smithi191.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-26T04:44:17.903 INFO:teuthology.orchestra.run.smithi191.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-26T04:44:17.903 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-26T04:44:17.903 INFO:teuthology.orchestra.run.smithi191.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-26T04:44:17.903 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-26T04:44:17.903 INFO:teuthology.orchestra.run.smithi191.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-26T04:44:17.903 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-26T04:44:17.903 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-26T04:44:17.904 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-26T04:44:17.904 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-26T04:44:17.904 INFO:teuthology.orchestra.run.smithi191.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-26T04:44:17.904 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-26T04:44:17.904 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-26T04:44:17.904 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-26T04:44:17.904 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 45 k 2024-02-26T04:44:17.904 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 131 k 2024-02-26T04:44:17.904 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-26T04:44:17.904 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-26T04:44:17.904 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-26T04:44:17.906 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-26T04:44:17.906 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-26T04:44:17.906 INFO:teuthology.orchestra.run.smithi191.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-26T04:44:17.906 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-26T04:44:17.906 INFO:teuthology.orchestra.run.smithi191.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-26T04:44:17.906 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-26T04:44:17.906 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-26T04:44:17.906 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-26T04:44:17.906 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-26T04:44:17.906 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-26T04:44:17.907 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-26T04:44:17.907 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-26T04:44:17.907 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-26T04:44:17.907 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-26T04:44:17.907 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-26T04:44:17.907 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-26T04:44:17.907 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-26T04:44:17.907 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-26T04:44:17.907 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-26T04:44:17.907 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-26T04:44:17.907 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-26T04:44:17.907 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-26T04:44:17.907 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-26T04:44:17.908 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-26T04:44:17.908 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-26T04:44:17.908 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-26T04:44:17.908 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-26T04:44:17.908 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-26T04:44:17.908 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-26T04:44:17.908 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-26T04:44:17.908 INFO:teuthology.orchestra.run.smithi191.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-26T04:44:17.908 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-26T04:44:17.908 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-26T04:44:17.908 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-26T04:44:17.908 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-26T04:44:17.909 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-26T04:44:17.909 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-26T04:44:17.909 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-26T04:44:17.909 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-26T04:44:17.909 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-26T04:44:17.909 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-26T04:44:17.909 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-26T04:44:17.909 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-26T04:44:17.909 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-26T04:44:17.909 INFO:teuthology.orchestra.run.smithi191.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-26T04:44:17.909 INFO:teuthology.orchestra.run.smithi191.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-26T04:44:17.909 INFO:teuthology.orchestra.run.smithi191.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-26T04:44:17.909 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-26T04:44:17.909 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout:Installing weak dependencies: 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:44:17.910 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-26T04:44:17.911 INFO:teuthology.orchestra.run.smithi191.stdout:========================================================================================= 2024-02-26T04:44:17.911 INFO:teuthology.orchestra.run.smithi191.stdout:Install 150 Packages 2024-02-26T04:44:17.911 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrade 16 Packages 2024-02-26T04:44:17.911 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:44:17.911 INFO:teuthology.orchestra.run.smithi060.stdout:(28/166): ceph-grafana-dashboards-19.0.0-1514.g 378 kB/s | 31 kB 00:00 2024-02-26T04:44:17.912 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 221 M 2024-02-26T04:44:17.912 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-26T04:44:18.019 INFO:teuthology.orchestra.run.smithi060.stdout:(29/166): ceph-mgr-cephadm-19.0.0-1514.g25296a8 1.2 MB/s | 138 kB 00:00 2024-02-26T04:44:18.336 INFO:teuthology.orchestra.run.smithi060.stdout:(30/166): rbd-mirror-19.0.0-1514.g25296a8f.el9. 4.5 MB/s | 3.1 MB 00:00 2024-02-26T04:44:18.372 INFO:teuthology.orchestra.run.smithi191.stdout:(1/166): ceph-19.0.0-1514.g25296a8f.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-02-26T04:44:18.461 INFO:teuthology.orchestra.run.smithi060.stdout:(31/166): ceph-mgr-dashboard-19.0.0-1514.g25296 4.0 MB/s | 1.8 MB 00:00 2024-02-26T04:44:18.560 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 9.5 MB/s | 20 MB 00:02 2024-02-26T04:44:18.603 INFO:teuthology.orchestra.run.smithi060.stdout:(32/166): ceph-mgr-modules-core-19.0.0-1514.g25 1.7 MB/s | 247 kB 00:00 2024-02-26T04:44:18.681 INFO:teuthology.orchestra.run.smithi191.stdout:(2/166): ceph-fuse-19.0.0-1514.g25296a8f.el9.x8 2.8 MB/s | 886 kB 00:00 2024-02-26T04:44:18.728 INFO:teuthology.orchestra.run.smithi060.stdout:(33/166): ceph-mgr-rook-19.0.0-1514.g25296a8f.e 417 kB/s | 52 kB 00:00 2024-02-26T04:44:18.748 INFO:teuthology.orchestra.run.smithi191.stdout:(3/166): ceph-immutable-object-cache-19.0.0-151 2.1 MB/s | 145 kB 00:00 2024-02-26T04:44:18.820 INFO:teuthology.orchestra.run.smithi060.stdout:(34/166): ceph-prometheus-alerts-19.0.0-1514.g2 167 kB/s | 15 kB 00:00 2024-02-26T04:44:18.937 INFO:teuthology.orchestra.run.smithi060.stdout:(35/166): ceph-volume-19.0.0-1514.g25296a8f.el9 2.2 MB/s | 264 kB 00:00 2024-02-26T04:44:19.187 INFO:teuthology.orchestra.run.smithi060.stdout:(36/166): cephadm-19.0.0-1514.g25296a8f.el9.noa 2.9 MB/s | 748 kB 00:00 2024-02-26T04:44:19.256 INFO:teuthology.orchestra.run.smithi191.stdout:(4/166): ceph-mds-19.0.0-1514.g25296a8f.el9.x86 4.2 MB/s | 2.1 MB 00:00 2024-02-26T04:44:19.271 INFO:teuthology.orchestra.run.smithi060.stdout:(37/166): cephfs-top-19.0.0-1514.g25296a8f.el9. 219 kB/s | 18 kB 00:00 2024-02-26T04:44:19.455 INFO:teuthology.orchestra.run.smithi060.stdout:(38/166): libaio-devel-0.3.112-1.el8.x86_64.rpm 96 kB/s | 18 kB 00:00 2024-02-26T04:44:19.523 INFO:teuthology.orchestra.run.smithi191.stdout:(5/166): ceph-mgr-19.0.0-1514.g25296a8f.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-02-26T04:44:19.572 INFO:teuthology.orchestra.run.smithi060.stdout:(39/166): fuse-2.9.9-15.el9.x86_64.rpm 688 kB/s | 80 kB 00:00 2024-02-26T04:44:19.638 INFO:teuthology.orchestra.run.smithi060.stdout:(40/166): jq-1.6-16.el9.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2024-02-26T04:44:19.722 INFO:teuthology.orchestra.run.smithi060.stdout:(41/166): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2024-02-26T04:44:19.772 INFO:teuthology.orchestra.run.smithi060.stdout:(42/166): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-02-26T04:44:19.815 INFO:teuthology.orchestra.run.smithi060.stdout:(43/166): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 218 kB 00:00 2024-02-26T04:44:19.865 INFO:teuthology.orchestra.run.smithi060.stdout:(44/166): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-02-26T04:44:19.940 INFO:teuthology.orchestra.run.smithi060.stdout:(45/166): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-02-26T04:44:19.982 INFO:teuthology.orchestra.run.smithi060.stdout:(46/166): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-02-26T04:44:20.083 INFO:teuthology.orchestra.run.smithi060.stdout:(47/166): ceph-mgr-diskprediction-local-19.0.0- 4.2 MB/s | 7.4 MB 00:01 2024-02-26T04:44:20.107 INFO:teuthology.orchestra.run.smithi191.stdout:(6/166): ceph-mon-19.0.0-1514.g25296a8f.el9.x86 8.0 MB/s | 4.7 MB 00:00 2024-02-26T04:44:20.108 INFO:teuthology.orchestra.run.smithi060.stdout:(48/166): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-02-26T04:44:20.158 INFO:teuthology.orchestra.run.smithi060.stdout:(49/166): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-02-26T04:44:20.208 INFO:teuthology.orchestra.run.smithi060.stdout:(50/166): python3-requests-2.25.1-8.el9.noarch. 997 kB/s | 125 kB 00:00 2024-02-26T04:44:20.232 INFO:teuthology.orchestra.run.smithi191.stdout:(7/166): ceph-base-19.0.0-1514.g25296a8f.el9.x8 2.6 MB/s | 5.3 MB 00:02 2024-02-26T04:44:20.284 INFO:teuthology.orchestra.run.smithi060.stdout:(51/166): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-02-26T04:44:20.367 INFO:teuthology.orchestra.run.smithi060.stdout:(52/166): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-02-26T04:44:20.392 INFO:teuthology.orchestra.run.smithi060.stdout:(53/166): boost-program-options-1.75.0-8.el9.x8 986 kB/s | 107 kB 00:00 2024-02-26T04:44:20.426 INFO:teuthology.orchestra.run.smithi060.stdout:(54/166): elfutils-libelf-devel-0.190-2.el9.x86 367 kB/s | 21 kB 00:00 2024-02-26T04:44:20.485 INFO:teuthology.orchestra.run.smithi060.stdout:(55/166): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-02-26T04:44:20.502 INFO:teuthology.orchestra.run.smithi060.stdout:(56/166): flexiblas-3.0.4-8.el9.x86_64.rpm 435 kB/s | 33 kB 00:00 2024-02-26T04:44:20.552 INFO:teuthology.orchestra.run.smithi060.stdout:(57/166): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-02-26T04:44:20.644 INFO:teuthology.orchestra.run.smithi060.stdout:(58/166): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-02-26T04:44:20.753 INFO:teuthology.orchestra.run.smithi060.stdout:(59/166): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-02-26T04:44:20.778 INFO:teuthology.orchestra.run.smithi060.stdout:(60/166): librabbitmq-0.11.0-7.el9.x86_64.rpm 338 kB/s | 45 kB 00:00 2024-02-26T04:44:20.878 INFO:teuthology.orchestra.run.smithi060.stdout:(61/166): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-02-26T04:44:20.920 INFO:teuthology.orchestra.run.smithi060.stdout:(62/166): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-02-26T04:44:20.954 INFO:teuthology.orchestra.run.smithi060.stdout:(63/166): libuuid-devel-2.37.4-15.el9.x86_64.rp 340 kB/s | 26 kB 00:00 2024-02-26T04:44:21.004 INFO:teuthology.orchestra.run.smithi060.stdout:(64/166): libzstd-devel-1.5.1-2.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-02-26T04:44:21.063 INFO:teuthology.orchestra.run.smithi060.stdout:(65/166): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-02-26T04:44:21.088 INFO:teuthology.orchestra.run.smithi060.stdout:(66/166): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-02-26T04:44:21.163 INFO:teuthology.orchestra.run.smithi060.stdout:(67/166): openblas-0.3.21-2.el9.x86_64.rpm 352 kB/s | 35 kB 00:00 2024-02-26T04:44:21.391 INFO:teuthology.orchestra.run.smithi191.stdout:(8/166): ceph-osd-19.0.0-1514.g25296a8f.el9.x86 13 MB/s | 17 MB 00:01 2024-02-26T04:44:21.433 INFO:teuthology.orchestra.run.smithi191.stdout:(9/166): ceph-selinux-19.0.0-1514.g25296a8f.el9 609 kB/s | 25 kB 00:00 2024-02-26T04:44:21.505 INFO:teuthology.orchestra.run.smithi060.stdout:(68/166): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-02-26T04:44:21.572 INFO:teuthology.orchestra.run.smithi060.stdout:(69/166): openssl-devel-3.0.7-25.el9.x86_64.rpm 10 MB/s | 4.1 MB 00:00 2024-02-26T04:44:21.722 INFO:teuthology.orchestra.run.smithi060.stdout:(70/166): postgresql-contrib-13.11-1.el9.x86_64 5.9 MB/s | 905 kB 00:00 2024-02-26T04:44:21.756 INFO:teuthology.orchestra.run.smithi060.stdout:(71/166): postgresql-13.11-1.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-02-26T04:44:21.856 INFO:teuthology.orchestra.run.smithi060.stdout:(72/166): postgresql-private-libs-13.11-1.el9.x 1.0 MB/s | 135 kB 00:00 2024-02-26T04:44:22.017 INFO:teuthology.orchestra.run.smithi060.stdout:(73/166): protobuf-3.14.0-13.el9.x86_64.rpm 6.3 MB/s | 1.0 MB 00:00 2024-02-26T04:44:22.140 INFO:teuthology.orchestra.run.smithi060.stdout:(74/166): postgresql-server-13.11-1.el9.x86_64. 15 MB/s | 5.7 MB 00:00 2024-02-26T04:44:22.165 INFO:teuthology.orchestra.run.smithi060.stdout:(75/166): python3-attrs-20.3.0-7.el9.noarch.rpm 591 kB/s | 87 kB 00:00 2024-02-26T04:44:22.399 INFO:teuthology.orchestra.run.smithi060.stdout:(76/166): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-02-26T04:44:22.449 INFO:teuthology.orchestra.run.smithi060.stdout:(77/166): python3-devel-3.9.17-2.el9.x86_64.rpm 865 kB/s | 245 kB 00:00 2024-02-26T04:44:22.483 INFO:teuthology.orchestra.run.smithi060.stdout:(78/166): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-02-26T04:44:22.550 INFO:teuthology.orchestra.run.smithi060.stdout:(79/166): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-02-26T04:44:22.576 INFO:teuthology.orchestra.run.smithi060.stdout:(80/166): python3-jsonpatch-1.21-16.el9.noarch. 203 kB/s | 26 kB 00:00 2024-02-26T04:44:22.626 INFO:teuthology.orchestra.run.smithi060.stdout:(81/166): python3-markupsafe-1.1.1-12.el9.x86_6 697 kB/s | 35 kB 00:00 2024-02-26T04:44:22.718 INFO:teuthology.orchestra.run.smithi060.stdout:(82/166): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-02-26T04:44:22.835 INFO:teuthology.orchestra.run.smithi060.stdout:(83/166): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-02-26T04:44:22.960 INFO:teuthology.orchestra.run.smithi060.stdout:(84/166): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-02-26T04:44:23.010 INFO:teuthology.orchestra.run.smithi060.stdout:(85/166): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-02-26T04:44:23.052 INFO:teuthology.orchestra.run.smithi060.stdout:(86/166): python3-packaging-20.9-5.el9.noarch.r 841 kB/s | 77 kB 00:00 2024-02-26T04:44:23.153 INFO:teuthology.orchestra.run.smithi060.stdout:(87/166): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-02-26T04:44:23.169 INFO:teuthology.orchestra.run.smithi060.stdout:(88/166): python3-prettytable-0.7.2-27.el9.noar 266 kB/s | 42 kB 00:00 2024-02-26T04:44:23.228 INFO:teuthology.orchestra.run.smithi060.stdout:(89/166): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-02-26T04:44:23.262 INFO:teuthology.orchestra.run.smithi060.stdout:(90/166): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-02-26T04:44:23.312 INFO:teuthology.orchestra.run.smithi060.stdout:(91/166): python3-pytz-2021.1-5.el9.noarch.rpm 612 kB/s | 51 kB 00:00 2024-02-26T04:44:23.362 INFO:teuthology.orchestra.run.smithi060.stdout:(92/166): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 54 kB 00:00 2024-02-26T04:44:23.488 INFO:teuthology.orchestra.run.smithi060.stdout:(93/166): python3-toml-0.10.2-6.el9.noarch.rpm 333 kB/s | 42 kB 00:00 2024-02-26T04:44:23.535 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-02-26T04:44:23.571 INFO:teuthology.orchestra.run.smithi060.stdout:(94/166): python3-wcwidth-0.2.5-8.el9.noarch.rp 531 kB/s | 44 kB 00:00 2024-02-26T04:44:23.680 INFO:teuthology.orchestra.run.smithi060.stdout:(95/166): qatlib-23.11.0-1.el9.x86_64.rpm 2.0 MB/s | 216 kB 00:00 2024-02-26T04:44:23.763 INFO:teuthology.orchestra.run.smithi060.stdout:(96/166): qatlib-service-23.11.0-1.el9.x86_64.r 418 kB/s | 35 kB 00:00 2024-02-26T04:44:23.931 INFO:teuthology.orchestra.run.smithi060.stdout:(97/166): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-02-26T04:44:24.006 INFO:teuthology.orchestra.run.smithi060.stdout:(98/166): qatzip-libs-1.1.2-2.el9.x86_64.rpm 184 kB/s | 44 kB 00:00 2024-02-26T04:44:24.048 INFO:teuthology.orchestra.run.smithi060.stdout:(99/166): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-02-26T04:44:24.123 INFO:teuthology.orchestra.run.smithi060.stdout:(100/166): uuid-1.6.2-55.el9.x86_64.rpm 494 kB/s | 58 kB 00:00 2024-02-26T04:44:24.156 INFO:teuthology.orchestra.run.smithi060.stdout:(101/166): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 862 kB/s | 94 kB 00:00 2024-02-26T04:44:24.232 INFO:teuthology.orchestra.run.smithi060.stdout:(102/166): xmlstarlet-1.6.1-20.el9.x86_64.rpm 587 kB/s | 64 kB 00:00 2024-02-26T04:44:24.257 INFO:teuthology.orchestra.run.smithi060.stdout:(103/166): zlib-devel-1.2.11-40.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-02-26T04:44:24.416 INFO:teuthology.orchestra.run.smithi060.stdout:(104/166): lua-devel-5.4.4-4.el9.x86_64.rpm 121 kB/s | 22 kB 00:00 2024-02-26T04:44:24.550 INFO:teuthology.orchestra.run.smithi191.stdout:(10/166): ceph-test-19.0.0-1514.g25296a8f.el9.x 15 MB/s | 47 MB 00:03 2024-02-26T04:44:24.667 INFO:teuthology.orchestra.run.smithi191.stdout:(11/166): ceph-radosgw-19.0.0-1514.g25296a8f.el 2.2 MB/s | 9.7 MB 00:04 2024-02-26T04:44:24.867 INFO:teuthology.orchestra.run.smithi191.stdout:(12/166): ceph-common-19.0.0-1514.g25296a8f.el9 3.0 MB/s | 20 MB 00:06 2024-02-26T04:44:24.883 INFO:teuthology.orchestra.run.smithi060.stdout:(105/166): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-02-26T04:44:24.893 INFO:teuthology.orchestra.run.smithi191.stdout:(13/166): cephfs-shell-19.0.0-1514.g25296a8f.el 120 kB/s | 27 kB 00:00 2024-02-26T04:44:24.919 INFO:teuthology.orchestra.run.smithi191.stdout:(14/166): cephfs-mirror-19.0.0-1514.g25296a8f.e 590 kB/s | 217 kB 00:00 2024-02-26T04:44:24.936 INFO:teuthology.orchestra.run.smithi191.stdout:(15/166): libcephfs-devel-19.0.0-1514.g25296a8f 457 kB/s | 31 kB 00:00 2024-02-26T04:44:24.978 INFO:teuthology.orchestra.run.smithi191.stdout:(16/166): libcephfs2-19.0.0-1514.g25296a8f.el9. 8.5 MB/s | 743 kB 00:00 2024-02-26T04:44:25.029 INFO:teuthology.orchestra.run.smithi191.stdout:(17/166): libcephsqlite-19.0.0-1514.g25296a8f.e 1.5 MB/s | 163 kB 00:00 2024-02-26T04:44:25.054 INFO:teuthology.orchestra.run.smithi191.stdout:(18/166): librados-devel-19.0.0-1514.g25296a8f. 1.1 MB/s | 127 kB 00:00 2024-02-26T04:44:25.079 INFO:teuthology.orchestra.run.smithi191.stdout:(19/166): libradosstriper1-19.0.0-1514.g25296a8 4.8 MB/s | 498 kB 00:00 2024-02-26T04:44:25.096 INFO:teuthology.orchestra.run.smithi191.stdout:(20/166): python3-ceph-argparse-19.0.0-1514.g25 1.1 MB/s | 45 kB 00:00 2024-02-26T04:44:25.130 INFO:teuthology.orchestra.run.smithi191.stdout:(21/166): python3-ceph-common-19.0.0-1514.g2529 2.6 MB/s | 131 kB 00:00 2024-02-26T04:44:25.180 INFO:teuthology.orchestra.run.smithi191.stdout:(22/166): python3-cephfs-19.0.0-1514.g25296a8f. 1.9 MB/s | 161 kB 00:00 2024-02-26T04:44:25.205 INFO:teuthology.orchestra.run.smithi191.stdout:(23/166): python3-rados-19.0.0-1514.g25296a8f.e 4.2 MB/s | 320 kB 00:00 2024-02-26T04:44:25.255 INFO:teuthology.orchestra.run.smithi191.stdout:(24/166): python3-rbd-19.0.0-1514.g25296a8f.el9 3.9 MB/s | 298 kB 00:00 2024-02-26T04:44:25.281 INFO:teuthology.orchestra.run.smithi191.stdout:(25/166): python3-rgw-19.0.0-1514.g25296a8f.el9 1.3 MB/s | 99 kB 00:00 2024-02-26T04:44:25.306 INFO:teuthology.orchestra.run.smithi191.stdout:(26/166): rbd-fuse-19.0.0-1514.g25296a8f.el9.x8 1.7 MB/s | 85 kB 00:00 2024-02-26T04:44:25.325 INFO:teuthology.orchestra.run.smithi060.stdout:(106/166): ceph-test-19.0.0-1514.g25296a8f.el9. 3.7 MB/s | 47 MB 00:12 2024-02-26T04:44:25.373 INFO:teuthology.orchestra.run.smithi191.stdout:(27/166): rbd-nbd-19.0.0-1514.g25296a8f.el9.x86 2.5 MB/s | 171 kB 00:00 2024-02-26T04:44:25.415 INFO:teuthology.orchestra.run.smithi191.stdout:(28/166): ceph-grafana-dashboards-19.0.0-1514.g 757 kB/s | 31 kB 00:00 2024-02-26T04:44:25.474 INFO:teuthology.orchestra.run.smithi191.stdout:(29/166): ceph-mgr-cephadm-19.0.0-1514.g25296a8 2.3 MB/s | 138 kB 00:00 2024-02-26T04:44:25.608 INFO:teuthology.orchestra.run.smithi060.stdout:(107/166): abseil-cpp-20211102.0-3.el9.x86_64.r 460 kB/s | 548 kB 00:01 2024-02-26T04:44:25.632 INFO:teuthology.orchestra.run.smithi191.stdout:(30/166): ceph-mgr-dashboard-19.0.0-1514.g25296 11 MB/s | 1.8 MB 00:00 2024-02-26T04:44:25.675 INFO:teuthology.orchestra.run.smithi060.stdout:(108/166): grpc-data-1.46.7-10.el9.noarch.rpm 60 kB/s | 19 kB 00:00 2024-02-26T04:44:25.809 INFO:teuthology.orchestra.run.smithi060.stdout:(109/166): libarrow-doc-9.0.0-10.el9.noarch.rpm 195 kB/s | 26 kB 00:00 2024-02-26T04:44:25.851 INFO:teuthology.orchestra.run.smithi060.stdout:(110/166): gperftools-libs-2.9.1-2.el9.x86_64.r 319 kB/s | 309 kB 00:00 2024-02-26T04:44:25.951 INFO:teuthology.orchestra.run.smithi060.stdout:(111/166): libunwind-1.6.2-1.el9.x86_64.rpm 674 kB/s | 67 kB 00:00 2024-02-26T04:44:25.985 INFO:teuthology.orchestra.run.smithi060.stdout:(112/166): liboath-2.6.7-2.el9.x86_64.rpm 291 kB/s | 49 kB 00:00 2024-02-26T04:44:25.991 INFO:teuthology.orchestra.run.smithi191.stdout:(31/166): rbd-mirror-19.0.0-1514.g25296a8f.el9. 4.4 MB/s | 3.1 MB 00:00 2024-02-26T04:44:26.077 INFO:teuthology.orchestra.run.smithi060.stdout:(113/166): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-02-26T04:44:26.129 INFO:teuthology.orchestra.run.smithi156.stderr:Modular dependency problems: 2024-02-26T04:44:26.129 INFO:teuthology.orchestra.run.smithi156.stderr: 2024-02-26T04:44:26.129 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 1: conflicting requests 2024-02-26T04:44:26.129 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.129 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 2: conflicting requests 2024-02-26T04:44:26.130 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.130 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 3: conflicting requests 2024-02-26T04:44:26.130 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.130 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 4: conflicting requests 2024-02-26T04:44:26.130 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.130 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 5: conflicting requests 2024-02-26T04:44:26.130 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.130 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 6: conflicting requests 2024-02-26T04:44:26.130 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.130 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 7: conflicting requests 2024-02-26T04:44:26.130 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.130 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 8: conflicting requests 2024-02-26T04:44:26.131 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.131 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 9: conflicting requests 2024-02-26T04:44:26.131 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.131 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 10: conflicting requests 2024-02-26T04:44:26.131 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.131 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 11: conflicting requests 2024-02-26T04:44:26.131 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.131 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 12: conflicting requests 2024-02-26T04:44:26.131 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.131 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 13: conflicting requests 2024-02-26T04:44:26.131 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.131 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 14: conflicting requests 2024-02-26T04:44:26.131 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.131 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 15: conflicting requests 2024-02-26T04:44:26.132 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.132 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 16: conflicting requests 2024-02-26T04:44:26.132 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.132 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 17: conflicting requests 2024-02-26T04:44:26.132 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.132 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 18: conflicting requests 2024-02-26T04:44:26.132 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.132 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 19: conflicting requests 2024-02-26T04:44:26.132 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.132 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 20: conflicting requests 2024-02-26T04:44:26.132 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.132 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 21: conflicting requests 2024-02-26T04:44:26.133 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.133 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 22: conflicting requests 2024-02-26T04:44:26.133 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.133 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 23: conflicting requests 2024-02-26T04:44:26.133 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.133 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 24: conflicting requests 2024-02-26T04:44:26.133 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.133 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 25: conflicting requests 2024-02-26T04:44:26.133 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.133 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 26: conflicting requests 2024-02-26T04:44:26.133 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.133 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 27: conflicting requests 2024-02-26T04:44:26.133 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.133 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 28: conflicting requests 2024-02-26T04:44:26.134 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.134 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 29: conflicting requests 2024-02-26T04:44:26.134 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.134 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 30: conflicting requests 2024-02-26T04:44:26.134 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.134 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 31: conflicting requests 2024-02-26T04:44:26.134 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.134 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 32: conflicting requests 2024-02-26T04:44:26.134 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.134 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 33: conflicting requests 2024-02-26T04:44:26.134 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.134 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 34: conflicting requests 2024-02-26T04:44:26.134 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.135 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 35: conflicting requests 2024-02-26T04:44:26.135 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.135 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 36: conflicting requests 2024-02-26T04:44:26.135 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.135 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.135 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.135 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T04:44:26.135 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.135 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.135 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.135 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.135 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 38: conflicting requests 2024-02-26T04:44:26.135 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.135 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.136 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.136 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.136 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.136 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.136 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T04:44:26.136 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.136 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.136 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.136 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.136 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T04:44:26.136 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.136 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.136 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.137 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.137 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T04:44:26.137 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.137 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.137 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.137 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.137 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T04:44:26.137 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.137 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.137 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.137 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.137 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T04:44:26.137 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.137 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.138 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.138 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.138 INFO:teuthology.orchestra.run.smithi156.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T04:44:26.138 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.138 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.138 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.138 INFO:teuthology.orchestra.run.smithi156.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T04:44:26.149 INFO:teuthology.orchestra.run.smithi191.stdout:(32/166): ceph-mgr-diskprediction-local-19.0.0- 14 MB/s | 7.4 MB 00:00 2024-02-26T04:44:26.216 INFO:teuthology.orchestra.run.smithi191.stdout:(33/166): librgw2-19.0.0-1514.g25296a8f.el9.x86 4.3 MB/s | 5.1 MB 00:01 2024-02-26T04:44:26.242 INFO:teuthology.orchestra.run.smithi191.stdout:(34/166): ceph-mgr-rook-19.0.0-1514.g25296a8f.e 571 kB/s | 52 kB 00:00 2024-02-26T04:44:26.244 INFO:teuthology.orchestra.run.smithi060.stdout:(114/166): python3-asyncssh-2.13.2-4.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-02-26T04:44:26.267 INFO:teuthology.orchestra.run.smithi191.stdout:(35/166): ceph-prometheus-alerts-19.0.0-1514.g2 306 kB/s | 15 kB 00:00 2024-02-26T04:44:26.292 INFO:teuthology.orchestra.run.smithi191.stdout:(36/166): ceph-mgr-modules-core-19.0.0-1514.g25 823 kB/s | 247 kB 00:00 2024-02-26T04:44:26.302 INFO:teuthology.orchestra.run.smithi060.stdout:(115/166): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-02-26T04:44:26.317 INFO:teuthology.orchestra.run.smithi191.stdout:(37/166): ceph-volume-19.0.0-1514.g25296a8f.el9 3.4 MB/s | 264 kB 00:00 2024-02-26T04:44:26.327 INFO:teuthology.orchestra.run.smithi060.stdout:(116/166): python3-bcrypt-3.2.2-1.el9.x86_64.rp 521 kB/s | 43 kB 00:00 2024-02-26T04:44:26.343 INFO:teuthology.orchestra.run.smithi191.stdout:(38/166): cephfs-top-19.0.0-1514.g25296a8f.el9. 363 kB/s | 18 kB 00:00 2024-02-26T04:44:26.386 INFO:teuthology.orchestra.run.smithi060.stdout:(117/166): python3-cachetools-4.2.4-1.el9.noarc 386 kB/s | 32 kB 00:00 2024-02-26T04:44:26.410 INFO:teuthology.orchestra.run.smithi191.stdout:(39/166): cephadm-19.0.0-1514.g25296a8f.el9.noa 5.1 MB/s | 748 kB 00:00 2024-02-26T04:44:26.411 INFO:teuthology.orchestra.run.smithi060.stdout:(118/166): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-02-26T04:44:26.451 INFO:teuthology.orchestra.run.smithi191.stdout:(40/166): libaio-devel-0.3.112-1.el8.x86_64.rpm 132 kB/s | 18 kB 00:00 2024-02-26T04:44:26.470 INFO:teuthology.orchestra.run.smithi060.stdout:(119/166): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-02-26T04:44:26.477 INFO:teuthology.orchestra.run.smithi191.stdout:(41/166): fuse-2.9.9-15.el9.x86_64.rpm 601 kB/s | 80 kB 00:00 2024-02-26T04:44:26.504 INFO:teuthology.orchestra.run.smithi060.stdout:(120/166): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-02-26T04:44:26.577 INFO:teuthology.orchestra.run.smithi191.stdout:(42/166): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-02-26T04:44:26.587 INFO:teuthology.orchestra.run.smithi060.stdout:(121/166): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-02-26T04:44:26.598 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T04:44:26.599 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T04:44:26.611 INFO:teuthology.orchestra.run.smithi191.stdout:(43/166): jq-1.6-16.el9.x86_64.rpm 939 kB/s | 187 kB 00:00 2024-02-26T04:44:26.612 INFO:teuthology.orchestra.run.smithi060.stdout:(122/166): python3-colorama-0.4.4-12.el9.noarch 325 kB/s | 35 kB 00:00 2024-02-26T04:44:26.613 INFO:teuthology.orchestra.run.smithi156.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-26T04:44:26.613 INFO:teuthology.orchestra.run.smithi156.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-26T04:44:26.614 INFO:teuthology.orchestra.run.smithi156.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-26T04:44:26.617 INFO:teuthology.orchestra.run.smithi156.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-26T04:44:26.618 INFO:teuthology.orchestra.run.smithi156.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-26T04:44:26.620 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-26T04:44:26.621 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-26T04:44:26.678 INFO:teuthology.orchestra.run.smithi191.stdout:(44/166): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.1 MB/s | 218 kB 00:00 2024-02-26T04:44:26.688 INFO:teuthology.orchestra.run.smithi060.stdout:(123/166): python3-google-auth-2.28.0-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-02-26T04:44:26.711 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-02-26T04:44:26.720 INFO:teuthology.orchestra.run.smithi191.stdout:(45/166): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-02-26T04:44:26.727 INFO:teuthology.orchestra.run.smithi156.stdout:========================================================================================= 2024-02-26T04:44:26.727 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2024-02-26T04:44:26.727 INFO:teuthology.orchestra.run.smithi156.stdout:========================================================================================= 2024-02-26T04:44:26.727 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-02-26T04:44:26.727 INFO:teuthology.orchestra.run.smithi156.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-26T04:44:26.728 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 6.6 k 2024-02-26T04:44:26.728 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.3 M 2024-02-26T04:44:26.728 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 886 k 2024-02-26T04:44:26.728 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 145 k 2024-02-26T04:44:26.728 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 1.5 M 2024-02-26T04:44:26.728 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 138 k 2024-02-26T04:44:26.728 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 1.8 M 2024-02-26T04:44:26.728 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 7.4 M 2024-02-26T04:44:26.728 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 52 k 2024-02-26T04:44:26.728 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 9.7 M 2024-02-26T04:44:26.728 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 47 M 2024-02-26T04:44:26.728 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 264 k 2024-02-26T04:44:26.728 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 748 k 2024-02-26T04:44:26.729 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 217 k 2024-02-26T04:44:26.729 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-shell x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 27 k 2024-02-26T04:44:26.729 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 18 k 2024-02-26T04:44:26.729 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-26T04:44:26.729 INFO:teuthology.orchestra.run.smithi156.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-26T04:44:26.729 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-26T04:44:26.729 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 31 k 2024-02-26T04:44:26.729 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 743 k 2024-02-26T04:44:26.729 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 127 k 2024-02-26T04:44:26.729 INFO:teuthology.orchestra.run.smithi156.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-26T04:44:26.729 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-26T04:44:26.730 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-26T04:44:26.730 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-26T04:44:26.730 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-26T04:44:26.730 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-26T04:44:26.730 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 161 k 2024-02-26T04:44:26.730 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 320 k 2024-02-26T04:44:26.730 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 298 k 2024-02-26T04:44:26.730 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 99 k 2024-02-26T04:44:26.730 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 85 k 2024-02-26T04:44:26.730 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T04:44:26.730 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 171 k 2024-02-26T04:44:26.731 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-26T04:44:26.731 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2024-02-26T04:44:26.731 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-26T04:44:26.731 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-26T04:44:26.731 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-26T04:44:26.731 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-26T04:44:26.731 INFO:teuthology.orchestra.run.smithi156.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-26T04:44:26.731 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-26T04:44:26.731 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-26T04:44:26.731 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-26T04:44:26.732 INFO:teuthology.orchestra.run.smithi156.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-26T04:44:26.732 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-26T04:44:26.732 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.4 M 2024-02-26T04:44:26.732 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 3.1 M 2024-02-26T04:44:26.732 INFO:teuthology.orchestra.run.smithi156.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-26T04:44:26.732 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-26T04:44:26.732 INFO:teuthology.orchestra.run.smithi156.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-26T04:44:26.732 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-26T04:44:26.732 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2024-02-26T04:44:26.732 INFO:teuthology.orchestra.run.smithi156.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-26T04:44:26.732 INFO:teuthology.orchestra.run.smithi156.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-26T04:44:26.732 INFO:teuthology.orchestra.run.smithi156.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-26T04:44:26.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 20 M 2024-02-26T04:44:26.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 31 k 2024-02-26T04:44:26.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 2.1 M 2024-02-26T04:44:26.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 247 k 2024-02-26T04:44:26.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 4.7 M 2024-02-26T04:44:26.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 17 M 2024-02-26T04:44:26.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1514.g25296a8f.el9 ceph-noarch 15 k 2024-02-26T04:44:26.733 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 25 k 2024-02-26T04:44:26.733 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-26T04:44:26.733 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-26T04:44:26.733 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-26T04:44:26.733 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-26T04:44:26.733 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-26T04:44:26.733 INFO:teuthology.orchestra.run.smithi156.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-26T04:44:26.734 INFO:teuthology.orchestra.run.smithi156.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-26T04:44:26.734 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-26T04:44:26.734 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-26T04:44:26.734 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 163 k 2024-02-26T04:44:26.734 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-26T04:44:26.734 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-26T04:44:26.734 INFO:teuthology.orchestra.run.smithi156.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-26T04:44:26.734 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-26T04:44:26.734 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-26T04:44:26.735 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 498 k 2024-02-26T04:44:26.735 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-26T04:44:26.735 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 5.1 M 2024-02-26T04:44:26.735 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-26T04:44:26.735 INFO:teuthology.orchestra.run.smithi156.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-26T04:44:26.735 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-26T04:44:26.735 INFO:teuthology.orchestra.run.smithi156.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-26T04:44:26.735 INFO:teuthology.orchestra.run.smithi156.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-26T04:44:26.735 INFO:teuthology.orchestra.run.smithi156.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-26T04:44:26.735 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-26T04:44:26.735 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-26T04:44:26.735 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-26T04:44:26.736 INFO:teuthology.orchestra.run.smithi156.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-26T04:44:26.736 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-26T04:44:26.736 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-26T04:44:26.736 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-26T04:44:26.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-26T04:44:26.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-26T04:44:26.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-26T04:44:26.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-26T04:44:26.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-26T04:44:26.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 45 k 2024-02-26T04:44:26.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:19.0.0-1514.g25296a8f.el9 ceph 131 k 2024-02-26T04:44:26.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-26T04:44:26.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-26T04:44:26.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-26T04:44:26.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-26T04:44:26.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-26T04:44:26.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-26T04:44:26.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-26T04:44:26.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-26T04:44:26.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-26T04:44:26.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-26T04:44:26.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-26T04:44:26.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-26T04:44:26.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-26T04:44:26.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-26T04:44:26.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-26T04:44:26.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-26T04:44:26.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-26T04:44:26.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-26T04:44:26.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-26T04:44:26.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-26T04:44:26.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-26T04:44:26.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-26T04:44:26.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-26T04:44:26.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-26T04:44:26.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-26T04:44:26.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-26T04:44:26.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-26T04:44:26.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-26T04:44:26.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-26T04:44:26.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-26T04:44:26.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-26T04:44:26.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-26T04:44:26.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-26T04:44:26.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-26T04:44:26.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-26T04:44:26.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-26T04:44:26.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-26T04:44:26.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-26T04:44:26.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-26T04:44:26.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-26T04:44:26.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-26T04:44:26.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-26T04:44:26.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-26T04:44:26.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-26T04:44:26.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-26T04:44:26.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-26T04:44:26.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-26T04:44:26.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-26T04:44:26.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-26T04:44:26.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-26T04:44:26.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-26T04:44:26.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-26T04:44:26.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-26T04:44:26.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-26T04:44:26.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-26T04:44:26.740 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-26T04:44:26.741 INFO:teuthology.orchestra.run.smithi156.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-26T04:44:26.741 INFO:teuthology.orchestra.run.smithi156.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-26T04:44:26.741 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-26T04:44:26.741 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-26T04:44:26.741 INFO:teuthology.orchestra.run.smithi156.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-26T04:44:26.741 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-26T04:44:26.741 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-26T04:44:26.741 INFO:teuthology.orchestra.run.smithi156.stdout:Installing weak dependencies: 2024-02-26T04:44:26.741 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-26T04:44:26.741 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:44:26.741 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-02-26T04:44:26.741 INFO:teuthology.orchestra.run.smithi156.stdout:========================================================================================= 2024-02-26T04:44:26.741 INFO:teuthology.orchestra.run.smithi156.stdout:Install 150 Packages 2024-02-26T04:44:26.741 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 16 Packages 2024-02-26T04:44:26.742 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:44:26.742 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 221 M 2024-02-26T04:44:26.743 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-02-26T04:44:26.779 INFO:teuthology.orchestra.run.smithi191.stdout:(46/166): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-02-26T04:44:26.780 INFO:teuthology.orchestra.run.smithi060.stdout:(124/166): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-02-26T04:44:26.812 INFO:teuthology.orchestra.run.smithi191.stdout:(47/166): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-02-26T04:44:26.822 INFO:teuthology.orchestra.run.smithi060.stdout:(125/166): python3-grpcio-1.46.7-10.el9.x86_64. 9.8 MB/s | 2.0 MB 00:00 2024-02-26T04:44:26.837 INFO:teuthology.orchestra.run.smithi191.stdout:(48/166): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-02-26T04:44:26.864 INFO:teuthology.orchestra.run.smithi060.stdout:(126/166): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-02-26T04:44:26.871 INFO:teuthology.orchestra.run.smithi191.stdout:(49/166): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-02-26T04:44:26.889 INFO:teuthology.orchestra.run.smithi060.stdout:(127/166): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-02-26T04:44:26.921 INFO:teuthology.orchestra.run.smithi191.stdout:(50/166): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-02-26T04:44:26.931 INFO:teuthology.orchestra.run.smithi060.stdout:(128/166): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-02-26T04:44:26.964 INFO:teuthology.orchestra.run.smithi060.stdout:(129/166): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-02-26T04:44:26.988 INFO:teuthology.orchestra.run.smithi191.stdout:(51/166): automake-1.16.2-8.el9.noarch.rpm 5.8 MB/s | 689 kB 00:00 2024-02-26T04:44:26.998 INFO:teuthology.orchestra.run.smithi060.stdout:(130/166): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-02-26T04:44:27.063 INFO:teuthology.orchestra.run.smithi191.stdout:(52/166): bison-3.7.4-5.el9.x86_64.rpm 6.5 MB/s | 943 kB 00:00 2024-02-26T04:44:27.082 INFO:teuthology.orchestra.run.smithi060.stdout:(131/166): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-02-26T04:44:27.089 INFO:teuthology.orchestra.run.smithi191.stdout:(53/166): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-02-26T04:44:27.107 INFO:teuthology.orchestra.run.smithi060.stdout:(132/166): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-02-26T04:44:27.131 INFO:teuthology.orchestra.run.smithi191.stdout:(54/166): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-02-26T04:44:27.157 INFO:teuthology.orchestra.run.smithi060.stdout:(133/166): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-02-26T04:44:27.183 INFO:teuthology.orchestra.run.smithi060.stdout:(134/166): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-02-26T04:44:27.197 INFO:teuthology.orchestra.run.smithi191.stdout:(55/166): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-02-26T04:44:27.223 INFO:teuthology.orchestra.run.smithi191.stdout:(56/166): flexiblas-3.0.4-8.el9.x86_64.rpm 357 kB/s | 33 kB 00:00 2024-02-26T04:44:27.250 INFO:teuthology.orchestra.run.smithi060.stdout:(135/166): libarrow-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 4.4 MB 00:01 2024-02-26T04:44:27.273 INFO:teuthology.orchestra.run.smithi191.stdout:(57/166): flexiblas-openblas-openmp-3.0.4-8.el9 366 kB/s | 18 kB 00:00 2024-02-26T04:44:27.275 INFO:teuthology.orchestra.run.smithi060.stdout:(136/166): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-02-26T04:44:27.300 INFO:teuthology.orchestra.run.smithi060.stdout:(137/166): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-02-26T04:44:27.306 INFO:teuthology.orchestra.run.smithi191.stdout:(58/166): python3-cryptography-36.0.1-4.el9.x86 2.0 MB/s | 1.2 MB 00:00 2024-02-26T04:44:27.343 INFO:teuthology.orchestra.run.smithi060.stdout:(138/166): python3-pyOpenSSL-21.0.0-1.el9.noarc 975 kB/s | 90 kB 00:00 2024-02-26T04:44:27.365 INFO:teuthology.orchestra.run.smithi191.stdout:(59/166): librabbitmq-0.11.0-7.el9.x86_64.rpm 779 kB/s | 45 kB 00:00 2024-02-26T04:44:27.367 INFO:teuthology.orchestra.run.smithi060.stdout:(139/166): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-02-26T04:44:27.390 INFO:teuthology.orchestra.run.smithi191.stdout:(60/166): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-02-26T04:44:27.393 INFO:teuthology.orchestra.run.smithi060.stdout:(140/166): python3-repoze-lru-0.7-16.el9.noarch 333 kB/s | 31 kB 00:00 2024-02-26T04:44:27.476 INFO:teuthology.orchestra.run.smithi060.stdout:(141/166): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-02-26T04:44:27.482 INFO:teuthology.orchestra.run.smithi191.stdout:(61/166): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-02-26T04:44:27.502 INFO:teuthology.orchestra.run.smithi060.stdout:(142/166): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-02-26T04:44:27.524 INFO:teuthology.orchestra.run.smithi191.stdout:(62/166): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-02-26T04:44:27.527 INFO:teuthology.orchestra.run.smithi060.stdout:(143/166): python3-tempora-5.0.0-2.el9.noarch.r 267 kB/s | 36 kB 00:00 2024-02-26T04:44:27.558 INFO:teuthology.orchestra.run.smithi191.stdout:(63/166): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-02-26T04:44:27.577 INFO:teuthology.orchestra.run.smithi060.stdout:(144/166): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2024-02-26T04:44:27.583 INFO:teuthology.orchestra.run.smithi191.stdout:(64/166): libuuid-devel-2.37.4-15.el9.x86_64.rp 255 kB/s | 26 kB 00:00 2024-02-26T04:44:27.628 INFO:teuthology.orchestra.run.smithi060.stdout:(145/166): python3-websocket-client-1.2.3-2.el9 894 kB/s | 90 kB 00:00 2024-02-26T04:44:27.629 INFO:teuthology.orchestra.run.smithi191.stdout:(65/166): libzstd-devel-1.5.1-2.el9.x86_64.rpm 454 kB/s | 47 kB 00:00 2024-02-26T04:44:27.661 INFO:teuthology.orchestra.run.smithi060.stdout:(146/166): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-02-26T04:44:27.686 INFO:teuthology.orchestra.run.smithi060.stdout:(147/166): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-02-26T04:44:27.693 INFO:teuthology.orchestra.run.smithi191.stdout:(66/166): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-02-26T04:44:27.718 INFO:teuthology.orchestra.run.smithi191.stdout:(67/166): openblas-0.3.21-2.el9.x86_64.rpm 398 kB/s | 35 kB 00:00 2024-02-26T04:44:27.720 INFO:teuthology.orchestra.run.smithi060.stdout:(148/166): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-02-26T04:44:27.743 INFO:teuthology.orchestra.run.smithi191.stdout:(68/166): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-02-26T04:44:27.812 INFO:teuthology.orchestra.run.smithi060.stdout:(149/166): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-02-26T04:44:27.885 INFO:teuthology.orchestra.run.smithi191.stdout:(69/166): postgresql-13.11-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-02-26T04:44:27.929 INFO:teuthology.orchestra.run.smithi060.stdout:(150/166): thrift-0.15.0-2.el9.x86_64.rpm 6.5 MB/s | 1.6 MB 00:00 2024-02-26T04:44:27.994 INFO:teuthology.orchestra.run.smithi191.stdout:(70/166): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-02-26T04:44:28.061 INFO:teuthology.orchestra.run.smithi191.stdout:(71/166): openssl-devel-3.0.7-25.el9.x86_64.rpm 12 MB/s | 4.1 MB 00:00 2024-02-26T04:44:28.119 INFO:teuthology.orchestra.run.smithi191.stdout:(72/166): postgresql-private-libs-13.11-1.el9.x 1.1 MB/s | 135 kB 00:00 2024-02-26T04:44:28.128 INFO:teuthology.orchestra.run.smithi156.stdout:(1/166): ceph-19.0.0-1514.g25296a8f.el9.x86_64. 11 kB/s | 6.6 kB 00:00 2024-02-26T04:44:28.129 INFO:teuthology.orchestra.run.smithi060.stdout:(151/166): NetworkManager-1.45.91-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-02-26T04:44:28.145 INFO:teuthology.orchestra.run.smithi191.stdout:(73/166): postgresql-contrib-13.11-1.el9.x86_64 3.4 MB/s | 905 kB 00:00 2024-02-26T04:44:28.203 INFO:teuthology.orchestra.run.smithi191.stdout:(74/166): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-02-26T04:44:28.230 INFO:teuthology.orchestra.run.smithi060.stdout:(152/166): NetworkManager-libnm-1.45.91-1.el9.x 18 MB/s | 1.8 MB 00:00 2024-02-26T04:44:28.245 INFO:teuthology.orchestra.run.smithi191.stdout:(75/166): protobuf-3.14.0-13.el9.x86_64.rpm 8.0 MB/s | 1.0 MB 00:00 2024-02-26T04:44:28.272 INFO:teuthology.orchestra.run.smithi060.stdout:(153/166): NetworkManager-team-1.45.91-1.el9.x8 1.0 MB/s | 42 kB 00:00 2024-02-26T04:44:28.313 INFO:teuthology.orchestra.run.smithi060.stdout:(154/166): NetworkManager-tui-1.45.91-1.el9.x86 5.8 MB/s | 247 kB 00:00 2024-02-26T04:44:28.329 INFO:teuthology.orchestra.run.smithi191.stdout:(76/166): python3-devel-3.9.17-2.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-02-26T04:44:28.358 INFO:teuthology.orchestra.run.smithi060.stdout:(155/166): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-02-26T04:44:28.398 INFO:teuthology.orchestra.run.smithi060.stdout:(156/166): elfutils-debuginfod-client-0.190-2.e 875 kB/s | 36 kB 00:00 2024-02-26T04:44:28.421 INFO:teuthology.orchestra.run.smithi191.stdout:(77/166): postgresql-server-13.11-1.el9.x86_64. 16 MB/s | 5.7 MB 00:00 2024-02-26T04:44:28.441 INFO:teuthology.orchestra.run.smithi060.stdout:(157/166): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2024-02-26T04:44:28.472 INFO:teuthology.orchestra.run.smithi191.stdout:(78/166): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-02-26T04:44:28.475 INFO:teuthology.orchestra.run.smithi060.stdout:(158/166): elfutils-libs-0.190-2.el9.x86_64.rpm 7.5 MB/s | 256 kB 00:00 2024-02-26T04:44:28.498 INFO:teuthology.orchestra.run.smithi191.stdout:(79/166): python3-jsonpatch-1.21-16.el9.noarch. 341 kB/s | 26 kB 00:00 2024-02-26T04:44:28.516 INFO:teuthology.orchestra.run.smithi060.stdout:(159/166): libacl-2.3.1-4.el9.x86_64.rpm 554 kB/s | 23 kB 00:00 2024-02-26T04:44:28.540 INFO:teuthology.orchestra.run.smithi191.stdout:(80/166): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-02-26T04:44:28.574 INFO:teuthology.orchestra.run.smithi060.stdout:(160/166): openssl-3.0.7-25.el9.x86_64.rpm 21 MB/s | 1.2 MB 00:00 2024-02-26T04:44:28.639 INFO:teuthology.orchestra.run.smithi191.stdout:(81/166): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-02-26T04:44:28.649 INFO:teuthology.orchestra.run.smithi060.stdout:(161/166): openssl-libs-3.0.7-25.el9.x86_64.rpm 29 MB/s | 2.2 MB 00:00 2024-02-26T04:44:28.664 INFO:teuthology.orchestra.run.smithi191.stdout:(82/166): python3-markupsafe-1.1.1-12.el9.x86_6 277 kB/s | 35 kB 00:00 2024-02-26T04:44:28.690 INFO:teuthology.orchestra.run.smithi191.stdout:(83/166): python3-mako-1.1.4-6.el9.noarch.rpm 894 kB/s | 172 kB 00:00 2024-02-26T04:44:28.708 INFO:teuthology.orchestra.run.smithi060.stdout:(162/166): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2024-02-26T04:44:28.764 INFO:teuthology.orchestra.run.smithi191.stdout:(84/166): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-02-26T04:44:28.766 INFO:teuthology.orchestra.run.smithi060.stdout:(163/166): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-02-26T04:44:28.798 INFO:teuthology.orchestra.run.smithi191.stdout:(85/166): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-02-26T04:44:28.848 INFO:teuthology.orchestra.run.smithi191.stdout:(86/166): python3-packaging-20.9-5.el9.noarch.r 922 kB/s | 77 kB 00:00 2024-02-26T04:44:28.858 INFO:teuthology.orchestra.run.smithi060.stdout:(164/166): librbd1-19.0.0-1514.g25296a8f.el9.x8 3.0 MB/s | 3.1 MB 00:01 2024-02-26T04:44:28.865 INFO:teuthology.orchestra.run.smithi191.stdout:(87/166): python3-prettytable-0.7.2-27.el9.noar 632 kB/s | 42 kB 00:00 2024-02-26T04:44:28.883 INFO:teuthology.orchestra.run.smithi060.stdout:(165/166): libacl-devel-2.3.1-4.el9.x86_64.rpm 636 kB/s | 74 kB 00:00 2024-02-26T04:44:28.970 INFO:teuthology.orchestra.run.smithi156.stdout:(2/166): ceph-fuse-19.0.0-1514.g25296a8f.el9.x8 1.0 MB/s | 886 kB 00:00 2024-02-26T04:44:28.991 INFO:teuthology.orchestra.run.smithi191.stdout:(88/166): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-02-26T04:44:29.016 INFO:teuthology.orchestra.run.smithi191.stdout:(89/166): python3-protobuf-3.14.0-13.el9.noarch 1.6 MB/s | 269 kB 00:00 2024-02-26T04:44:29.041 INFO:teuthology.orchestra.run.smithi191.stdout:(90/166): python3-pyasn1-0.4.8-6.el9.noarch.rpm 904 kB/s | 159 kB 00:00 2024-02-26T04:44:29.070 INFO:teuthology.orchestra.run.smithi156.stdout:(3/166): ceph-base-19.0.0-1514.g25296a8f.el9.x8 3.5 MB/s | 5.3 MB 00:01 2024-02-26T04:44:29.084 INFO:teuthology.orchestra.run.smithi060.stdout:(166/166): librados2-19.0.0-1514.g25296a8f.el9. 2.5 MB/s | 3.4 MB 00:01 2024-02-26T04:44:29.090 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-02-26T04:44:29.090 INFO:teuthology.orchestra.run.smithi060.stdout:Total 9.9 MB/s | 221 MB 00:22 2024-02-26T04:44:29.091 INFO:teuthology.orchestra.run.smithi191.stdout:(91/166): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-02-26T04:44:29.117 INFO:teuthology.orchestra.run.smithi191.stdout:(92/166): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-02-26T04:44:29.142 INFO:teuthology.orchestra.run.smithi191.stdout:(93/166): python3-requests-oauthlib-1.3.0-12.el 535 kB/s | 54 kB 00:00 2024-02-26T04:44:29.145 INFO:teuthology.orchestra.run.smithi156.stdout:(4/166): ceph-immutable-object-cache-19.0.0-151 829 kB/s | 145 kB 00:00 2024-02-26T04:44:29.192 INFO:teuthology.orchestra.run.smithi191.stdout:(94/166): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-02-26T04:44:29.217 INFO:teuthology.orchestra.run.smithi191.stdout:(95/166): python3-wcwidth-0.2.5-8.el9.noarch.rp 587 kB/s | 44 kB 00:00 2024-02-26T04:44:29.268 INFO:teuthology.orchestra.run.smithi191.stdout:(96/166): qatlib-service-23.11.0-1.el9.x86_64.r 697 kB/s | 35 kB 00:00 2024-02-26T04:44:29.270 INFO:teuthology.orchestra.run.smithi156.stdout:(5/166): ceph-mds-19.0.0-1514.g25296a8f.el9.x86 11 MB/s | 2.1 MB 00:00 2024-02-26T04:44:29.293 INFO:teuthology.orchestra.run.smithi191.stdout:(97/166): qatlib-23.11.0-1.el9.x86_64.rpm 2.1 MB/s | 216 kB 00:00 2024-02-26T04:44:29.344 INFO:teuthology.orchestra.run.smithi191.stdout:(98/166): qatzip-libs-1.1.2-2.el9.x86_64.rpm 590 kB/s | 44 kB 00:00 2024-02-26T04:44:29.402 INFO:teuthology.orchestra.run.smithi191.stdout:(99/166): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-02-26T04:44:29.427 INFO:teuthology.orchestra.run.smithi191.stdout:(100/166): uuid-1.6.2-55.el9.x86_64.rpm 692 kB/s | 58 kB 00:00 2024-02-26T04:44:29.477 INFO:teuthology.orchestra.run.smithi191.stdout:(101/166): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-02-26T04:44:29.503 INFO:teuthology.orchestra.run.smithi191.stdout:(102/166): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-02-26T04:44:29.553 INFO:teuthology.orchestra.run.smithi191.stdout:(103/166): zlib-devel-1.2.11-40.el9.x86_64.rpm 599 kB/s | 45 kB 00:00 2024-02-26T04:44:29.587 INFO:teuthology.orchestra.run.smithi191.stdout:(104/166): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-02-26T04:44:29.612 INFO:teuthology.orchestra.run.smithi156.stdout:(6/166): ceph-mon-19.0.0-1514.g25296a8f.el9.x86 14 MB/s | 4.7 MB 00:00 2024-02-26T04:44:29.729 INFO:teuthology.orchestra.run.smithi156.stdout:(7/166): ceph-mgr-19.0.0-1514.g25296a8f.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-02-26T04:44:29.837 INFO:teuthology.orchestra.run.smithi191.stdout:(105/166): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-02-26T04:44:29.879 INFO:teuthology.orchestra.run.smithi191.stdout:(106/166): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-02-26T04:44:29.938 INFO:teuthology.orchestra.run.smithi191.stdout:(107/166): gperftools-libs-2.9.1-2.el9.x86_64.r 3.1 MB/s | 309 kB 00:00 2024-02-26T04:44:29.963 INFO:teuthology.orchestra.run.smithi191.stdout:(108/166): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-02-26T04:44:29.989 INFO:teuthology.orchestra.run.smithi191.stdout:(109/166): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-02-26T04:44:30.047 INFO:teuthology.orchestra.run.smithi191.stdout:(110/166): libarrow-doc-9.0.0-10.el9.noarch.rpm 457 kB/s | 26 kB 00:00 2024-02-26T04:44:30.072 INFO:teuthology.orchestra.run.smithi191.stdout:(111/166): liboath-2.6.7-2.el9.x86_64.rpm 593 kB/s | 49 kB 00:00 2024-02-26T04:44:30.164 INFO:teuthology.orchestra.run.smithi191.stdout:(112/166): libarrow-9.0.0-10.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-02-26T04:44:30.189 INFO:teuthology.orchestra.run.smithi191.stdout:(113/166): libunwind-1.6.2-1.el9.x86_64.rpm 477 kB/s | 67 kB 00:00 2024-02-26T04:44:30.214 INFO:teuthology.orchestra.run.smithi191.stdout:(114/166): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-02-26T04:44:30.249 INFO:teuthology.orchestra.run.smithi191.stdout:(115/166): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-02-26T04:44:30.291 INFO:teuthology.orchestra.run.smithi191.stdout:(116/166): python3-bcrypt-3.2.2-1.el9.x86_64.rp 588 kB/s | 43 kB 00:00 2024-02-26T04:44:30.315 INFO:teuthology.orchestra.run.smithi191.stdout:(117/166): python3-asyncssh-2.13.2-4.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-02-26T04:44:30.332 INFO:teuthology.orchestra.run.smithi191.stdout:(118/166): python3-cachetools-4.2.4-1.el9.noarc 384 kB/s | 32 kB 00:00 2024-02-26T04:44:30.334 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-02-26T04:44:30.358 INFO:teuthology.orchestra.run.smithi191.stdout:(119/166): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-02-26T04:44:30.399 INFO:teuthology.orchestra.run.smithi191.stdout:(120/166): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-02-26T04:44:30.424 INFO:teuthology.orchestra.run.smithi191.stdout:(121/166): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-02-26T04:44:30.450 INFO:teuthology.orchestra.run.smithi191.stdout:(122/166): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-02-26T04:44:30.467 INFO:teuthology.orchestra.run.smithi191.stdout:(123/166): python3-colorama-0.4.4-12.el9.noarch 530 kB/s | 35 kB 00:00 2024-02-26T04:44:30.492 INFO:teuthology.orchestra.run.smithi191.stdout:(124/166): python3-google-auth-2.28.0-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-02-26T04:44:30.559 INFO:teuthology.orchestra.run.smithi191.stdout:(125/166): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-02-26T04:44:30.609 INFO:teuthology.orchestra.run.smithi191.stdout:(126/166): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-02-26T04:44:30.635 INFO:teuthology.orchestra.run.smithi191.stdout:(127/166): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-02-26T04:44:30.654 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-02-26T04:44:30.654 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-02-26T04:44:30.660 INFO:teuthology.orchestra.run.smithi191.stdout:(128/166): python3-jaraco-classes-3.2.1-5.el9.n 178 kB/s | 18 kB 00:00 2024-02-26T04:44:30.685 INFO:teuthology.orchestra.run.smithi191.stdout:(129/166): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-02-26T04:44:30.710 INFO:teuthology.orchestra.run.smithi191.stdout:(130/166): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-02-26T04:44:30.736 INFO:teuthology.orchestra.run.smithi191.stdout:(131/166): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-02-26T04:44:30.769 INFO:teuthology.orchestra.run.smithi191.stdout:(132/166): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-02-26T04:44:30.796 INFO:teuthology.orchestra.run.smithi191.stdout:(133/166): python3-logutils-0.3.5-21.el9.noarch 553 kB/s | 46 kB 00:00 2024-02-26T04:44:30.813 INFO:teuthology.orchestra.run.smithi156.stdout:(8/166): ceph-osd-19.0.0-1514.g25296a8f.el9.x86 14 MB/s | 17 MB 00:01 2024-02-26T04:44:30.828 INFO:teuthology.orchestra.run.smithi191.stdout:(134/166): python3-more-itertools-8.12.0-2.el9. 872 kB/s | 79 kB 00:00 2024-02-26T04:44:30.854 INFO:teuthology.orchestra.run.smithi191.stdout:(135/166): python3-natsort-7.1.1-5.el9.noarch.r 687 kB/s | 58 kB 00:00 2024-02-26T04:44:30.871 INFO:teuthology.orchestra.run.smithi156.stdout:(9/166): ceph-selinux-19.0.0-1514.g25296a8f.el9 434 kB/s | 25 kB 00:00 2024-02-26T04:44:30.887 INFO:teuthology.orchestra.run.smithi191.stdout:(136/166): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-02-26T04:44:30.912 INFO:teuthology.orchestra.run.smithi191.stdout:(137/166): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-02-26T04:44:30.938 INFO:teuthology.orchestra.run.smithi191.stdout:(138/166): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-02-26T04:44:30.963 INFO:teuthology.orchestra.run.smithi191.stdout:(139/166): python3-pyperclip-1.8.2-2.el9.noarch 376 kB/s | 28 kB 00:00 2024-02-26T04:44:30.988 INFO:teuthology.orchestra.run.smithi191.stdout:(140/166): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-02-26T04:44:31.014 INFO:teuthology.orchestra.run.smithi191.stdout:(141/166): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-02-26T04:44:31.039 INFO:teuthology.orchestra.run.smithi191.stdout:(142/166): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-02-26T04:44:31.064 INFO:teuthology.orchestra.run.smithi191.stdout:(143/166): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-02-26T04:44:31.108 INFO:teuthology.orchestra.run.smithi191.stdout:(144/166): python3-typing-extensions-4.4.0-2.el 766 kB/s | 51 kB 00:00 2024-02-26T04:44:31.140 INFO:teuthology.orchestra.run.smithi191.stdout:(145/166): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-02-26T04:44:31.165 INFO:teuthology.orchestra.run.smithi191.stdout:(146/166): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-02-26T04:44:31.215 INFO:teuthology.orchestra.run.smithi191.stdout:(147/166): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-02-26T04:44:31.241 INFO:teuthology.orchestra.run.smithi191.stdout:(148/166): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-02-26T04:44:31.265 INFO:teuthology.orchestra.run.smithi191.stdout:(149/166): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-02-26T04:44:31.432 INFO:teuthology.orchestra.run.smithi191.stdout:(150/166): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-02-26T04:44:31.975 INFO:teuthology.orchestra.run.smithi191.stdout:(151/166): NetworkManager-1.45.91-1.el9.x86_64. 4.2 MB/s | 2.3 MB 00:00 2024-02-26T04:44:32.260 INFO:teuthology.orchestra.run.smithi191.stdout:(152/166): NetworkManager-libnm-1.45.91-1.el9.x 6.4 MB/s | 1.8 MB 00:00 2024-02-26T04:44:32.325 INFO:teuthology.orchestra.run.smithi191.stdout:(153/166): NetworkManager-team-1.45.91-1.el9.x8 640 kB/s | 42 kB 00:00 2024-02-26T04:44:32.400 INFO:teuthology.orchestra.run.smithi191.stdout:(154/166): NetworkManager-tui-1.45.91-1.el9.x86 3.2 MB/s | 247 kB 00:00 2024-02-26T04:44:32.450 INFO:teuthology.orchestra.run.smithi191.stdout:(155/166): librados2-19.0.0-1514.g25296a8f.el9. 2.8 MB/s | 3.4 MB 00:01 2024-02-26T04:44:32.484 INFO:teuthology.orchestra.run.smithi191.stdout:(156/166): acl-2.3.1-4.el9.x86_64.rpm 855 kB/s | 71 kB 00:00 2024-02-26T04:44:32.526 INFO:teuthology.orchestra.run.smithi191.stdout:(157/166): elfutils-debuginfod-client-0.190-2.e 484 kB/s | 36 kB 00:00 2024-02-26T04:44:32.560 INFO:teuthology.orchestra.run.smithi191.stdout:(158/166): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-02-26T04:44:32.627 INFO:teuthology.orchestra.run.smithi191.stdout:(159/166): libacl-2.3.1-4.el9.x86_64.rpm 346 kB/s | 23 kB 00:00 2024-02-26T04:44:32.660 INFO:teuthology.orchestra.run.smithi191.stdout:(160/166): elfutils-libs-0.190-2.el9.x86_64.rpm 1.9 MB/s | 256 kB 00:00 2024-02-26T04:44:32.710 INFO:teuthology.orchestra.run.smithi191.stdout:(161/166): librbd1-19.0.0-1514.g25296a8f.el9.x8 2.2 MB/s | 3.1 MB 00:01 2024-02-26T04:44:32.844 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-02-26T04:44:32.844 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-02-26T04:44:32.852 INFO:teuthology.orchestra.run.smithi191.stdout:(162/166): openssl-3.0.7-25.el9.x86_64.rpm 5.4 MB/s | 1.2 MB 00:00 2024-02-26T04:44:32.936 INFO:teuthology.orchestra.run.smithi191.stdout:(163/166): xfsdump-3.1.12-4.el9.x86_64.rpm 1.4 MB/s | 323 kB 00:00 2024-02-26T04:44:32.995 INFO:teuthology.orchestra.run.smithi191.stdout:(164/166): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 74 kB 00:00 2024-02-26T04:44:33.028 INFO:teuthology.orchestra.run.smithi191.stdout:(165/166): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.1 MB/s | 1.1 MB 00:00 2024-02-26T04:44:33.595 INFO:teuthology.orchestra.run.smithi191.stdout:(166/166): openssl-libs-3.0.7-25.el9.x86_64.rpm 2.3 MB/s | 2.2 MB 00:00 2024-02-26T04:44:33.599 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-26T04:44:33.599 INFO:teuthology.orchestra.run.smithi191.stdout:Total 14 MB/s | 221 MB 00:15 2024-02-26T04:44:34.863 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-26T04:44:35.124 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-26T04:44:35.124 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-26T04:44:35.438 INFO:teuthology.orchestra.run.smithi156.stdout:(10/166): ceph-common-19.0.0-1514.g25296a8f.el9 2.5 MB/s | 20 MB 00:07 2024-02-26T04:44:35.696 INFO:teuthology.orchestra.run.smithi156.stdout:(11/166): cephfs-mirror-19.0.0-1514.g25296a8f.e 843 kB/s | 217 kB 00:00 2024-02-26T04:44:35.822 INFO:teuthology.orchestra.run.smithi156.stdout:(12/166): cephfs-shell-19.0.0-1514.g25296a8f.el 217 kB/s | 27 kB 00:00 2024-02-26T04:44:35.930 INFO:teuthology.orchestra.run.smithi156.stdout:(13/166): libcephfs-devel-19.0.0-1514.g25296a8f 288 kB/s | 31 kB 00:00 2024-02-26T04:44:36.356 INFO:teuthology.orchestra.run.smithi156.stdout:(14/166): libcephfs2-19.0.0-1514.g25296a8f.el9. 1.7 MB/s | 743 kB 00:00 2024-02-26T04:44:36.472 INFO:teuthology.orchestra.run.smithi156.stdout:(15/166): libcephsqlite-19.0.0-1514.g25296a8f.e 1.4 MB/s | 163 kB 00:00 2024-02-26T04:44:36.589 INFO:teuthology.orchestra.run.smithi156.stdout:(16/166): librados-devel-19.0.0-1514.g25296a8f. 1.1 MB/s | 127 kB 00:00 2024-02-26T04:44:36.916 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-26T04:44:36.917 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-26T04:44:37.015 INFO:teuthology.orchestra.run.smithi156.stdout:(17/166): ceph-test-19.0.0-1514.g25296a8f.el9.x 7.7 MB/s | 47 MB 00:06 2024-02-26T04:44:37.049 INFO:teuthology.orchestra.run.smithi156.stdout:(18/166): libradosstriper1-19.0.0-1514.g25296a8 1.1 MB/s | 498 kB 00:00 2024-02-26T04:44:37.149 INFO:teuthology.orchestra.run.smithi156.stdout:(19/166): python3-ceph-argparse-19.0.0-1514.g25 450 kB/s | 45 kB 00:00 2024-02-26T04:44:37.266 INFO:teuthology.orchestra.run.smithi156.stdout:(20/166): python3-ceph-common-19.0.0-1514.g2529 1.1 MB/s | 131 kB 00:00 2024-02-26T04:44:37.449 INFO:teuthology.orchestra.run.smithi156.stdout:(21/166): ceph-radosgw-19.0.0-1514.g25296a8f.el 1.3 MB/s | 9.7 MB 00:07 2024-02-26T04:44:37.475 INFO:teuthology.orchestra.run.smithi156.stdout:(22/166): python3-cephfs-19.0.0-1514.g25296a8f. 774 kB/s | 161 kB 00:00 2024-02-26T04:44:37.716 INFO:teuthology.orchestra.run.smithi156.stdout:(23/166): python3-rbd-19.0.0-1514.g25296a8f.el9 1.2 MB/s | 298 kB 00:00 2024-02-26T04:44:37.850 INFO:teuthology.orchestra.run.smithi156.stdout:(24/166): python3-rgw-19.0.0-1514.g25296a8f.el9 745 kB/s | 99 kB 00:00 2024-02-26T04:44:37.925 INFO:teuthology.orchestra.run.smithi156.stdout:(25/166): librgw2-19.0.0-1514.g25296a8f.el9.x86 5.6 MB/s | 5.1 MB 00:00 2024-02-26T04:44:37.951 INFO:teuthology.orchestra.run.smithi156.stdout:(26/166): python3-rados-19.0.0-1514.g25296a8f.e 639 kB/s | 320 kB 00:00 2024-02-26T04:44:37.976 INFO:teuthology.orchestra.run.smithi156.stdout:(27/166): rbd-fuse-19.0.0-1514.g25296a8f.el9.x8 680 kB/s | 85 kB 00:00 2024-02-26T04:44:38.101 INFO:teuthology.orchestra.run.smithi156.stdout:(28/166): ceph-grafana-dashboards-19.0.0-1514.g 251 kB/s | 31 kB 00:00 2024-02-26T04:44:38.184 INFO:teuthology.orchestra.run.smithi156.stdout:(29/166): rbd-nbd-19.0.0-1514.g25296a8f.el9.x86 731 kB/s | 171 kB 00:00 2024-02-26T04:44:38.243 INFO:teuthology.orchestra.run.smithi156.stdout:(30/166): ceph-mgr-cephadm-19.0.0-1514.g25296a8 973 kB/s | 138 kB 00:00 2024-02-26T04:44:38.460 INFO:teuthology.orchestra.run.smithi156.stdout:(31/166): rbd-mirror-19.0.0-1514.g25296a8f.el9. 5.8 MB/s | 3.1 MB 00:00 2024-02-26T04:44:38.552 INFO:teuthology.orchestra.run.smithi156.stdout:(32/166): ceph-mgr-modules-core-19.0.0-1514.g25 2.6 MB/s | 247 kB 00:00 2024-02-26T04:44:38.652 INFO:teuthology.orchestra.run.smithi156.stdout:(33/166): ceph-mgr-rook-19.0.0-1514.g25296a8f.e 523 kB/s | 52 kB 00:00 2024-02-26T04:44:38.736 INFO:teuthology.orchestra.run.smithi156.stdout:(34/166): ceph-prometheus-alerts-19.0.0-1514.g2 184 kB/s | 15 kB 00:00 2024-02-26T04:44:38.828 INFO:teuthology.orchestra.run.smithi156.stdout:(35/166): ceph-volume-19.0.0-1514.g25296a8f.el9 2.8 MB/s | 264 kB 00:00 2024-02-26T04:44:38.995 INFO:teuthology.orchestra.run.smithi156.stdout:(36/166): cephadm-19.0.0-1514.g25296a8f.el9.noa 4.4 MB/s | 748 kB 00:00 2024-02-26T04:44:39.087 INFO:teuthology.orchestra.run.smithi156.stdout:(37/166): cephfs-top-19.0.0-1514.g25296a8f.el9. 199 kB/s | 18 kB 00:00 2024-02-26T04:44:39.170 INFO:teuthology.orchestra.run.smithi156.stdout:(38/166): libaio-devel-0.3.112-1.el8.x86_64.rpm 212 kB/s | 18 kB 00:00 2024-02-26T04:44:39.296 INFO:teuthology.orchestra.run.smithi156.stdout:(39/166): fuse-2.9.9-15.el9.x86_64.rpm 644 kB/s | 80 kB 00:00 2024-02-26T04:44:39.387 INFO:teuthology.orchestra.run.smithi156.stdout:(40/166): jq-1.6-16.el9.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2024-02-26T04:44:39.730 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-02-26T04:44:39.738 INFO:teuthology.orchestra.run.smithi156.stdout:(41/166): ceph-mgr-dashboard-19.0.0-1514.g25296 1.1 MB/s | 1.8 MB 00:01 2024-02-26T04:44:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/182 2024-02-26T04:44:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/182 2024-02-26T04:44:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/182 2024-02-26T04:44:39.964 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/182 2024-02-26T04:44:39.990 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/182 2024-02-26T04:44:40.044 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/182 2024-02-26T04:44:40.399 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/182 2024-02-26T04:44:40.448 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T04:44:40.570 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T04:44:40.599 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T04:44:40.674 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T04:44:40.679 INFO:teuthology.orchestra.run.smithi156.stdout:(42/166): libquadmath-11.4.1-3.el9.x86_64.rpm 207 kB/s | 195 kB 00:00 2024-02-26T04:44:40.701 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T04:44:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/182 2024-02-26T04:44:40.881 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/182 2024-02-26T04:44:41.064 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/182 2024-02-26T04:44:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/182 2024-02-26T04:44:41.139 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T04:44:41.227 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T04:44:41.256 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T04:44:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T04:44:41.332 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T04:44:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T04:44:41.463 INFO:teuthology.orchestra.run.smithi156.stdout:(43/166): oniguruma-6.9.6-1.el9.5.x86_64.rpm 278 kB/s | 218 kB 00:00 2024-02-26T04:44:41.586 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T04:44:41.611 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T04:44:41.663 INFO:teuthology.orchestra.run.smithi156.stdout:(44/166): ceph-mgr-diskprediction-local-19.0.0- 2.2 MB/s | 7.4 MB 00:03 2024-02-26T04:44:41.779 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 19/182 2024-02-26T04:44:42.021 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T04:44:42.082 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T04:44:42.167 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T04:44:42.220 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T04:44:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T04:44:42.554 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : re2-1:20211101-3.el9.x86_64 22/182 2024-02-26T04:44:42.649 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 23/182 2024-02-26T04:44:42.800 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 24/182 2024-02-26T04:44:42.835 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : liboath-2.6.7-2.el9.x86_64 25/182 2024-02-26T04:44:42.847 INFO:teuthology.orchestra.run.smithi156.stdout:(45/166): python3-cffi-1.14.5-5.el9.x86_64.rpm 183 kB/s | 253 kB 00:01 2024-02-26T04:44:42.909 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/182 2024-02-26T04:44:42.942 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/182 2024-02-26T04:44:42.947 INFO:teuthology.orchestra.run.smithi156.stdout:(46/166): libgfortran-11.4.1-3.el9.x86_64.rpm 226 kB/s | 803 kB 00:03 2024-02-26T04:44:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/182 2024-02-26T04:44:43.027 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/182 2024-02-26T04:44:43.054 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/182 2024-02-26T04:44:43.097 INFO:teuthology.orchestra.run.smithi156.stdout:(47/166): python3-ply-3.11-14.el9.noarch.rpm 426 kB/s | 106 kB 00:00 2024-02-26T04:44:43.126 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/182 2024-02-26T04:44:43.178 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 32/182 2024-02-26T04:44:43.207 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/182 2024-02-26T04:44:43.243 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/182 2024-02-26T04:44:43.320 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/182 2024-02-26T04:44:43.354 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 36/182 2024-02-26T04:44:43.414 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 37/182 2024-02-26T04:44:43.427 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-26T04:44:43.442 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 38/182 2024-02-26T04:44:43.514 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/182 2024-02-26T04:44:43.532 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/182 2024-02-26T04:44:43.557 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : luarocks-3.9.2-1.el9.noarch 40/182 2024-02-26T04:44:43.570 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/182 2024-02-26T04:44:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/182 2024-02-26T04:44:43.633 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/182 2024-02-26T04:44:43.656 INFO:teuthology.orchestra.run.smithi156.stdout:(48/166): python3-pycparser-2.20-6.el9.noarch.r 191 kB/s | 135 kB 00:00 2024-02-26T04:44:43.657 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/182 2024-02-26T04:44:43.691 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/182 2024-02-26T04:44:43.734 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/182 2024-02-26T04:44:43.739 INFO:teuthology.orchestra.run.smithi156.stdout:(49/166): python3-requests-2.25.1-8.el9.noarch. 194 kB/s | 125 kB 00:00 2024-02-26T04:44:43.769 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/182 2024-02-26T04:44:43.910 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/182 2024-02-26T04:44:44.060 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/182 2024-02-26T04:44:44.107 INFO:teuthology.orchestra.run.smithi156.stdout:(50/166): python3-urllib3-1.26.5-5.el9.noarch.r 476 kB/s | 215 kB 00:00 2024-02-26T04:44:44.109 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T04:44:44.229 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T04:44:44.260 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T04:44:44.342 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T04:44:44.365 INFO:teuthology.orchestra.run.smithi156.stdout:(51/166): python3-cryptography-36.0.1-4.el9.x86 465 kB/s | 1.2 MB 00:02 2024-02-26T04:44:44.373 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T04:44:44.448 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/182 2024-02-26T04:44:44.461 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/182 2024-02-26T04:44:44.473 INFO:teuthology.orchestra.run.smithi156.stdout:(52/166): automake-1.16.2-8.el9.noarch.rpm 939 kB/s | 689 kB 00:00 2024-02-26T04:44:44.533 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/182 2024-02-26T04:44:44.557 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/182 2024-02-26T04:44:44.574 INFO:teuthology.orchestra.run.smithi156.stdout:(53/166): elfutils-libelf-devel-0.190-2.el9.x86 215 kB/s | 21 kB 00:00 2024-02-26T04:44:44.590 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 46/182 2024-02-26T04:44:44.626 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 47/182 2024-02-26T04:44:44.667 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 48/182 2024-02-26T04:44:44.693 INFO:teuthology.orchestra.run.smithi156.stdout:(54/166): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-02-26T04:44:44.698 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 49/182 2024-02-26T04:44:44.709 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/182 2024-02-26T04:44:44.741 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/182 2024-02-26T04:44:44.756 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/182 2024-02-26T04:44:44.784 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T04:44:44.810 INFO:teuthology.orchestra.run.smithi156.stdout:(55/166): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-02-26T04:44:44.835 INFO:teuthology.orchestra.run.smithi156.stdout:(56/166): boost-program-options-1.75.0-8.el9.x8 227 kB/s | 107 kB 00:00 2024-02-26T04:44:44.886 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T04:44:44.907 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 51/182 2024-02-26T04:44:44.910 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T04:44:44.919 INFO:teuthology.orchestra.run.smithi156.stdout:(57/166): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-02-26T04:44:44.953 INFO:teuthology.orchestra.run.smithi156.stdout:(58/166): flexiblas-openblas-openmp-3.0.4-8.el9 155 kB/s | 18 kB 00:00 2024-02-26T04:44:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 52/182 2024-02-26T04:44:44.981 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T04:44:45.003 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T04:44:45.018 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T04:44:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/182 2024-02-26T04:44:45.053 INFO:teuthology.orchestra.run.smithi156.stdout:(59/166): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-02-26T04:44:45.087 INFO:teuthology.orchestra.run.smithi156.stdout:(60/166): librabbitmq-0.11.0-7.el9.x86_64.rpm 338 kB/s | 45 kB 00:00 2024-02-26T04:44:45.099 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/182 2024-02-26T04:44:45.135 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/182 2024-02-26T04:44:45.172 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/182 2024-02-26T04:44:45.211 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T04:44:45.228 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/182 2024-02-26T04:44:45.240 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T04:44:45.263 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/182 2024-02-26T04:44:45.307 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/182 2024-02-26T04:44:45.345 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 19/182 2024-02-26T04:44:45.349 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/182 2024-02-26T04:44:45.452 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T04:44:45.489 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/182 2024-02-26T04:44:45.510 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T04:44:45.529 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/182 2024-02-26T04:44:45.559 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 63/182 2024-02-26T04:44:45.562 INFO:teuthology.orchestra.run.smithi156.stdout:(61/166): libtool-2.4.6-45.el9.x86_64.rpm 1.2 MB/s | 582 kB 00:00 2024-02-26T04:44:45.586 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T04:44:45.601 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 64/182 2024-02-26T04:44:45.637 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T04:44:45.697 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 65/182 2024-02-26T04:44:45.704 INFO:teuthology.orchestra.run.smithi156.stdout:(62/166): libuuid-devel-2.37.4-15.el9.x86_64.rp 180 kB/s | 26 kB 00:00 2024-02-26T04:44:45.734 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 66/182 2024-02-26T04:44:45.776 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 67/182 2024-02-26T04:44:45.807 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 68/182 2024-02-26T04:44:45.829 INFO:teuthology.orchestra.run.smithi156.stdout:(63/166): libzstd-devel-1.5.1-2.el9.x86_64.rpm 380 kB/s | 47 kB 00:00 2024-02-26T04:44:45.833 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 69/182 2024-02-26T04:44:45.864 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T04:44:45.879 INFO:teuthology.orchestra.run.smithi156.stdout:(64/166): librdkafka-1.6.1-102.el9.x86_64.rpm 802 kB/s | 662 kB 00:00 2024-02-26T04:44:45.915 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 70/182 2024-02-26T04:44:45.963 INFO:teuthology.orchestra.run.smithi156.stdout:(65/166): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-02-26T04:44:45.975 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : re2-1:20211101-3.el9.x86_64 22/182 2024-02-26T04:44:46.005 INFO:teuthology.orchestra.run.smithi156.stdout:(66/166): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-02-26T04:44:46.045 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 71/182 2024-02-26T04:44:46.062 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 23/182 2024-02-26T04:44:46.080 INFO:teuthology.orchestra.run.smithi156.stdout:(67/166): openblas-0.3.21-2.el9.x86_64.rpm 301 kB/s | 35 kB 00:00 2024-02-26T04:44:46.081 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 72/182 2024-02-26T04:44:46.108 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 24/182 2024-02-26T04:44:46.110 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/182 2024-02-26T04:44:46.143 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : liboath-2.6.7-2.el9.x86_64 25/182 2024-02-26T04:44:46.148 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/182 2024-02-26T04:44:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/182 2024-02-26T04:44:46.206 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/182 2024-02-26T04:44:46.223 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/182 2024-02-26T04:44:46.242 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/182 2024-02-26T04:44:46.252 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/182 2024-02-26T04:44:46.285 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/182 2024-02-26T04:44:46.287 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/182 2024-02-26T04:44:46.318 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/182 2024-02-26T04:44:46.320 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/182 2024-02-26T04:44:46.346 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/182 2024-02-26T04:44:46.362 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/182 2024-02-26T04:44:46.396 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/182 2024-02-26T04:44:46.413 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/182 2024-02-26T04:44:46.468 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 32/182 2024-02-26T04:44:46.485 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/182 2024-02-26T04:44:46.507 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/182 2024-02-26T04:44:46.527 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/182 2024-02-26T04:44:46.542 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/182 2024-02-26T04:44:46.566 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/182 2024-02-26T04:44:46.614 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/182 2024-02-26T04:44:46.617 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/182 2024-02-26T04:44:46.653 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 36/182 2024-02-26T04:44:46.689 INFO:teuthology.orchestra.run.smithi156.stdout:(68/166): openssl-devel-3.0.7-25.el9.x86_64.rpm 6.8 MB/s | 4.1 MB 00:00 2024-02-26T04:44:46.696 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 37/182 2024-02-26T04:44:46.724 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-packaging-20.9-5.el9.noarch 86/182 2024-02-26T04:44:46.734 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 38/182 2024-02-26T04:44:46.777 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/182 2024-02-26T04:44:46.802 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/182 2024-02-26T04:44:46.848 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : luarocks-3.9.2-1.el9.noarch 40/182 2024-02-26T04:44:46.921 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/182 2024-02-26T04:44:47.053 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/182 2024-02-26T04:44:47.117 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/182 2024-02-26T04:44:47.542 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/182 2024-02-26T04:44:47.598 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/182 2024-02-26T04:44:47.632 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/182 2024-02-26T04:44:47.634 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/182 2024-02-26T04:44:47.713 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/182 2024-02-26T04:44:47.734 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/182 2024-02-26T04:44:47.740 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 46/182 2024-02-26T04:44:47.770 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 47/182 2024-02-26T04:44:47.782 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 92/182 2024-02-26T04:44:47.819 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 48/182 2024-02-26T04:44:47.865 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 49/182 2024-02-26T04:44:47.873 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/182 2024-02-26T04:44:47.915 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/182 2024-02-26T04:44:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 94/182 2024-02-26T04:44:48.042 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 51/182 2024-02-26T04:44:48.100 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/182 2024-02-26T04:44:48.111 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 52/182 2024-02-26T04:44:48.140 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/182 2024-02-26T04:44:48.164 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/182 2024-02-26T04:44:48.220 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/182 2024-02-26T04:44:48.237 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/182 2024-02-26T04:44:48.277 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/182 2024-02-26T04:44:48.313 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/182 2024-02-26T04:44:48.360 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/182 2024-02-26T04:44:48.396 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/182 2024-02-26T04:44:48.439 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/182 2024-02-26T04:44:48.490 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/182 2024-02-26T04:44:48.622 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/182 2024-02-26T04:44:48.897 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/182 2024-02-26T04:44:48.914 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/182 2024-02-26T04:44:48.952 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 63/182 2024-02-26T04:44:48.994 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 64/182 2024-02-26T04:44:49.098 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 65/182 2024-02-26T04:44:49.145 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 66/182 2024-02-26T04:44:49.166 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/182 2024-02-26T04:44:49.183 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 67/182 2024-02-26T04:44:49.208 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 68/182 2024-02-26T04:44:49.233 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 69/182 2024-02-26T04:44:49.307 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 70/182 2024-02-26T04:44:49.364 INFO:teuthology.orchestra.run.smithi156.stdout:(69/166): openblas-openmp-0.3.21-2.el9.x86_64.r 1.4 MB/s | 4.7 MB 00:03 2024-02-26T04:44:49.441 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 71/182 2024-02-26T04:44:49.482 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 72/182 2024-02-26T04:44:49.510 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/182 2024-02-26T04:44:49.544 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/182 2024-02-26T04:44:49.583 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/182 2024-02-26T04:44:49.614 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/182 2024-02-26T04:44:49.644 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/182 2024-02-26T04:44:49.679 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/182 2024-02-26T04:44:49.712 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/182 2024-02-26T04:44:49.756 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/182 2024-02-26T04:44:49.780 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/182 2024-02-26T04:44:49.866 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/182 2024-02-26T04:44:49.918 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/182 2024-02-26T04:44:49.958 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/182 2024-02-26T04:44:50.005 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/182 2024-02-26T04:44:50.103 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-packaging-20.9-5.el9.noarch 86/182 2024-02-26T04:44:50.151 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/182 2024-02-26T04:44:50.485 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/182 2024-02-26T04:44:50.569 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/182 2024-02-26T04:44:50.589 INFO:teuthology.orchestra.run.smithi156.stdout:(70/166): postgresql-contrib-13.11-1.el9.x86_64 739 kB/s | 905 kB 00:01 2024-02-26T04:44:50.597 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/182 2024-02-26T04:44:50.630 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/182 2024-02-26T04:44:50.869 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/182 2024-02-26T04:44:50.897 INFO:teuthology.orchestra.run.smithi156.stdout:(71/166): postgresql-private-libs-13.11-1.el9.x 438 kB/s | 135 kB 00:00 2024-02-26T04:44:50.915 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/182 2024-02-26T04:44:50.954 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/182 2024-02-26T04:44:50.957 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/182 2024-02-26T04:44:50.985 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T04:44:51.059 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/182 2024-02-26T04:44:51.076 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T04:44:51.106 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 106/182 2024-02-26T04:44:51.121 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 92/182 2024-02-26T04:44:51.145 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/182 2024-02-26T04:44:51.210 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/182 2024-02-26T04:44:51.231 INFO:teuthology.orchestra.run.smithi156.stdout:(72/166): flexiblas-netlib-3.0.4-8.el9.x86_64.r 477 kB/s | 3.0 MB 00:06 2024-02-26T04:44:51.242 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 94/182 2024-02-26T04:44:51.420 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/182 2024-02-26T04:44:51.457 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/182 2024-02-26T04:44:51.542 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/182 2024-02-26T04:44:51.624 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/182 2024-02-26T04:44:51.655 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T04:44:51.709 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T04:44:51.740 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 110/182 2024-02-26T04:44:52.031 INFO:teuthology.orchestra.run.smithi156.stdout:(73/166): protobuf-3.14.0-13.el9.x86_64.rpm 1.3 MB/s | 1.0 MB 00:00 2024-02-26T04:44:52.148 INFO:teuthology.orchestra.run.smithi156.stdout:(74/166): python3-attrs-20.3.0-7.el9.noarch.rpm 750 kB/s | 87 kB 00:00 2024-02-26T04:44:52.184 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/182 2024-02-26T04:44:52.498 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/182 2024-02-26T04:44:53.548 INFO:teuthology.orchestra.run.smithi156.stdout:(75/166): postgresql-13.11-1.el9.x86_64.rpm 236 kB/s | 1.6 MB 00:06 2024-02-26T04:44:53.623 INFO:teuthology.orchestra.run.smithi156.stdout:(76/166): python3-babel-2.9.1-2.el9.noarch.rpm 4.0 MB/s | 6.0 MB 00:01 2024-02-26T04:44:53.657 INFO:teuthology.orchestra.run.smithi156.stdout:(77/166): python3-devel-3.9.17-2.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-02-26T04:44:53.765 INFO:teuthology.orchestra.run.smithi156.stdout:(78/166): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-02-26T04:44:53.796 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/182 2024-02-26T04:44:53.866 INFO:teuthology.orchestra.run.smithi156.stdout:(79/166): python3-jsonpointer-2.0-4.el9.noarch. 192 kB/s | 19 kB 00:00 2024-02-26T04:44:53.886 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/182 2024-02-26T04:44:53.891 INFO:teuthology.orchestra.run.smithi156.stdout:(80/166): python3-jinja2-2.11.3-5.el9.noarch.rp 927 kB/s | 248 kB 00:00 2024-02-26T04:44:53.914 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/182 2024-02-26T04:44:53.947 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/182 2024-02-26T04:44:53.964 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T04:44:53.975 INFO:teuthology.orchestra.run.smithi156.stdout:(81/166): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-02-26T04:44:54.000 INFO:teuthology.orchestra.run.smithi156.stdout:(82/166): python3-markupsafe-1.1.1-12.el9.x86_6 319 kB/s | 35 kB 00:00 2024-02-26T04:44:54.002 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T04:44:54.254 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/182 2024-02-26T04:44:54.285 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T04:44:54.292 INFO:teuthology.orchestra.run.smithi156.stdout:(83/166): python3-numpy-f2py-1.20.1-5.el9.x86_6 859 kB/s | 250 kB 00:00 2024-02-26T04:44:54.374 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T04:44:54.406 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 106/182 2024-02-26T04:44:54.417 INFO:teuthology.orchestra.run.smithi156.stdout:(84/166): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-02-26T04:44:54.442 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/182 2024-02-26T04:44:54.518 INFO:teuthology.orchestra.run.smithi156.stdout:(85/166): python3-oauthlib-3.1.1-5.el9.noarch.r 986 kB/s | 222 kB 00:00 2024-02-26T04:44:54.543 INFO:teuthology.orchestra.run.smithi156.stdout:(86/166): python3-packaging-20.9-5.el9.noarch.r 617 kB/s | 77 kB 00:00 2024-02-26T04:44:54.582 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T04:44:54.623 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T04:44:54.626 INFO:teuthology.orchestra.run.smithi156.stdout:(87/166): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-02-26T04:44:54.660 INFO:teuthology.orchestra.run.smithi156.stdout:(88/166): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-02-26T04:44:54.699 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T04:44:54.699 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-26T04:44:54.700 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-26T04:44:54.700 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:44:54.741 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T04:44:54.785 INFO:teuthology.orchestra.run.smithi156.stdout:(89/166): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-02-26T04:44:54.827 INFO:teuthology.orchestra.run.smithi156.stdout:(90/166): python3-pyasn1-0.4.8-6.el9.noarch.rpm 791 kB/s | 159 kB 00:00 2024-02-26T04:44:54.899 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/182 2024-02-26T04:44:54.902 INFO:teuthology.orchestra.run.smithi156.stdout:(91/166): python3-pytz-2021.1-5.el9.noarch.rpm 436 kB/s | 51 kB 00:00 2024-02-26T04:44:54.929 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T04:44:54.953 INFO:teuthology.orchestra.run.smithi156.stdout:(92/166): python3-requests-oauthlib-1.3.0-12.el 428 kB/s | 54 kB 00:00 2024-02-26T04:44:54.986 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T04:44:55.008 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 110/182 2024-02-26T04:44:55.094 INFO:teuthology.orchestra.run.smithi156.stdout:(93/166): python3-toml-0.10.2-6.el9.noarch.rpm 294 kB/s | 42 kB 00:00 2024-02-26T04:44:55.203 INFO:teuthology.orchestra.run.smithi156.stdout:(94/166): python3-wcwidth-0.2.5-8.el9.noarch.rp 407 kB/s | 44 kB 00:00 2024-02-26T04:44:55.428 INFO:teuthology.orchestra.run.smithi156.stdout:(95/166): qatlib-23.11.0-1.el9.x86_64.rpm 963 kB/s | 216 kB 00:00 2024-02-26T04:44:55.695 INFO:teuthology.orchestra.run.smithi156.stdout:(96/166): postgresql-server-13.11-1.el9.x86_64. 1.2 MB/s | 5.7 MB 00:04 2024-02-26T04:44:55.721 INFO:teuthology.orchestra.run.smithi156.stdout:(97/166): qatlib-service-23.11.0-1.el9.x86_64.r 119 kB/s | 35 kB 00:00 2024-02-26T04:44:55.829 INFO:teuthology.orchestra.run.smithi156.stdout:(98/166): qatzip-libs-1.1.2-2.el9.x86_64.rpm 334 kB/s | 44 kB 00:00 2024-02-26T04:44:56.023 INFO:teuthology.orchestra.run.smithi156.stdout:(99/166): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 305 kB 00:00 2024-02-26T04:44:56.047 INFO:teuthology.orchestra.run.smithi156.stdout:(100/166): uuid-1.6.2-55.el9.x86_64.rpm 266 kB/s | 58 kB 00:00 2024-02-26T04:44:56.155 INFO:teuthology.orchestra.run.smithi156.stdout:(101/166): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 714 kB/s | 94 kB 00:00 2024-02-26T04:44:56.180 INFO:teuthology.orchestra.run.smithi156.stdout:(102/166): xmlstarlet-1.6.1-20.el9.x86_64.rpm 477 kB/s | 64 kB 00:00 2024-02-26T04:44:56.297 INFO:teuthology.orchestra.run.smithi156.stdout:(103/166): zlib-devel-1.2.11-40.el9.x86_64.rpm 318 kB/s | 45 kB 00:00 2024-02-26T04:44:56.364 INFO:teuthology.orchestra.run.smithi156.stdout:(104/166): lua-devel-5.4.4-4.el9.x86_64.rpm 121 kB/s | 22 kB 00:00 2024-02-26T04:44:56.890 INFO:teuthology.orchestra.run.smithi156.stdout:(105/166): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-02-26T04:44:56.940 INFO:teuthology.orchestra.run.smithi156.stdout:(106/166): abseil-cpp-20211102.0-3.el9.x86_64.r 952 kB/s | 548 kB 00:00 2024-02-26T04:44:57.040 INFO:teuthology.orchestra.run.smithi156.stdout:(107/166): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-02-26T04:44:57.199 INFO:teuthology.orchestra.run.smithi156.stdout:(108/166): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-02-26T04:44:57.217 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T04:44:57.249 INFO:teuthology.orchestra.run.smithi156.stdout:(109/166): libarrow-doc-9.0.0-10.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-02-26T04:44:57.254 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T04:44:57.316 INFO:teuthology.orchestra.run.smithi156.stdout:(110/166): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-02-26T04:44:57.336 INFO:teuthology.orchestra.run.smithi156.stdout:(111/166): liboath-2.6.7-2.el9.x86_64.rpm 580 kB/s | 49 kB 00:00 2024-02-26T04:44:57.367 INFO:teuthology.orchestra.run.smithi156.stdout:(112/166): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-02-26T04:44:57.400 INFO:teuthology.orchestra.run.smithi156.stdout:(113/166): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-02-26T04:44:57.451 INFO:teuthology.orchestra.run.smithi156.stdout:(114/166): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-02-26T04:44:57.526 INFO:teuthology.orchestra.run.smithi156.stdout:(115/166): python3-asyncssh-2.13.2-4.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-02-26T04:44:57.551 INFO:teuthology.orchestra.run.smithi156.stdout:(116/166): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-02-26T04:44:57.602 INFO:teuthology.orchestra.run.smithi156.stdout:(117/166): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-02-26T04:44:57.627 INFO:teuthology.orchestra.run.smithi156.stdout:(118/166): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-02-26T04:44:57.794 INFO:teuthology.orchestra.run.smithi156.stdout:(119/166): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.6 MB/s | 16 MB 00:02 2024-02-26T04:44:57.794 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T04:44:57.819 INFO:teuthology.orchestra.run.smithi156.stdout:(120/166): python3-cheroot-8.6.0-4.el9.noarch.r 792 kB/s | 172 kB 00:00 2024-02-26T04:44:57.841 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T04:44:57.844 INFO:teuthology.orchestra.run.smithi156.stdout:(121/166): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2024-02-26T04:44:57.878 INFO:teuthology.orchestra.run.smithi156.stdout:(122/166): python3-colorama-0.4.4-12.el9.noarch 606 kB/s | 35 kB 00:00 2024-02-26T04:44:57.922 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T04:44:57.922 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-26T04:44:57.922 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-26T04:44:57.922 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:44:57.951 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T04:44:57.961 INFO:teuthology.orchestra.run.smithi156.stdout:(123/166): python3-google-auth-2.28.0-1.el9.noa 1.7 MB/s | 203 kB 00:00 2024-02-26T04:44:58.028 INFO:teuthology.orchestra.run.smithi156.stdout:(124/166): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-02-26T04:44:58.054 INFO:teuthology.orchestra.run.smithi156.stdout:(125/166): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-02-26T04:44:58.087 INFO:teuthology.orchestra.run.smithi156.stdout:(126/166): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-02-26T04:44:58.113 INFO:teuthology.orchestra.run.smithi156.stdout:(127/166): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-02-26T04:44:58.138 INFO:teuthology.orchestra.run.smithi156.stdout:(128/166): python3-cmd2-2.4.2-2.el9.noarch.rpm 883 kB/s | 303 kB 00:00 2024-02-26T04:44:58.163 INFO:teuthology.orchestra.run.smithi156.stdout:(129/166): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-02-26T04:44:58.188 INFO:teuthology.orchestra.run.smithi156.stdout:(130/166): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-02-26T04:44:58.213 INFO:teuthology.orchestra.run.smithi156.stdout:(131/166): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-02-26T04:44:58.256 INFO:teuthology.orchestra.run.smithi156.stdout:(132/166): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-02-26T04:44:58.281 INFO:teuthology.orchestra.run.smithi156.stdout:(133/166): python3-logutils-0.3.5-21.el9.noarch 502 kB/s | 46 kB 00:00 2024-02-26T04:44:58.306 INFO:teuthology.orchestra.run.smithi156.stdout:(134/166): python3-more-itertools-8.12.0-2.el9. 854 kB/s | 79 kB 00:00 2024-02-26T04:44:58.331 INFO:teuthology.orchestra.run.smithi156.stdout:(135/166): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-02-26T04:44:58.356 INFO:teuthology.orchestra.run.smithi156.stdout:(136/166): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-02-26T04:44:58.382 INFO:teuthology.orchestra.run.smithi156.stdout:(137/166): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-02-26T04:44:58.399 INFO:teuthology.orchestra.run.smithi156.stdout:(138/166): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-02-26T04:44:58.433 INFO:teuthology.orchestra.run.smithi156.stdout:(139/166): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-02-26T04:44:58.458 INFO:teuthology.orchestra.run.smithi156.stdout:(140/166): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-02-26T04:44:58.483 INFO:teuthology.orchestra.run.smithi156.stdout:(141/166): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-02-26T04:44:58.508 INFO:teuthology.orchestra.run.smithi156.stdout:(142/166): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-02-26T04:44:58.533 INFO:teuthology.orchestra.run.smithi156.stdout:(143/166): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-02-26T04:44:58.559 INFO:teuthology.orchestra.run.smithi156.stdout:(144/166): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-02-26T04:44:58.601 INFO:teuthology.orchestra.run.smithi156.stdout:(145/166): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-02-26T04:44:58.626 INFO:teuthology.orchestra.run.smithi156.stdout:(146/166): python3-websocket-client-1.2.3-2.el9 971 kB/s | 90 kB 00:00 2024-02-26T04:44:58.651 INFO:teuthology.orchestra.run.smithi156.stdout:(147/166): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-02-26T04:44:58.718 INFO:teuthology.orchestra.run.smithi156.stdout:(148/166): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-02-26T04:44:59.093 INFO:teuthology.orchestra.run.smithi156.stdout:(149/166): re2-20211101-3.el9.x86_64.rpm 410 kB/s | 192 kB 00:00 2024-02-26T04:44:59.144 INFO:teuthology.orchestra.run.smithi156.stdout:(150/166): thrift-0.15.0-2.el9.x86_64.rpm 3.2 MB/s | 1.6 MB 00:00 2024-02-26T04:44:59.511 INFO:teuthology.orchestra.run.smithi156.stdout:(151/166): librbd1-19.0.0-1514.g25296a8f.el9.x8 7.5 MB/s | 3.1 MB 00:00 2024-02-26T04:44:59.778 INFO:teuthology.orchestra.run.smithi156.stdout:(152/166): librados2-19.0.0-1514.g25296a8f.el9. 3.2 MB/s | 3.4 MB 00:01 2024-02-26T04:44:59.878 INFO:teuthology.orchestra.run.smithi156.stdout:(153/166): NetworkManager-team-1.45.91-1.el9.x8 425 kB/s | 42 kB 00:00 2024-02-26T04:44:59.995 INFO:teuthology.orchestra.run.smithi156.stdout:(154/166): NetworkManager-tui-1.45.91-1.el9.x86 2.1 MB/s | 247 kB 00:00 2024-02-26T04:45:00.045 INFO:teuthology.orchestra.run.smithi156.stdout:(155/166): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-02-26T04:45:00.087 INFO:teuthology.orchestra.run.smithi156.stdout:(156/166): elfutils-debuginfod-client-0.190-2.e 878 kB/s | 36 kB 00:00 2024-02-26T04:45:00.154 INFO:teuthology.orchestra.run.smithi156.stdout:(157/166): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-02-26T04:45:00.221 INFO:teuthology.orchestra.run.smithi156.stdout:(158/166): elfutils-libs-0.190-2.el9.x86_64.rpm 3.8 MB/s | 256 kB 00:00 2024-02-26T04:45:00.263 INFO:teuthology.orchestra.run.smithi156.stdout:(159/166): libacl-2.3.1-4.el9.x86_64.rpm 556 kB/s | 23 kB 00:00 2024-02-26T04:45:00.471 INFO:teuthology.orchestra.run.smithi156.stdout:(160/166): openssl-3.0.7-25.el9.x86_64.rpm 5.8 MB/s | 1.2 MB 00:00 2024-02-26T04:45:00.605 INFO:teuthology.orchestra.run.smithi156.stdout:(161/166): NetworkManager-libnm-1.45.91-1.el9.x 1.7 MB/s | 1.8 MB 00:01 2024-02-26T04:45:00.788 INFO:teuthology.orchestra.run.smithi156.stdout:(162/166): xfsdump-3.1.12-4.el9.x86_64.rpm 1.7 MB/s | 323 kB 00:00 2024-02-26T04:45:00.830 INFO:teuthology.orchestra.run.smithi156.stdout:(163/166): openssl-libs-3.0.7-25.el9.x86_64.rpm 6.0 MB/s | 2.2 MB 00:00 2024-02-26T04:45:01.164 INFO:teuthology.orchestra.run.smithi156.stdout:(164/166): libacl-devel-2.3.1-4.el9.x86_64.rpm 223 kB/s | 74 kB 00:00 2024-02-26T04:45:01.373 INFO:teuthology.orchestra.run.smithi156.stdout:(165/166): xfsprogs-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 1.1 MB 00:00 2024-02-26T04:45:01.539 INFO:teuthology.orchestra.run.smithi156.stdout:(166/166): NetworkManager-1.45.91-1.el9.x86_64. 982 kB/s | 2.3 MB 00:02 2024-02-26T04:45:01.542 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-02-26T04:45:01.543 INFO:teuthology.orchestra.run.smithi156.stdout:Total 6.4 MB/s | 221 MB 00:34 2024-02-26T04:45:02.747 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-02-26T04:45:03.010 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-02-26T04:45:03.010 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-02-26T04:45:04.804 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-02-26T04:45:04.805 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-02-26T04:45:06.714 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T04:45:06.714 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /sys 2024-02-26T04:45:06.714 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /proc 2024-02-26T04:45:06.714 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /mnt 2024-02-26T04:45:06.714 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /var/tmp 2024-02-26T04:45:06.715 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /home 2024-02-26T04:45:06.715 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /root 2024-02-26T04:45:06.715 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /tmp 2024-02-26T04:45:06.715 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:45:06.895 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T04:45:06.910 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T04:45:06.910 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:06.910 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-26T04:45:06.910 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:45:08.448 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T04:45:08.473 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T04:45:08.473 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:08.473 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-26T04:45:08.473 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T04:45:08.473 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T04:45:08.474 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:45:08.698 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T04:45:08.721 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T04:45:08.721 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:08.721 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-26T04:45:08.721 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T04:45:08.722 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T04:45:08.722 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:45:09.191 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T04:45:09.213 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T04:45:09.214 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:09.214 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-26T04:45:09.214 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T04:45:09.214 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T04:45:09.214 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:45:09.322 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : automake-1.16.2-8.el9.noarch 118/182 2024-02-26T04:45:09.379 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ply-3.11-14.el9.noarch 119/182 2024-02-26T04:45:09.435 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 120/182 2024-02-26T04:45:09.628 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 121/182 2024-02-26T04:45:09.649 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T04:45:09.650 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /sys 2024-02-26T04:45:09.650 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /proc 2024-02-26T04:45:09.650 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /mnt 2024-02-26T04:45:09.650 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /var/tmp 2024-02-26T04:45:09.650 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /home 2024-02-26T04:45:09.650 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /root 2024-02-26T04:45:09.650 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /tmp 2024-02-26T04:45:09.650 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:45:09.675 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 122/182 2024-02-26T04:45:09.754 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/182 2024-02-26T04:45:09.820 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T04:45:09.845 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T04:45:09.845 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:09.845 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-26T04:45:09.845 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:45:09.855 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 124/182 2024-02-26T04:45:09.996 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 125/182 2024-02-26T04:45:10.040 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 126/182 2024-02-26T04:45:10.074 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 127/182 2024-02-26T04:45:10.106 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 128/182 2024-02-26T04:45:10.142 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : jq-1.6-16.el9.x86_64 129/182 2024-02-26T04:45:10.169 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fuse-2.9.9-15.el9.x86_64 130/182 2024-02-26T04:45:10.620 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T04:45:10.652 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T04:45:10.683 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 132/182 2024-02-26T04:45:10.737 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 133/182 2024-02-26T04:45:10.757 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T04:45:11.000 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T04:45:11.033 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T04:45:11.332 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T04:45:11.339 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-02-26T04:45:11.357 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T04:45:11.358 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:11.358 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-26T04:45:11.358 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T04:45:11.358 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T04:45:11.358 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:45:11.467 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/182 2024-02-26T04:45:11.505 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/182 2024-02-26T04:45:11.561 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/182 2024-02-26T04:45:11.574 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T04:45:11.596 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/182 2024-02-26T04:45:11.596 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T04:45:11.597 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:11.597 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-26T04:45:11.597 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T04:45:11.597 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T04:45:11.597 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:45:11.621 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/182 2024-02-26T04:45:11.663 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/182 2024-02-26T04:45:11.980 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/182 2024-02-26T04:45:12.038 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T04:45:12.041 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T04:45:12.046 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T04:45:12.063 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T04:45:12.063 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:12.064 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-26T04:45:12.064 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T04:45:12.064 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T04:45:12.064 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:45:12.075 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T04:45:12.160 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T04:45:12.163 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : automake-1.16.2-8.el9.noarch 118/182 2024-02-26T04:45:12.182 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T04:45:12.198 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T04:45:12.228 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ply-3.11-14.el9.noarch 119/182 2024-02-26T04:45:12.272 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T04:45:12.292 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 120/182 2024-02-26T04:45:12.302 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T04:45:12.345 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 137/182 2024-02-26T04:45:12.369 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T04:45:12.374 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/182 2024-02-26T04:45:12.392 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T04:45:12.392 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:12.392 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-26T04:45:12.392 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T04:45:12.392 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T04:45:12.393 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:45:12.437 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T04:45:12.448 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T04:45:12.453 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/182 2024-02-26T04:45:12.477 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 121/182 2024-02-26T04:45:12.525 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 122/182 2024-02-26T04:45:12.549 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 140/182 2024-02-26T04:45:12.602 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/182 2024-02-26T04:45:12.638 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/182 2024-02-26T04:45:12.685 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/182 2024-02-26T04:45:12.717 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 124/182 2024-02-26T04:45:12.718 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T04:45:12.809 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 15/182 2024-02-26T04:45:12.831 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T04:45:12.847 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 125/182 2024-02-26T04:45:12.886 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 16/182 2024-02-26T04:45:12.900 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 126/182 2024-02-26T04:45:12.907 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T04:45:12.923 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T04:45:12.933 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 127/182 2024-02-26T04:45:12.965 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 128/182 2024-02-26T04:45:13.000 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : jq-1.6-16.el9.x86_64 129/182 2024-02-26T04:45:13.028 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : fuse-2.9.9-15.el9.x86_64 130/182 2024-02-26T04:45:13.108 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T04:45:13.137 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T04:45:13.234 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 19/182 2024-02-26T04:45:13.339 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T04:45:13.370 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T04:45:13.382 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T04:45:13.401 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T04:45:13.433 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 132/182 2024-02-26T04:45:13.457 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T04:45:13.493 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 133/182 2024-02-26T04:45:13.517 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T04:45:13.524 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T04:45:13.722 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T04:45:13.755 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T04:45:13.784 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T04:45:13.830 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : re2-1:20211101-3.el9.x86_64 22/182 2024-02-26T04:45:13.926 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 23/182 2024-02-26T04:45:13.964 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 24/182 2024-02-26T04:45:13.999 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.7-2.el9.x86_64 25/182 2024-02-26T04:45:14.063 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/182 2024-02-26T04:45:14.097 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/182 2024-02-26T04:45:14.141 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/182 2024-02-26T04:45:14.175 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/182 2024-02-26T04:45:14.202 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/182 2024-02-26T04:45:14.271 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/182 2024-02-26T04:45:14.325 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 32/182 2024-02-26T04:45:14.364 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/182 2024-02-26T04:45:14.444 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/182 2024-02-26T04:45:14.526 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/182 2024-02-26T04:45:14.560 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 36/182 2024-02-26T04:45:14.604 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 37/182 2024-02-26T04:45:14.632 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 38/182 2024-02-26T04:45:14.702 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/182 2024-02-26T04:45:14.744 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T04:45:14.747 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : luarocks-3.9.2-1.el9.noarch 40/182 2024-02-26T04:45:14.784 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T04:45:14.821 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/182 2024-02-26T04:45:14.884 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T04:45:14.967 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/182 2024-02-26T04:45:15.024 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/182 2024-02-26T04:45:15.045 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 137/182 2024-02-26T04:45:15.077 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T04:45:15.103 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T04:45:15.103 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:15.103 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-26T04:45:15.103 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T04:45:15.104 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T04:45:15.104 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:45:15.145 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T04:45:15.156 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T04:45:15.254 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 140/182 2024-02-26T04:45:15.562 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/182 2024-02-26T04:45:15.653 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/182 2024-02-26T04:45:15.689 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 46/182 2024-02-26T04:45:15.718 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 47/182 2024-02-26T04:45:15.768 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 48/182 2024-02-26T04:45:15.806 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 49/182 2024-02-26T04:45:15.848 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/182 2024-02-26T04:45:15.991 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 51/182 2024-02-26T04:45:16.042 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 52/182 2024-02-26T04:45:16.106 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/182 2024-02-26T04:45:16.180 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/182 2024-02-26T04:45:16.210 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/182 2024-02-26T04:45:16.246 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/182 2024-02-26T04:45:16.292 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/182 2024-02-26T04:45:16.337 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/182 2024-02-26T04:45:16.380 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/182 2024-02-26T04:45:16.423 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/182 2024-02-26T04:45:16.556 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/182 2024-02-26T04:45:16.603 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/182 2024-02-26T04:45:16.633 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 63/182 2024-02-26T04:45:16.675 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 64/182 2024-02-26T04:45:16.768 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 65/182 2024-02-26T04:45:16.817 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 66/182 2024-02-26T04:45:16.857 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 67/182 2024-02-26T04:45:16.883 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 68/182 2024-02-26T04:45:16.907 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 69/182 2024-02-26T04:45:16.980 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 70/182 2024-02-26T04:45:17.108 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 71/182 2024-02-26T04:45:17.145 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 72/182 2024-02-26T04:45:17.173 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/182 2024-02-26T04:45:17.195 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 141/182 2024-02-26T04:45:17.207 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/182 2024-02-26T04:45:17.246 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/182 2024-02-26T04:45:17.268 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 142/182 2024-02-26T04:45:17.278 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/182 2024-02-26T04:45:17.308 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/182 2024-02-26T04:45:17.342 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/182 2024-02-26T04:45:17.375 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/182 2024-02-26T04:45:17.426 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/182 2024-02-26T04:45:17.460 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/182 2024-02-26T04:45:17.547 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/182 2024-02-26T04:45:17.590 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/182 2024-02-26T04:45:17.622 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/182 2024-02-26T04:45:17.669 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/182 2024-02-26T04:45:17.766 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-packaging-20.9-5.el9.noarch 86/182 2024-02-26T04:45:17.816 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/182 2024-02-26T04:45:18.186 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libtool-2.4.6-45.el9.x86_64 143/182 2024-02-26T04:45:18.218 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T04:45:18.240 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T04:45:18.241 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:18.241 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-26T04:45:18.241 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T04:45:18.241 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T04:45:18.241 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:45:18.301 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T04:45:18.323 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T04:45:18.323 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:18.323 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-26T04:45:18.323 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:45:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T04:45:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T04:45:18.400 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:18.400 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-26T04:45:18.400 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T04:45:18.400 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T04:45:18.400 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:45:18.544 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/182 2024-02-26T04:45:18.596 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/182 2024-02-26T04:45:18.630 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/182 2024-02-26T04:45:18.729 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/182 2024-02-26T04:45:18.736 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T04:45:18.759 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T04:45:18.759 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:18.759 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-26T04:45:18.759 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T04:45:18.759 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T04:45:18.759 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:45:18.797 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 92/182 2024-02-26T04:45:18.811 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 148/182 2024-02-26T04:45:18.853 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 149/182 2024-02-26T04:45:18.885 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/182 2024-02-26T04:45:18.886 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 150/182 2024-02-26T04:45:18.911 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 151/182 2024-02-26T04:45:18.915 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 94/182 2024-02-26T04:45:19.092 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/182 2024-02-26T04:45:19.112 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 152/182 2024-02-26T04:45:19.122 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/182 2024-02-26T04:45:19.158 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 153/182 2024-02-26T04:45:19.190 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 154/182 2024-02-26T04:45:19.199 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/182 2024-02-26T04:45:19.231 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 155/182 2024-02-26T04:45:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 156/182 2024-02-26T04:45:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T04:45:19.391 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 158/182 2024-02-26T04:45:19.426 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 159/182 2024-02-26T04:45:19.626 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 141/182 2024-02-26T04:45:19.688 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 142/182 2024-02-26T04:45:19.840 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/182 2024-02-26T04:45:19.951 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/182 2024-02-26T04:45:20.025 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T04:45:20.057 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T04:45:20.092 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T04:45:20.146 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/182 2024-02-26T04:45:20.544 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libtool-2.4.6-45.el9.x86_64 143/182 2024-02-26T04:45:20.576 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T04:45:20.599 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T04:45:20.599 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:20.599 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-26T04:45:20.599 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T04:45:20.600 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T04:45:20.600 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:45:20.643 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T04:45:20.665 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T04:45:20.665 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:20.665 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-26T04:45:20.666 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:45:20.710 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T04:45:20.736 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T04:45:20.736 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:20.737 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-26T04:45:20.737 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T04:45:20.737 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T04:45:20.737 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:45:21.016 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 162/182 2024-02-26T04:45:21.045 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T04:45:21.067 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T04:45:21.067 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:21.068 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-26T04:45:21.068 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T04:45:21.068 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T04:45:21.068 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:45:21.119 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 148/182 2024-02-26T04:45:21.119 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 163/182 2024-02-26T04:45:21.161 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/182 2024-02-26T04:45:21.161 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 149/182 2024-02-26T04:45:21.193 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 150/182 2024-02-26T04:45:21.206 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flex-2.6.4-9.el9.x86_64 164/182 2024-02-26T04:45:21.220 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 151/182 2024-02-26T04:45:21.235 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/182 2024-02-26T04:45:21.239 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : bison-3.7.4-5.el9.x86_64 165/182 2024-02-26T04:45:21.263 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/182 2024-02-26T04:45:21.264 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 166/182 2024-02-26T04:45:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 167/182 2024-02-26T04:45:21.296 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/182 2024-02-26T04:45:21.316 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/182 2024-02-26T04:45:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 169/182 2024-02-26T04:45:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T04:45:21.399 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 152/182 2024-02-26T04:45:21.429 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T04:45:21.441 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 153/182 2024-02-26T04:45:21.459 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 171/182 2024-02-26T04:45:21.474 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 154/182 2024-02-26T04:45:21.484 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 172/182 2024-02-26T04:45:21.506 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 155/182 2024-02-26T04:45:21.510 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 173/182 2024-02-26T04:45:21.535 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 174/182 2024-02-26T04:45:21.535 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T04:45:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 156/182 2024-02-26T04:45:21.589 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T04:45:21.604 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/182 2024-02-26T04:45:21.637 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T04:45:21.641 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T04:45:21.641 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 158/182 2024-02-26T04:45:21.671 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 176/182 2024-02-26T04:45:21.676 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 159/182 2024-02-26T04:45:21.678 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T04:45:21.678 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T04:45:21.707 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 105/182 2024-02-26T04:45:21.739 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 106/182 2024-02-26T04:45:21.776 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/182 2024-02-26T04:45:21.858 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T04:45:21.858 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T04:45:21.933 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T04:45:21.960 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/182 2024-02-26T04:45:21.983 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 180/182 2024-02-26T04:45:22.016 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 181/182 2024-02-26T04:45:22.084 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T04:45:22.213 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T04:45:22.231 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/182 2024-02-26T04:45:22.248 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T04:45:22.254 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T04:45:22.275 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T04:45:22.309 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 109/182 2024-02-26T04:45:22.340 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 110/182 2024-02-26T04:45:22.346 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/182 2024-02-26T04:45:23.174 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 162/182 2024-02-26T04:45:23.261 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 163/182 2024-02-26T04:45:23.340 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flex-2.6.4-9.el9.x86_64 164/182 2024-02-26T04:45:23.373 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : bison-3.7.4-5.el9.x86_64 165/182 2024-02-26T04:45:23.407 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 166/182 2024-02-26T04:45:23.432 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 167/182 2024-02-26T04:45:23.457 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/182 2024-02-26T04:45:23.492 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 169/182 2024-02-26T04:45:23.492 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T04:45:23.588 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T04:45:23.618 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 171/182 2024-02-26T04:45:23.643 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 172/182 2024-02-26T04:45:23.669 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 173/182 2024-02-26T04:45:23.694 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 174/182 2024-02-26T04:45:23.694 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T04:45:23.761 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T04:45:23.787 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 176/182 2024-02-26T04:45:23.793 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T04:45:23.793 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T04:45:23.981 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T04:45:23.981 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T04:45:24.048 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T04:45:24.072 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/182 2024-02-26T04:45:24.099 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 180/182 2024-02-26T04:45:24.141 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 181/182 2024-02-26T04:45:24.204 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T04:45:24.529 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T04:45:24.569 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T04:45:25.113 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 111/182 2024-02-26T04:45:25.157 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T04:45:25.239 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 112/182 2024-02-26T04:45:25.240 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-26T04:45:25.240 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-26T04:45:25.240 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:45:25.258 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T04:45:27.084 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T04:45:27.084 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 1/182 2024-02-26T04:45:27.084 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2/182 2024-02-26T04:45:27.085 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 3/182 2024-02-26T04:45:27.085 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 4/182 2024-02-26T04:45:27.085 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1514.g25296 5/182 2024-02-26T04:45:27.085 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 6/182 2024-02-26T04:45:27.085 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 7/182 2024-02-26T04:45:27.085 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T04:45:27.085 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T04:45:27.085 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T04:45:27.085 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 11/182 2024-02-26T04:45:27.085 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 12/182 2024-02-26T04:45:27.085 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 13/182 2024-02-26T04:45:27.085 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 14/182 2024-02-26T04:45:27.086 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 15/182 2024-02-26T04:45:27.086 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 16/182 2024-02-26T04:45:27.086 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 17/182 2024-02-26T04:45:27.086 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 18/182 2024-02-26T04:45:27.086 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 19/182 2024-02-26T04:45:27.086 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 20/182 2024-02-26T04:45:27.086 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 21/182 2024-02-26T04:45:27.086 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 22/182 2024-02-26T04:45:27.086 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 23/182 2024-02-26T04:45:27.086 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 24/182 2024-02-26T04:45:27.086 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 25/182 2024-02-26T04:45:27.086 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 26/182 2024-02-26T04:45:27.087 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 27/182 2024-02-26T04:45:27.087 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 28/182 2024-02-26T04:45:27.087 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 29/182 2024-02-26T04:45:27.087 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 30/182 2024-02-26T04:45:27.087 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 31/182 2024-02-26T04:45:27.087 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 32/182 2024-02-26T04:45:27.087 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 33/182 2024-02-26T04:45:27.087 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 34/182 2024-02-26T04:45:27.087 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 35/182 2024-02-26T04:45:27.087 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 36/182 2024-02-26T04:45:27.087 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 37/182 2024-02-26T04:45:27.087 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 38/182 2024-02-26T04:45:27.087 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 39/182 2024-02-26T04:45:27.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/182 2024-02-26T04:45:27.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/182 2024-02-26T04:45:27.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : jq-1.6-16.el9.x86_64 42/182 2024-02-26T04:45:27.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/182 2024-02-26T04:45:27.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/182 2024-02-26T04:45:27.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/182 2024-02-26T04:45:27.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/182 2024-02-26T04:45:27.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/182 2024-02-26T04:45:27.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/182 2024-02-26T04:45:27.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/182 2024-02-26T04:45:27.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/182 2024-02-26T04:45:27.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/182 2024-02-26T04:45:27.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/182 2024-02-26T04:45:27.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/182 2024-02-26T04:45:27.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/182 2024-02-26T04:45:27.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/182 2024-02-26T04:45:27.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/182 2024-02-26T04:45:27.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/182 2024-02-26T04:45:27.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/182 2024-02-26T04:45:27.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/182 2024-02-26T04:45:27.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/182 2024-02-26T04:45:27.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/182 2024-02-26T04:45:27.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/182 2024-02-26T04:45:27.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/182 2024-02-26T04:45:27.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/182 2024-02-26T04:45:27.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/182 2024-02-26T04:45:27.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/182 2024-02-26T04:45:27.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/182 2024-02-26T04:45:27.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/182 2024-02-26T04:45:27.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/182 2024-02-26T04:45:27.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/182 2024-02-26T04:45:27.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/182 2024-02-26T04:45:27.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/182 2024-02-26T04:45:27.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/182 2024-02-26T04:45:27.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/182 2024-02-26T04:45:27.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/182 2024-02-26T04:45:27.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/182 2024-02-26T04:45:27.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/182 2024-02-26T04:45:27.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/182 2024-02-26T04:45:27.091 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/182 2024-02-26T04:45:27.091 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/182 2024-02-26T04:45:27.091 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/182 2024-02-26T04:45:27.091 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/182 2024-02-26T04:45:27.091 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/182 2024-02-26T04:45:27.091 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/182 2024-02-26T04:45:27.091 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/182 2024-02-26T04:45:27.091 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/182 2024-02-26T04:45:27.091 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/182 2024-02-26T04:45:27.091 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/182 2024-02-26T04:45:27.091 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/182 2024-02-26T04:45:27.091 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/182 2024-02-26T04:45:27.091 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/182 2024-02-26T04:45:27.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/182 2024-02-26T04:45:27.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/182 2024-02-26T04:45:27.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/182 2024-02-26T04:45:27.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/182 2024-02-26T04:45:27.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/182 2024-02-26T04:45:27.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/182 2024-02-26T04:45:27.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/182 2024-02-26T04:45:27.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/182 2024-02-26T04:45:27.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/182 2024-02-26T04:45:27.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/182 2024-02-26T04:45:27.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/182 2024-02-26T04:45:27.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/182 2024-02-26T04:45:27.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/182 2024-02-26T04:45:27.093 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/182 2024-02-26T04:45:27.093 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/182 2024-02-26T04:45:27.093 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/182 2024-02-26T04:45:27.093 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/182 2024-02-26T04:45:27.093 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/182 2024-02-26T04:45:27.093 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/182 2024-02-26T04:45:27.093 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/182 2024-02-26T04:45:27.093 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/182 2024-02-26T04:45:27.093 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/182 2024-02-26T04:45:27.093 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/182 2024-02-26T04:45:27.093 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/182 2024-02-26T04:45:27.093 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/182 2024-02-26T04:45:27.093 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/182 2024-02-26T04:45:27.093 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/182 2024-02-26T04:45:27.094 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/182 2024-02-26T04:45:27.094 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/182 2024-02-26T04:45:27.094 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/182 2024-02-26T04:45:27.094 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/182 2024-02-26T04:45:27.094 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/182 2024-02-26T04:45:27.094 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/182 2024-02-26T04:45:27.094 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/182 2024-02-26T04:45:27.094 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/182 2024-02-26T04:45:27.094 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/182 2024-02-26T04:45:27.094 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/182 2024-02-26T04:45:27.095 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/182 2024-02-26T04:45:27.095 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/182 2024-02-26T04:45:27.095 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/182 2024-02-26T04:45:27.095 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/182 2024-02-26T04:45:27.095 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/182 2024-02-26T04:45:27.095 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/182 2024-02-26T04:45:27.095 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/182 2024-02-26T04:45:27.095 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/182 2024-02-26T04:45:27.095 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/182 2024-02-26T04:45:27.095 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/182 2024-02-26T04:45:27.095 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/182 2024-02-26T04:45:27.095 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/182 2024-02-26T04:45:27.095 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/182 2024-02-26T04:45:27.096 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/182 2024-02-26T04:45:27.096 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/182 2024-02-26T04:45:27.096 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/182 2024-02-26T04:45:27.096 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/182 2024-02-26T04:45:27.096 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/182 2024-02-26T04:45:27.096 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/182 2024-02-26T04:45:27.096 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/182 2024-02-26T04:45:27.096 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/182 2024-02-26T04:45:27.096 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/182 2024-02-26T04:45:27.096 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 151/182 2024-02-26T04:45:27.096 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/182 2024-02-26T04:45:27.096 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 153/182 2024-02-26T04:45:27.096 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/182 2024-02-26T04:45:27.097 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 155/182 2024-02-26T04:45:27.097 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/182 2024-02-26T04:45:27.097 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T04:45:27.097 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/182 2024-02-26T04:45:27.097 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 159/182 2024-02-26T04:45:27.097 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/182 2024-02-26T04:45:27.097 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 161/182 2024-02-26T04:45:27.097 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/182 2024-02-26T04:45:27.097 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/182 2024-02-26T04:45:27.097 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/182 2024-02-26T04:45:27.097 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/182 2024-02-26T04:45:27.097 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/182 2024-02-26T04:45:27.097 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/182 2024-02-26T04:45:27.098 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/182 2024-02-26T04:45:27.098 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/182 2024-02-26T04:45:27.098 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/182 2024-02-26T04:45:27.098 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/182 2024-02-26T04:45:27.098 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/182 2024-02-26T04:45:27.098 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 173/182 2024-02-26T04:45:27.098 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 174/182 2024-02-26T04:45:27.098 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 175/182 2024-02-26T04:45:27.098 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 176/182 2024-02-26T04:45:27.098 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 177/182 2024-02-26T04:45:27.098 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 178/182 2024-02-26T04:45:27.098 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 179/182 2024-02-26T04:45:27.098 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 180/182 2024-02-26T04:45:27.098 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 181/182 2024-02-26T04:45:29.563 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T04:45:29.564 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 1/182 2024-02-26T04:45:29.564 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2/182 2024-02-26T04:45:29.564 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 3/182 2024-02-26T04:45:29.564 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 4/182 2024-02-26T04:45:29.564 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1514.g25296 5/182 2024-02-26T04:45:29.564 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 6/182 2024-02-26T04:45:29.564 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 7/182 2024-02-26T04:45:29.564 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T04:45:29.564 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T04:45:29.564 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T04:45:29.564 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 11/182 2024-02-26T04:45:29.565 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 12/182 2024-02-26T04:45:29.565 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 13/182 2024-02-26T04:45:29.565 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 14/182 2024-02-26T04:45:29.565 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 15/182 2024-02-26T04:45:29.565 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 16/182 2024-02-26T04:45:29.565 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 17/182 2024-02-26T04:45:29.565 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 18/182 2024-02-26T04:45:29.565 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 19/182 2024-02-26T04:45:29.565 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 20/182 2024-02-26T04:45:29.565 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 21/182 2024-02-26T04:45:29.565 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 22/182 2024-02-26T04:45:29.565 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 23/182 2024-02-26T04:45:29.565 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 24/182 2024-02-26T04:45:29.565 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 25/182 2024-02-26T04:45:29.566 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 26/182 2024-02-26T04:45:29.566 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 27/182 2024-02-26T04:45:29.566 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 28/182 2024-02-26T04:45:29.566 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 29/182 2024-02-26T04:45:29.566 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 30/182 2024-02-26T04:45:29.566 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 31/182 2024-02-26T04:45:29.566 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 32/182 2024-02-26T04:45:29.566 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 33/182 2024-02-26T04:45:29.566 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 34/182 2024-02-26T04:45:29.566 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 35/182 2024-02-26T04:45:29.566 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 36/182 2024-02-26T04:45:29.566 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 37/182 2024-02-26T04:45:29.566 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 38/182 2024-02-26T04:45:29.566 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 39/182 2024-02-26T04:45:29.567 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/182 2024-02-26T04:45:29.567 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/182 2024-02-26T04:45:29.567 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : jq-1.6-16.el9.x86_64 42/182 2024-02-26T04:45:29.567 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/182 2024-02-26T04:45:29.567 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/182 2024-02-26T04:45:29.567 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/182 2024-02-26T04:45:29.567 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/182 2024-02-26T04:45:29.567 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/182 2024-02-26T04:45:29.567 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/182 2024-02-26T04:45:29.567 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/182 2024-02-26T04:45:29.567 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/182 2024-02-26T04:45:29.567 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/182 2024-02-26T04:45:29.567 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/182 2024-02-26T04:45:29.567 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/182 2024-02-26T04:45:29.568 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/182 2024-02-26T04:45:29.568 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/182 2024-02-26T04:45:29.568 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/182 2024-02-26T04:45:29.568 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/182 2024-02-26T04:45:29.568 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/182 2024-02-26T04:45:29.568 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/182 2024-02-26T04:45:29.568 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/182 2024-02-26T04:45:29.568 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/182 2024-02-26T04:45:29.568 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/182 2024-02-26T04:45:29.568 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/182 2024-02-26T04:45:29.568 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/182 2024-02-26T04:45:29.568 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/182 2024-02-26T04:45:29.568 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/182 2024-02-26T04:45:29.568 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/182 2024-02-26T04:45:29.569 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/182 2024-02-26T04:45:29.569 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/182 2024-02-26T04:45:29.569 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/182 2024-02-26T04:45:29.569 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/182 2024-02-26T04:45:29.569 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/182 2024-02-26T04:45:29.569 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/182 2024-02-26T04:45:29.569 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/182 2024-02-26T04:45:29.569 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/182 2024-02-26T04:45:29.569 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/182 2024-02-26T04:45:29.569 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/182 2024-02-26T04:45:29.569 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/182 2024-02-26T04:45:29.569 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/182 2024-02-26T04:45:29.569 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/182 2024-02-26T04:45:29.569 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/182 2024-02-26T04:45:29.570 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/182 2024-02-26T04:45:29.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/182 2024-02-26T04:45:29.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/182 2024-02-26T04:45:29.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/182 2024-02-26T04:45:29.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/182 2024-02-26T04:45:29.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/182 2024-02-26T04:45:29.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/182 2024-02-26T04:45:29.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/182 2024-02-26T04:45:29.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/182 2024-02-26T04:45:29.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/182 2024-02-26T04:45:29.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/182 2024-02-26T04:45:29.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/182 2024-02-26T04:45:29.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/182 2024-02-26T04:45:29.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/182 2024-02-26T04:45:29.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/182 2024-02-26T04:45:29.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/182 2024-02-26T04:45:29.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/182 2024-02-26T04:45:29.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/182 2024-02-26T04:45:29.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/182 2024-02-26T04:45:29.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/182 2024-02-26T04:45:29.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/182 2024-02-26T04:45:29.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/182 2024-02-26T04:45:29.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/182 2024-02-26T04:45:29.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/182 2024-02-26T04:45:29.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/182 2024-02-26T04:45:29.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/182 2024-02-26T04:45:29.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/182 2024-02-26T04:45:29.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/182 2024-02-26T04:45:29.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/182 2024-02-26T04:45:29.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/182 2024-02-26T04:45:29.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/182 2024-02-26T04:45:29.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/182 2024-02-26T04:45:29.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/182 2024-02-26T04:45:29.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/182 2024-02-26T04:45:29.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/182 2024-02-26T04:45:29.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/182 2024-02-26T04:45:29.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/182 2024-02-26T04:45:29.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/182 2024-02-26T04:45:29.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/182 2024-02-26T04:45:29.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/182 2024-02-26T04:45:29.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/182 2024-02-26T04:45:29.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/182 2024-02-26T04:45:29.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 151/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/182 2024-02-26T04:45:29.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 153/182 2024-02-26T04:45:29.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/182 2024-02-26T04:45:29.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 155/182 2024-02-26T04:45:29.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/182 2024-02-26T04:45:29.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T04:45:29.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/182 2024-02-26T04:45:29.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 159/182 2024-02-26T04:45:29.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/182 2024-02-26T04:45:29.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 161/182 2024-02-26T04:45:29.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/182 2024-02-26T04:45:29.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/182 2024-02-26T04:45:29.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/182 2024-02-26T04:45:29.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/182 2024-02-26T04:45:29.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/182 2024-02-26T04:45:29.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/182 2024-02-26T04:45:29.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/182 2024-02-26T04:45:29.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/182 2024-02-26T04:45:29.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/182 2024-02-26T04:45:29.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/182 2024-02-26T04:45:29.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/182 2024-02-26T04:45:29.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 173/182 2024-02-26T04:45:29.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 174/182 2024-02-26T04:45:29.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 175/182 2024-02-26T04:45:29.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 176/182 2024-02-26T04:45:29.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 177/182 2024-02-26T04:45:29.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 178/182 2024-02-26T04:45:29.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 179/182 2024-02-26T04:45:29.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 180/182 2024-02-26T04:45:29.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 181/182 2024-02-26T04:45:29.806 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 182/182 2024-02-26T04:45:29.806 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:45:29.806 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2024-02-26T04:45:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-26T04:45:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-26T04:45:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-26T04:45:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-26T04:45:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: acl-2.3.1-4.el9.x86_64 2024-02-26T04:45:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-26T04:45:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-26T04:45:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-26T04:45:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-26T04:45:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-26T04:45:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-26T04:45:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-26T04:45:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-26T04:45:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-26T04:45:29.808 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-02-26T04:45:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-26T04:45:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: automake-1.16.2-8.el9.noarch 2024-02-26T04:45:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: bison-3.7.4-5.el9.x86_64 2024-02-26T04:45:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-26T04:45:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-26T04:45:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: flex-2.6.4-9.el9.x86_64 2024-02-26T04:45:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-26T04:45:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-26T04:45:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-26T04:45:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-26T04:45:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-26T04:45:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-26T04:45:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: jq-1.6-16.el9.x86_64 2024-02-26T04:45:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-26T04:45:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-26T04:45:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-26T04:45:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-26T04:45:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-26T04:45:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-26T04:45:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-26T04:45:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-26T04:45:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-26T04:45:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-26T04:45:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-26T04:45:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-26T04:45:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-26T04:45:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-26T04:45:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: lua-5.4.4-4.el9.x86_64 2024-02-26T04:45:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-26T04:45:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-26T04:45:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-26T04:45:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-26T04:45:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-26T04:45:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-26T04:45:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-26T04:45:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-13.11-1.el9.x86_64 2024-02-26T04:45:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-26T04:45:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-26T04:45:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-26T04:45:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-26T04:45:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-26T04:45:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-26T04:45:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-26T04:45:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-26T04:45:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-26T04:45:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-26T04:45:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-26T04:45:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-26T04:45:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-26T04:45:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-26T04:45:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-26T04:45:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-26T04:45:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-26T04:45:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-26T04:45:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-26T04:45:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-26T04:45:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-26T04:45:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-26T04:45:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-26T04:45:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-26T04:45:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-26T04:45:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-26T04:45:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-26T04:45:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-26T04:45:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-26T04:45:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-26T04:45:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-26T04:45:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-26T04:45:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-26T04:45:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-26T04:45:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-26T04:45:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-26T04:45:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-26T04:45:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-26T04:45:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-26T04:45:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-26T04:45:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply-3.11-14.el9.noarch 2024-02-26T04:45:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-26T04:45:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-26T04:45:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-26T04:45:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-26T04:45:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-26T04:45:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-26T04:45:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-26T04:45:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-26T04:45:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-26T04:45:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-26T04:45:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-26T04:45:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-26T04:45:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-26T04:45:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-26T04:45:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-26T04:45:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-26T04:45:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-26T04:45:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-26T04:45:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-26T04:45:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-26T04:45:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-26T04:45:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-26T04:45:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-26T04:45:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-26T04:45:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-26T04:45:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-26T04:45:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-26T04:45:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: re2-1:20211101-3.el9.x86_64 2024-02-26T04:45:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-26T04:45:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-26T04:45:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-26T04:45:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-26T04:45:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-26T04:45:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-26T04:45:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:45:29.820 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-02-26T04:45:30.298 DEBUG:teuthology.parallel:result is None 2024-02-26T04:45:32.249 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 182/182 2024-02-26T04:45:32.250 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:45:32.250 INFO:teuthology.orchestra.run.smithi191.stdout:Upgraded: 2024-02-26T04:45:32.250 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-26T04:45:32.250 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-26T04:45:32.250 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-26T04:45:32.250 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-26T04:45:32.250 INFO:teuthology.orchestra.run.smithi191.stdout: acl-2.3.1-4.el9.x86_64 2024-02-26T04:45:32.250 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-26T04:45:32.250 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-26T04:45:32.250 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-26T04:45:32.250 INFO:teuthology.orchestra.run.smithi191.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-26T04:45:32.250 INFO:teuthology.orchestra.run.smithi191.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-26T04:45:32.251 INFO:teuthology.orchestra.run.smithi191.stdout: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.251 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.251 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-26T04:45:32.251 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-26T04:45:32.251 INFO:teuthology.orchestra.run.smithi191.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-26T04:45:32.251 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-26T04:45:32.251 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-26T04:45:32.251 INFO:teuthology.orchestra.run.smithi191.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-26T04:45:32.251 INFO:teuthology.orchestra.run.smithi191.stdout: automake-1.16.2-8.el9.noarch 2024-02-26T04:45:32.251 INFO:teuthology.orchestra.run.smithi191.stdout: bison-3.7.4-5.el9.x86_64 2024-02-26T04:45:32.251 INFO:teuthology.orchestra.run.smithi191.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-26T04:45:32.251 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.251 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.251 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.252 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.252 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:32.252 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.252 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.252 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.252 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:32.252 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:32.252 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:32.252 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:32.252 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:32.252 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.252 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.252 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:32.253 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.253 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.253 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.253 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:32.253 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:32.253 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.253 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.253 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:32.253 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-26T04:45:32.253 INFO:teuthology.orchestra.run.smithi191.stdout: flex-2.6.4-9.el9.x86_64 2024-02-26T04:45:32.253 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-26T04:45:32.253 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-26T04:45:32.253 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-26T04:45:32.253 INFO:teuthology.orchestra.run.smithi191.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-26T04:45:32.254 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-26T04:45:32.254 INFO:teuthology.orchestra.run.smithi191.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-26T04:45:32.254 INFO:teuthology.orchestra.run.smithi191.stdout: jq-1.6-16.el9.x86_64 2024-02-26T04:45:32.254 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-26T04:45:32.254 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-26T04:45:32.254 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-26T04:45:32.254 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.254 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.254 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.254 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-26T04:45:32.254 INFO:teuthology.orchestra.run.smithi191.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-26T04:45:32.254 INFO:teuthology.orchestra.run.smithi191.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-26T04:45:32.254 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-26T04:45:32.254 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-26T04:45:32.255 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.255 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.255 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-26T04:45:32.255 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.255 INFO:teuthology.orchestra.run.smithi191.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-26T04:45:32.255 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-26T04:45:32.255 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-26T04:45:32.255 INFO:teuthology.orchestra.run.smithi191.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-26T04:45:32.255 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-26T04:45:32.255 INFO:teuthology.orchestra.run.smithi191.stdout: lua-5.4.4-4.el9.x86_64 2024-02-26T04:45:32.255 INFO:teuthology.orchestra.run.smithi191.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-26T04:45:32.255 INFO:teuthology.orchestra.run.smithi191.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-26T04:45:32.255 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-26T04:45:32.256 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-26T04:45:32.256 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-26T04:45:32.256 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-26T04:45:32.256 INFO:teuthology.orchestra.run.smithi191.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-26T04:45:32.256 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-13.11-1.el9.x86_64 2024-02-26T04:45:32.256 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-26T04:45:32.256 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-26T04:45:32.256 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-26T04:45:32.256 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-26T04:45:32.256 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-26T04:45:32.256 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-26T04:45:32.256 INFO:teuthology.orchestra.run.smithi191.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-26T04:45:32.256 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-26T04:45:32.256 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-26T04:45:32.257 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-26T04:45:32.257 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.257 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.257 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.257 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-26T04:45:32.257 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-26T04:45:32.257 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-26T04:45:32.257 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-26T04:45:32.257 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-26T04:45:32.257 INFO:teuthology.orchestra.run.smithi191.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-26T04:45:32.257 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-26T04:45:32.257 INFO:teuthology.orchestra.run.smithi191.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-26T04:45:32.257 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-26T04:45:32.257 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-26T04:45:32.258 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-26T04:45:32.258 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-26T04:45:32.258 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-26T04:45:32.258 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-26T04:45:32.258 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-26T04:45:32.258 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-26T04:45:32.258 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-26T04:45:32.258 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-26T04:45:32.258 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-26T04:45:32.258 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-26T04:45:32.258 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-26T04:45:32.258 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-26T04:45:32.258 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-26T04:45:32.258 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-26T04:45:32.259 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-26T04:45:32.259 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-26T04:45:32.259 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-26T04:45:32.259 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-26T04:45:32.259 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-26T04:45:32.259 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-26T04:45:32.259 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ply-3.11-14.el9.noarch 2024-02-26T04:45:32.259 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-26T04:45:32.259 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-26T04:45:32.259 INFO:teuthology.orchestra.run.smithi191.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-26T04:45:32.259 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-26T04:45:32.259 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-26T04:45:32.259 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-26T04:45:32.259 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-26T04:45:32.260 INFO:teuthology.orchestra.run.smithi191.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-26T04:45:32.261 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-26T04:45:32.261 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-26T04:45:32.261 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-26T04:45:32.261 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-26T04:45:32.261 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-26T04:45:32.261 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-26T04:45:32.261 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-26T04:45:32.261 INFO:teuthology.orchestra.run.smithi191.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-26T04:45:32.261 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.261 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.261 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:32.261 INFO:teuthology.orchestra.run.smithi191.stdout: re2-1:20211101-3.el9.x86_64 2024-02-26T04:45:32.261 INFO:teuthology.orchestra.run.smithi191.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-26T04:45:32.261 INFO:teuthology.orchestra.run.smithi191.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-26T04:45:32.262 INFO:teuthology.orchestra.run.smithi191.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-26T04:45:32.262 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-26T04:45:32.262 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-26T04:45:32.262 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-26T04:45:32.262 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:45:32.262 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-26T04:45:32.777 DEBUG:teuthology.parallel:result is None 2024-02-26T04:45:36.818 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 113/182 2024-02-26T04:45:36.818 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2024-02-26T04:45:36.818 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2024-02-26T04:45:36.818 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2024-02-26T04:45:36.818 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2024-02-26T04:45:36.819 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2024-02-26T04:45:36.819 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /root 2024-02-26T04:45:36.819 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2024-02-26T04:45:36.819 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:45:36.989 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T04:45:37.004 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 114/182 2024-02-26T04:45:37.004 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:37.004 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-26T04:45:37.004 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:45:38.475 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T04:45:38.500 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 115/182 2024-02-26T04:45:38.500 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:38.500 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-26T04:45:38.500 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T04:45:38.501 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T04:45:38.501 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:45:38.709 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T04:45:38.731 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 116/182 2024-02-26T04:45:38.731 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:38.732 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-26T04:45:38.732 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T04:45:38.732 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T04:45:38.732 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:45:39.176 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T04:45:39.199 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 117/182 2024-02-26T04:45:39.199 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:39.199 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-26T04:45:39.199 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T04:45:39.199 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T04:45:39.199 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:45:39.298 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : automake-1.16.2-8.el9.noarch 118/182 2024-02-26T04:45:39.364 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ply-3.11-14.el9.noarch 119/182 2024-02-26T04:45:39.420 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 120/182 2024-02-26T04:45:39.607 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 121/182 2024-02-26T04:45:39.661 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 122/182 2024-02-26T04:45:39.747 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/182 2024-02-26T04:45:39.844 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 124/182 2024-02-26T04:45:39.994 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 125/182 2024-02-26T04:45:40.028 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 126/182 2024-02-26T04:45:40.061 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 127/182 2024-02-26T04:45:40.092 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 128/182 2024-02-26T04:45:40.128 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : jq-1.6-16.el9.x86_64 129/182 2024-02-26T04:45:40.156 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.9-15.el9.x86_64 130/182 2024-02-26T04:45:40.565 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T04:45:40.596 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 131/182 2024-02-26T04:45:40.627 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 132/182 2024-02-26T04:45:40.688 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 133/182 2024-02-26T04:45:40.727 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T04:45:40.961 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 134/182 2024-02-26T04:45:40.995 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T04:45:41.963 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 135/182 2024-02-26T04:45:41.996 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T04:45:42.096 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 136/182 2024-02-26T04:45:42.258 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 137/182 2024-02-26T04:45:42.283 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T04:45:42.304 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 138/182 2024-02-26T04:45:42.304 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:42.304 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-26T04:45:42.304 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T04:45:42.305 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T04:45:42.305 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:45:42.357 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T04:45:42.367 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 139/182 2024-02-26T04:45:42.465 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 140/182 2024-02-26T04:45:46.896 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 141/182 2024-02-26T04:45:46.959 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 142/182 2024-02-26T04:45:47.848 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libtool-2.4.6-45.el9.x86_64 143/182 2024-02-26T04:45:47.880 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T04:45:47.903 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 144/182 2024-02-26T04:45:47.904 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:47.904 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-26T04:45:47.904 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T04:45:47.904 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T04:45:47.904 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:45:47.948 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T04:45:47.970 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1514.g25296 145/182 2024-02-26T04:45:47.970 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:47.970 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-26T04:45:47.970 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:45:48.015 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T04:45:48.038 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 146/182 2024-02-26T04:45:48.038 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:48.038 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-26T04:45:48.038 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T04:45:48.038 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T04:45:48.039 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:45:48.341 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T04:45:48.363 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 147/182 2024-02-26T04:45:48.363 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T04:45:48.363 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-26T04:45:48.363 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T04:45:48.363 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T04:45:48.363 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:45:48.416 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 148/182 2024-02-26T04:45:48.457 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 149/182 2024-02-26T04:45:48.489 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 150/182 2024-02-26T04:45:48.516 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 151/182 2024-02-26T04:45:48.699 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 152/182 2024-02-26T04:45:48.740 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 153/182 2024-02-26T04:45:48.773 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 154/182 2024-02-26T04:45:48.813 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 155/182 2024-02-26T04:45:48.843 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 156/182 2024-02-26T04:45:48.905 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T04:45:48.967 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 158/182 2024-02-26T04:45:49.000 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 159/182 2024-02-26T04:45:49.611 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T04:45:49.648 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T04:45:49.674 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T04:45:49.729 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/182 2024-02-26T04:45:50.566 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 162/182 2024-02-26T04:45:50.678 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 163/182 2024-02-26T04:45:50.759 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flex-2.6.4-9.el9.x86_64 164/182 2024-02-26T04:45:50.806 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : bison-3.7.4-5.el9.x86_64 165/182 2024-02-26T04:45:50.832 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 166/182 2024-02-26T04:45:50.866 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 167/182 2024-02-26T04:45:50.891 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/182 2024-02-26T04:45:50.926 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 169/182 2024-02-26T04:45:50.926 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T04:45:50.997 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T04:45:51.027 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 171/182 2024-02-26T04:45:51.052 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 172/182 2024-02-26T04:45:51.078 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 173/182 2024-02-26T04:45:51.102 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 174/182 2024-02-26T04:45:51.103 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T04:45:51.170 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T04:45:51.195 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 176/182 2024-02-26T04:45:51.204 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T04:45:51.204 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T04:45:51.390 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T04:45:51.390 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T04:45:51.475 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T04:45:51.498 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/182 2024-02-26T04:45:51.524 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 180/182 2024-02-26T04:45:51.558 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 181/182 2024-02-26T04:45:51.622 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T04:45:56.593 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T04:45:56.594 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 1/182 2024-02-26T04:45:56.594 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2/182 2024-02-26T04:45:56.594 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 3/182 2024-02-26T04:45:56.594 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 4/182 2024-02-26T04:45:56.594 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1514.g25296 5/182 2024-02-26T04:45:56.594 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 6/182 2024-02-26T04:45:56.594 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 7/182 2024-02-26T04:45:56.594 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 8/182 2024-02-26T04:45:56.594 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 9/182 2024-02-26T04:45:56.595 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 10/182 2024-02-26T04:45:56.595 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 11/182 2024-02-26T04:45:56.595 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 12/182 2024-02-26T04:45:56.595 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 13/182 2024-02-26T04:45:56.595 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 14/182 2024-02-26T04:45:56.595 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_ 15/182 2024-02-26T04:45:56.595 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 16/182 2024-02-26T04:45:56.595 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 17/182 2024-02-26T04:45:56.595 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_6 18/182 2024-02-26T04:45:56.595 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86 19/182 2024-02-26T04:45:56.595 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 20/182 2024-02-26T04:45:56.595 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el 21/182 2024-02-26T04:45:56.595 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:19.0.0-1514.g25296a8f.el9. 22/182 2024-02-26T04:45:56.596 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_6 23/182 2024-02-26T04:45:56.596 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 24/182 2024-02-26T04:45:56.596 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 25/182 2024-02-26T04:45:56.596 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 26/182 2024-02-26T04:45:56.596 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 27/182 2024-02-26T04:45:56.596 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 28/182 2024-02-26T04:45:56.596 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 29/182 2024-02-26T04:45:56.596 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f. 30/182 2024-02-26T04:45:56.596 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noa 31/182 2024-02-26T04:45:56.596 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.n 32/182 2024-02-26T04:45:56.596 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1514.g252 33/182 2024-02-26T04:45:56.596 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el 34/182 2024-02-26T04:45:56.596 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 35/182 2024-02-26T04:45:56.597 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.e 36/182 2024-02-26T04:45:56.597 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 37/182 2024-02-26T04:45:56.597 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 38/182 2024-02-26T04:45:56.597 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 39/182 2024-02-26T04:45:56.597 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/182 2024-02-26T04:45:56.597 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/182 2024-02-26T04:45:56.597 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : jq-1.6-16.el9.x86_64 42/182 2024-02-26T04:45:56.597 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/182 2024-02-26T04:45:56.597 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/182 2024-02-26T04:45:56.597 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/182 2024-02-26T04:45:56.597 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/182 2024-02-26T04:45:56.597 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/182 2024-02-26T04:45:56.597 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/182 2024-02-26T04:45:56.598 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/182 2024-02-26T04:45:56.598 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/182 2024-02-26T04:45:56.598 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/182 2024-02-26T04:45:56.598 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/182 2024-02-26T04:45:56.598 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/182 2024-02-26T04:45:56.598 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/182 2024-02-26T04:45:56.598 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/182 2024-02-26T04:45:56.598 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/182 2024-02-26T04:45:56.598 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/182 2024-02-26T04:45:56.598 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/182 2024-02-26T04:45:56.598 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/182 2024-02-26T04:45:56.598 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/182 2024-02-26T04:45:56.598 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/182 2024-02-26T04:45:56.599 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/182 2024-02-26T04:45:56.599 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/182 2024-02-26T04:45:56.599 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/182 2024-02-26T04:45:56.599 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/182 2024-02-26T04:45:56.599 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/182 2024-02-26T04:45:56.599 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/182 2024-02-26T04:45:56.599 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/182 2024-02-26T04:45:56.599 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/182 2024-02-26T04:45:56.599 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/182 2024-02-26T04:45:56.599 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/182 2024-02-26T04:45:56.599 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/182 2024-02-26T04:45:56.599 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/182 2024-02-26T04:45:56.599 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/182 2024-02-26T04:45:56.600 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/182 2024-02-26T04:45:56.600 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/182 2024-02-26T04:45:56.600 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/182 2024-02-26T04:45:56.600 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/182 2024-02-26T04:45:56.600 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/182 2024-02-26T04:45:56.600 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/182 2024-02-26T04:45:56.600 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/182 2024-02-26T04:45:56.600 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/182 2024-02-26T04:45:56.600 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/182 2024-02-26T04:45:56.600 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/182 2024-02-26T04:45:56.600 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/182 2024-02-26T04:45:56.600 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/182 2024-02-26T04:45:56.600 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/182 2024-02-26T04:45:56.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/182 2024-02-26T04:45:56.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/182 2024-02-26T04:45:56.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/182 2024-02-26T04:45:56.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/182 2024-02-26T04:45:56.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/182 2024-02-26T04:45:56.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/182 2024-02-26T04:45:56.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/182 2024-02-26T04:45:56.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/182 2024-02-26T04:45:56.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/182 2024-02-26T04:45:56.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/182 2024-02-26T04:45:56.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/182 2024-02-26T04:45:56.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/182 2024-02-26T04:45:56.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/182 2024-02-26T04:45:56.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/182 2024-02-26T04:45:56.602 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/182 2024-02-26T04:45:56.602 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/182 2024-02-26T04:45:56.602 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/182 2024-02-26T04:45:56.602 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/182 2024-02-26T04:45:56.602 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/182 2024-02-26T04:45:56.602 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/182 2024-02-26T04:45:56.602 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/182 2024-02-26T04:45:56.602 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/182 2024-02-26T04:45:56.602 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/182 2024-02-26T04:45:56.602 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/182 2024-02-26T04:45:56.602 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/182 2024-02-26T04:45:56.602 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/182 2024-02-26T04:45:56.602 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/182 2024-02-26T04:45:56.603 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/182 2024-02-26T04:45:56.603 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/182 2024-02-26T04:45:56.603 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/182 2024-02-26T04:45:56.603 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/182 2024-02-26T04:45:56.603 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/182 2024-02-26T04:45:56.603 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/182 2024-02-26T04:45:56.603 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/182 2024-02-26T04:45:56.603 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/182 2024-02-26T04:45:56.603 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/182 2024-02-26T04:45:56.603 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/182 2024-02-26T04:45:56.603 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/182 2024-02-26T04:45:56.603 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/182 2024-02-26T04:45:56.603 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/182 2024-02-26T04:45:56.603 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/182 2024-02-26T04:45:56.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/182 2024-02-26T04:45:56.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/182 2024-02-26T04:45:56.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/182 2024-02-26T04:45:56.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/182 2024-02-26T04:45:56.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/182 2024-02-26T04:45:56.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/182 2024-02-26T04:45:56.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/182 2024-02-26T04:45:56.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/182 2024-02-26T04:45:56.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/182 2024-02-26T04:45:56.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/182 2024-02-26T04:45:56.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/182 2024-02-26T04:45:56.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/182 2024-02-26T04:45:56.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/182 2024-02-26T04:45:56.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/182 2024-02-26T04:45:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/182 2024-02-26T04:45:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/182 2024-02-26T04:45:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/182 2024-02-26T04:45:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/182 2024-02-26T04:45:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/182 2024-02-26T04:45:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/182 2024-02-26T04:45:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/182 2024-02-26T04:45:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/182 2024-02-26T04:45:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 151/182 2024-02-26T04:45:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/182 2024-02-26T04:45:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 153/182 2024-02-26T04:45:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/182 2024-02-26T04:45:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 155/182 2024-02-26T04:45:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/182 2024-02-26T04:45:56.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T04:45:56.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/182 2024-02-26T04:45:56.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 159/182 2024-02-26T04:45:56.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/182 2024-02-26T04:45:56.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 161/182 2024-02-26T04:45:56.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/182 2024-02-26T04:45:56.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/182 2024-02-26T04:45:56.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/182 2024-02-26T04:45:56.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/182 2024-02-26T04:45:56.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/182 2024-02-26T04:45:56.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/182 2024-02-26T04:45:56.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/182 2024-02-26T04:45:56.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/182 2024-02-26T04:45:56.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/182 2024-02-26T04:45:56.607 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/182 2024-02-26T04:45:56.607 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/182 2024-02-26T04:45:56.607 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 173/182 2024-02-26T04:45:56.607 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 174/182 2024-02-26T04:45:56.607 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 175/182 2024-02-26T04:45:56.607 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 176/182 2024-02-26T04:45:56.607 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 177/182 2024-02-26T04:45:56.607 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 178/182 2024-02-26T04:45:56.607 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 179/182 2024-02-26T04:45:56.607 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 180/182 2024-02-26T04:45:56.607 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 181/182 2024-02-26T04:45:59.340 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 182/182 2024-02-26T04:45:59.340 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:45:59.340 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2024-02-26T04:45:59.340 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-26T04:45:59.340 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-26T04:45:59.341 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-26T04:45:59.341 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-26T04:45:59.341 INFO:teuthology.orchestra.run.smithi156.stdout: acl-2.3.1-4.el9.x86_64 2024-02-26T04:45:59.341 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-26T04:45:59.341 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-26T04:45:59.341 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-26T04:45:59.341 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-26T04:45:59.341 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-26T04:45:59.341 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.341 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.342 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-26T04:45:59.342 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-26T04:45:59.342 INFO:teuthology.orchestra.run.smithi156.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-26T04:45:59.342 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-26T04:45:59.342 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-02-26T04:45:59.342 INFO:teuthology.orchestra.run.smithi156.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-26T04:45:59.342 INFO:teuthology.orchestra.run.smithi156.stdout: automake-1.16.2-8.el9.noarch 2024-02-26T04:45:59.342 INFO:teuthology.orchestra.run.smithi156.stdout: bison-3.7.4-5.el9.x86_64 2024-02-26T04:45:59.342 INFO:teuthology.orchestra.run.smithi156.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-26T04:45:59.342 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.342 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.342 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.342 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.343 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:59.343 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.343 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.343 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.343 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:59.343 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:59.343 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:59.343 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:59.343 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:59.343 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.343 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.343 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:59.343 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.343 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.344 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.344 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:59.344 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:59.344 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.344 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-shell-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.344 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top-2:19.0.0-1514.g25296a8f.el9.noarch 2024-02-26T04:45:59.344 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-26T04:45:59.344 INFO:teuthology.orchestra.run.smithi156.stdout: flex-2.6.4-9.el9.x86_64 2024-02-26T04:45:59.344 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-26T04:45:59.344 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-26T04:45:59.344 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-26T04:45:59.344 INFO:teuthology.orchestra.run.smithi156.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-26T04:45:59.345 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-26T04:45:59.345 INFO:teuthology.orchestra.run.smithi156.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-26T04:45:59.345 INFO:teuthology.orchestra.run.smithi156.stdout: jq-1.6-16.el9.x86_64 2024-02-26T04:45:59.345 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-26T04:45:59.345 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-26T04:45:59.345 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-26T04:45:59.345 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.345 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.345 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.345 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-26T04:45:59.345 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-26T04:45:59.345 INFO:teuthology.orchestra.run.smithi156.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-26T04:45:59.345 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-26T04:45:59.346 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-26T04:45:59.346 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.346 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.346 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-26T04:45:59.346 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.346 INFO:teuthology.orchestra.run.smithi156.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-26T04:45:59.346 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-26T04:45:59.346 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-26T04:45:59.346 INFO:teuthology.orchestra.run.smithi156.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-26T04:45:59.346 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-26T04:45:59.346 INFO:teuthology.orchestra.run.smithi156.stdout: lua-5.4.4-4.el9.x86_64 2024-02-26T04:45:59.346 INFO:teuthology.orchestra.run.smithi156.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-26T04:45:59.346 INFO:teuthology.orchestra.run.smithi156.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-26T04:45:59.346 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-26T04:45:59.347 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-26T04:45:59.347 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-26T04:45:59.347 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-26T04:45:59.347 INFO:teuthology.orchestra.run.smithi156.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-26T04:45:59.347 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-13.11-1.el9.x86_64 2024-02-26T04:45:59.347 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-26T04:45:59.347 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-26T04:45:59.347 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-26T04:45:59.347 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-26T04:45:59.347 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-26T04:45:59.347 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-26T04:45:59.347 INFO:teuthology.orchestra.run.smithi156.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-26T04:45:59.347 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-26T04:45:59.348 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-26T04:45:59.348 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-26T04:45:59.348 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.348 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.348 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.348 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-26T04:45:59.348 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-26T04:45:59.348 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-26T04:45:59.348 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-26T04:45:59.348 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-26T04:45:59.348 INFO:teuthology.orchestra.run.smithi156.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-26T04:45:59.348 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-26T04:45:59.348 INFO:teuthology.orchestra.run.smithi156.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-26T04:45:59.349 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-26T04:45:59.349 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-26T04:45:59.349 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-26T04:45:59.349 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-26T04:45:59.349 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-26T04:45:59.349 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-26T04:45:59.349 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-26T04:45:59.349 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-26T04:45:59.349 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-26T04:45:59.349 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-26T04:45:59.349 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-26T04:45:59.349 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-26T04:45:59.349 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ply-3.11-14.el9.noarch 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-26T04:45:59.350 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-26T04:45:59.351 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-26T04:45:59.351 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-26T04:45:59.351 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-26T04:45:59.351 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-26T04:45:59.351 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.351 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.351 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-26T04:45:59.351 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-26T04:45:59.351 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-26T04:45:59.351 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.351 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-26T04:45:59.352 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-26T04:45:59.352 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-26T04:45:59.352 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-26T04:45:59.352 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-26T04:45:59.352 INFO:teuthology.orchestra.run.smithi156.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-26T04:45:59.352 INFO:teuthology.orchestra.run.smithi156.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-26T04:45:59.352 INFO:teuthology.orchestra.run.smithi156.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-26T04:45:59.352 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-26T04:45:59.352 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-26T04:45:59.352 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-26T04:45:59.352 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-26T04:45:59.352 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-26T04:45:59.352 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-26T04:45:59.353 INFO:teuthology.orchestra.run.smithi156.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-26T04:45:59.353 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.353 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.353 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:19.0.0-1514.g25296a8f.el9.x86_64 2024-02-26T04:45:59.353 INFO:teuthology.orchestra.run.smithi156.stdout: re2-1:20211101-3.el9.x86_64 2024-02-26T04:45:59.353 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-26T04:45:59.353 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-26T04:45:59.353 INFO:teuthology.orchestra.run.smithi156.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-26T04:45:59.353 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-26T04:45:59.353 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-26T04:45:59.354 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-26T04:45:59.354 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-02-26T04:45:59.354 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-02-26T04:45:59.780 DEBUG:teuthology.parallel:result is None 2024-02-26T04:45:59.781 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T04:45:59.999 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-26T04:46:00.025 INFO:teuthology.orchestra.run.smithi060.stdout:19.0.0-1514.g25296a8f.el9 2024-02-26T04:46:00.026 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1514.g25296a8f.el9 2024-02-26T04:46:00.026 INFO:teuthology.task.install:The correct ceph version 19.0.0-1514.g25296a8f is installed. 2024-02-26T04:46:00.028 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T04:46:00.243 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-26T04:46:00.264 INFO:teuthology.orchestra.run.smithi156.stdout:19.0.0-1514.g25296a8f.el9 2024-02-26T04:46:00.265 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1514.g25296a8f.el9 2024-02-26T04:46:00.265 INFO:teuthology.task.install:The correct ceph version 19.0.0-1514.g25296a8f is installed. 2024-02-26T04:46:00.268 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T04:46:00.407 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-26T04:46:00.432 INFO:teuthology.orchestra.run.smithi191.stdout:19.0.0-1514.g25296a8f.el9 2024-02-26T04:46:00.433 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1514.g25296a8f.el9 2024-02-26T04:46:00.433 INFO:teuthology.task.install:The correct ceph version 19.0.0-1514.g25296a8f is installed. 2024-02-26T04:46:00.436 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-26T04:46:00.460 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:46:00.460 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-26T04:46:00.495 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-26T04:46:00.495 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-26T04:46:00.525 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-26T04:46:00.525 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-26T04:46:00.558 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-26T04:46:00.558 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:46:00.559 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2024-02-26T04:46:00.588 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-26T04:46:00.657 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-26T04:46:00.657 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2024-02-26T04:46:00.685 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-26T04:46:00.752 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-26T04:46:00.752 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2024-02-26T04:46:00.784 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-26T04:46:00.855 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-26T04:46:00.855 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:46:00.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-26T04:46:00.885 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-26T04:46:00.953 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-26T04:46:00.954 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-26T04:46:00.980 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-26T04:46:01.048 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-26T04:46:01.048 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-26T04:46:01.075 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-26T04:46:01.142 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-26T04:46:01.142 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:46:01.142 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2024-02-26T04:46:01.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-26T04:46:01.240 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-26T04:46:01.240 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2024-02-26T04:46:01.266 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-26T04:46:01.335 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-26T04:46:01.335 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/stdin-killer 2024-02-26T04:46:01.363 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-26T04:46:01.440 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-26T04:46:01.830 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-02-26T04:46:01.830 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T04:46:01.831 INFO:tasks.cephadm:Cluster fsid is f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:46:01.831 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-26T04:46:01.831 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.60', 'mon.b': '172.21.15.156', 'mon.c': '172.21.15.191'} 2024-02-26T04:46:01.831 INFO:tasks.cephadm:First mon is mon.a on smithi060 2024-02-26T04:46:01.831 INFO:tasks.cephadm:First mgr is x 2024-02-26T04:46:01.831 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-26T04:46:01.832 DEBUG:teuthology.orchestra.run.smithi060:> sudo hostname $(hostname -s) 2024-02-26T04:46:01.864 DEBUG:teuthology.orchestra.run.smithi156:> sudo hostname $(hostname -s) 2024-02-26T04:46:01.894 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname $(hostname -s) 2024-02-26T04:46:01.924 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-26T04:46:01.927 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T04:46:02.072 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '25296a8f176256aea5ed8b5850b25167dce0ae33', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76813/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1514-g25296a8f', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1514.g25296a8f'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-16 07:18:11.982893', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240216.060239', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-02-26T04:46:02.222 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-vshankar-testing-20240216.060239, sha1 25296a8f176256aea5ed8b5850b25167dce0ae33 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T04:46:02.224 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/x86_64/flavors/default/cephadm 2024-02-26T04:46:02.224 DEBUG:teuthology.orchestra.run.smithi060:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-26T04:46:02.725 INFO:teuthology.orchestra.run.smithi060.stdout:-rw-r--r--. 1 ubuntu ubuntu 766906 Feb 26 04:46 /home/ubuntu/cephtest/cephadm 2024-02-26T04:46:02.725 DEBUG:teuthology.orchestra.run.smithi156:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-26T04:46:03.357 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r--. 1 ubuntu ubuntu 766906 Feb 26 04:46 /home/ubuntu/cephtest/cephadm 2024-02-26T04:46:03.357 DEBUG:teuthology.orchestra.run.smithi191:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240216.060239/25296a8f176256aea5ed8b5850b25167dce0ae33/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-26T04:46:03.953 INFO:teuthology.orchestra.run.smithi191.stdout:-rw-r--r--. 1 ubuntu ubuntu 766906 Feb 26 04:46 /home/ubuntu/cephtest/cephadm 2024-02-26T04:46:03.954 DEBUG:teuthology.orchestra.run.smithi060:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-26T04:46:03.975 DEBUG:teuthology.orchestra.run.smithi156:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-26T04:46:03.995 DEBUG:teuthology.orchestra.run.smithi191:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-26T04:46:04.031 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 on all hosts... 2024-02-26T04:46:04.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 pull 2024-02-26T04:46:04.034 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 pull 2024-02-26T04:46:04.037 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 pull 2024-02-26T04:46:04.834 INFO:teuthology.orchestra.run.smithi156.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T04:46:04.862 INFO:teuthology.orchestra.run.smithi191.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T04:46:04.882 INFO:teuthology.orchestra.run.smithi060.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T04:46:40.500 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-02-26T04:46:40.501 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev)", 2024-02-26T04:46:40.501 INFO:teuthology.orchestra.run.smithi191.stdout: "image_id": "a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86", 2024-02-26T04:46:40.501 INFO:teuthology.orchestra.run.smithi191.stdout: "repo_digests": [ 2024-02-26T04:46:40.501 INFO:teuthology.orchestra.run.smithi191.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a" 2024-02-26T04:46:40.501 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-02-26T04:46:40.502 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-02-26T04:47:03.333 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-02-26T04:47:03.333 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev)", 2024-02-26T04:47:03.333 INFO:teuthology.orchestra.run.smithi156.stdout: "image_id": "a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86", 2024-02-26T04:47:03.334 INFO:teuthology.orchestra.run.smithi156.stdout: "repo_digests": [ 2024-02-26T04:47:03.334 INFO:teuthology.orchestra.run.smithi156.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a" 2024-02-26T04:47:03.334 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2024-02-26T04:47:03.334 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-02-26T04:47:03.942 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:47:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev)", 2024-02-26T04:47:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: "image_id": "a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86", 2024-02-26T04:47:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: "repo_digests": [ 2024-02-26T04:47:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a" 2024-02-26T04:47:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T04:47:03.942 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:47:03.967 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph 2024-02-26T04:47:04.005 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph 2024-02-26T04:47:04.037 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph 2024-02-26T04:47:04.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /etc/ceph 2024-02-26T04:47:04.101 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /etc/ceph 2024-02-26T04:47:04.129 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /etc/ceph 2024-02-26T04:47:04.160 INFO:tasks.cephadm:Writing seed config... 2024-02-26T04:47:04.163 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-26T04:47:04.164 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-26T04:47:04.164 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-26T04:47:04.164 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-26T04:47:04.164 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-26T04:47:04.164 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-02-26T04:47:04.164 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-26T04:47:04.164 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-26T04:47:04.164 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-26T04:47:04.164 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-02-26T04:47:04.164 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-02-26T04:47:04.164 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-02-26T04:47:04.164 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-02-26T04:47:04.164 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-26T04:47:04.165 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-26T04:47:04.165 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-26T04:47:04.165 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-26T04:47:04.165 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-02-26T04:47:04.165 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-26T04:47:04.165 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-26T04:47:04.165 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-26T04:47:04.165 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-02-26T04:47:04.165 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-26T04:47:04.165 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-26T04:47:04.166 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-26T04:47:04.166 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-26T04:47:04.167 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-26T04:47:04.167 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-26T04:47:04.167 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-26T04:47:04.168 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-02-26T04:47:04.168 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-26T04:47:04.168 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-02-26T04:47:04.168 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-02-26T04:47:04.168 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-26T04:47:04.168 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-26T04:47:04.168 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-02-26T04:47:04.168 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-26T04:47:04.168 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-26T04:47:04.168 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-26T04:47:04.168 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-26T04:47:04.168 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-26T04:47:04.169 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-26T04:47:04.169 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-26T04:47:04.169 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-02-26T04:47:04.169 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:47:04.170 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-26T04:47:04.185 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f12e2206-d461-11ee-95c3-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-26T04:47:04.186 DEBUG:teuthology.orchestra.run.smithi060:mon.a> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.a.service 2024-02-26T04:47:04.228 DEBUG:teuthology.orchestra.run.smithi060:mgr.x> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mgr.x.service 2024-02-26T04:47:04.271 INFO:tasks.cephadm:Bootstrapping... 2024-02-26T04:47:04.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 -v bootstrap --fsid f12e2206-d461-11ee-95c3-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.60 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-26T04:47:04.467 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-02-26T04:47:04.467 INFO:teuthology.orchestra.run.smithi060.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33', '-v', 'bootstrap', '--fsid', 'f12e2206-d461-11ee-95c3-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.60', '--skip-admin-label'] 2024-02-26T04:47:04.497 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-26T04:47:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-26T04:47:04.497 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying podman|docker is present... 2024-02-26T04:47:04.522 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-26T04:47:04.522 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying lvm2 is present... 2024-02-26T04:47:04.523 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying time synchronization is in place... 2024-02-26T04:47:04.529 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-26T04:47:04.529 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-26T04:47:04.535 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-26T04:47:04.535 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-02-26T04:47:04.541 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout enabled 2024-02-26T04:47:04.547 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout active 2024-02-26T04:47:04.547 INFO:teuthology.orchestra.run.smithi060.stdout:Unit chronyd.service is enabled and running 2024-02-26T04:47:04.548 INFO:teuthology.orchestra.run.smithi060.stdout:Repeating the final host check... 2024-02-26T04:47:04.569 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-26T04:47:04.569 INFO:teuthology.orchestra.run.smithi060.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-26T04:47:04.569 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl is present 2024-02-26T04:47:04.569 INFO:teuthology.orchestra.run.smithi060.stdout:lvcreate is present 2024-02-26T04:47:04.575 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-26T04:47:04.575 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-26T04:47:04.581 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-26T04:47:04.581 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-02-26T04:47:04.587 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout enabled 2024-02-26T04:47:04.593 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout active 2024-02-26T04:47:04.593 INFO:teuthology.orchestra.run.smithi060.stdout:Unit chronyd.service is enabled and running 2024-02-26T04:47:04.593 INFO:teuthology.orchestra.run.smithi060.stdout:Host looks OK 2024-02-26T04:47:04.594 INFO:teuthology.orchestra.run.smithi060.stdout:Cluster fsid: f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:47:04.594 INFO:teuthology.orchestra.run.smithi060.stdout:Acquiring lock 139764550787024 on /run/cephadm/f12e2206-d461-11ee-95c3-87774f69a715.lock 2024-02-26T04:47:04.594 INFO:teuthology.orchestra.run.smithi060.stdout:Lock 139764550787024 acquired on /run/cephadm/f12e2206-d461-11ee-95c3-87774f69a715.lock 2024-02-26T04:47:04.595 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying IP 172.21.15.60 port 3300 ... 2024-02-26T04:47:04.595 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying IP 172.21.15.60 port 6789 ... 2024-02-26T04:47:04.595 INFO:teuthology.orchestra.run.smithi060.stdout:Base mon IP(s) is [172.21.15.60:3300, 172.21.15.60:6789], mon addrv is [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-02-26T04:47:04.597 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.60 metric 100 2024-02-26T04:47:04.597 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.60 metric 100 2024-02-26T04:47:04.600 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-26T04:47:04.601 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-02-26T04:47:04.601 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-02-26T04:47:04.602 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-26T04:47:04.602 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-26T04:47:04.602 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-26T04:47:04.602 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-26T04:47:04.602 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-02-26T04:47:04.602 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-26T04:47:04.603 INFO:teuthology.orchestra.run.smithi060.stdout:Mon IP `172.21.15.60` is in CIDR network `172.21.0.0/20` 2024-02-26T04:47:04.603 INFO:teuthology.orchestra.run.smithi060.stdout:Mon IP `172.21.15.60` is in CIDR network `172.21.0.0/20` 2024-02-26T04:47:04.603 INFO:teuthology.orchestra.run.smithi060.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-26T04:47:04.603 INFO:teuthology.orchestra.run.smithi060.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-26T04:47:04.604 INFO:teuthology.orchestra.run.smithi060.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T04:47:05.094 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 2024-02-26T04:47:05.095 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33... 2024-02-26T04:47:05.095 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Getting image source signatures 2024-02-26T04:47:05.095 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying blob sha256:3023f2ea73c33e75bbdc7c6f4507678b80f8a5636a9a11c7f009befea7a064c0 2024-02-26T04:47:05.095 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-26T04:47:05.095 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying config sha256:a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 2024-02-26T04:47:05.095 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-26T04:47:06.024 INFO:teuthology.orchestra.run.smithi060.stdout:ceph: stdout ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev) 2024-02-26T04:47:06.024 INFO:teuthology.orchestra.run.smithi060.stdout:Ceph version: ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev) 2024-02-26T04:47:06.024 INFO:teuthology.orchestra.run.smithi060.stdout:Extracting ceph user uid/gid from container image... 2024-02-26T04:47:06.855 INFO:teuthology.orchestra.run.smithi060.stdout:stat: stdout 167 167 2024-02-26T04:47:06.855 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial keys... 2024-02-26T04:47:07.918 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQDLF9xlVyw0ChAATiUDEYHuKkYRiIiaCCd1RQ== 2024-02-26T04:47:08.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQDMF9xlYib3ChAAmjQVP8d+oX5562NFevuO/Q== 2024-02-26T04:47:09.622 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQDNF9xlLi3bBRAAyDg0rOlldPNtvOF5gkc9xg== 2024-02-26T04:47:09.623 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial monmap... 2024-02-26T04:47:12.862 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-26T04:47:12.862 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-26T04:47:12.862 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:47:12.862 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-26T04:47:12.862 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool for a [v2:172.21.15.60:3300,v1:172.21.15.60:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-26T04:47:12.863 INFO:teuthology.orchestra.run.smithi060.stdout:setting min_mon_release = quincy 2024-02-26T04:47:12.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: set fsid to f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:47:12.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-26T04:47:12.863 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:47:12.863 INFO:teuthology.orchestra.run.smithi060.stdout:Creating mon... 2024-02-26T04:47:20.919 INFO:teuthology.orchestra.run.smithi060.stdout:create mon.a on 2024-02-26T04:47:21.123 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-26T04:47:21.261 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-26T04:47:21.545 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f12e2206-d461-11ee-95c3-87774f69a715.target → /etc/systemd/system/ceph-f12e2206-d461-11ee-95c3-87774f69a715.target. 2024-02-26T04:47:21.545 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f12e2206-d461-11ee-95c3-87774f69a715.target → /etc/systemd/system/ceph-f12e2206-d461-11ee-95c3-87774f69a715.target. 2024-02-26T04:47:21.792 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.a 2024-02-26T04:47:21.792 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.a.service: Unit ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.a.service not loaded. 2024-02-26T04:47:21.954 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f12e2206-d461-11ee-95c3-87774f69a715.target.wants/ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.a.service → /etc/systemd/system/ceph-f12e2206-d461-11ee-95c3-87774f69a715@.service. 2024-02-26T04:47:22.453 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:22 smithi060 podman[31989]: 2024-02-26 04:47:22.451158785 +0000 UTC m=+0.380923884 container init 4b706d85372c3928bcd8d9c1f9d613352703db5e84f666fa7d7bdaa2932b9315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=) 2024-02-26T04:47:22.582 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T04:47:22.582 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-02-26T04:47:22.588 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T04:47:22.589 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-02-26T04:47:22.589 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-02-26T04:47:22.589 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to enable service . firewalld.service is not available 2024-02-26T04:47:22.589 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mon to start... 2024-02-26T04:47:22.589 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mon... 2024-02-26T04:47:22.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:22 smithi060 podman[31989]: 2024-02-26 04:47:22.454130629 +0000 UTC m=+0.383895729 container start 4b706d85372c3928bcd8d9c1f9d613352703db5e84f666fa7d7bdaa2932b9315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-26T04:47:22.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:22 smithi060 ceph-mon[32008]: mkfs f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:47:22.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:22 smithi060 ceph-mon[32008]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-26T04:47:22.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:22 smithi060 bash[31989]: 4b706d85372c3928bcd8d9c1f9d613352703db5e84f666fa7d7bdaa2932b9315 2024-02-26T04:47:22.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:22 smithi060 systemd[1]: Started Ceph mon.a for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:47:23.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:23 smithi060 ceph-mon[32008]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-26T04:47:23.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:23 smithi060 ceph-mon[32008]: monmap e1: 1 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T04:47:23.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:23 smithi060 ceph-mon[32008]: fsmap 2024-02-26T04:47:23.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:23 smithi060 ceph-mon[32008]: osdmap e1: 0 total, 0 up, 0 in 2024-02-26T04:47:23.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:23 smithi060 ceph-mon[32008]: mgrmap e1: no daemons active 2024-02-26T04:47:23.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:23 smithi060 ceph-mon[32008]: from='client.? 172.21.15.60:0/356732348' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-26T04:47:23.654 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout cluster: 2024-02-26T04:47:23.660 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout id: f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:47:23.661 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-26T04:47:23.661 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T04:47:23.661 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout services: 2024-02-26T04:47:23.661 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.554381s) 2024-02-26T04:47:23.661 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-26T04:47:23.661 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-26T04:47:23.661 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T04:47:23.661 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout data: 2024-02-26T04:47:23.661 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-26T04:47:23.661 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-26T04:47:23.661 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-26T04:47:23.661 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout pgs: 2024-02-26T04:47:23.661 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T04:47:23.661 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.993+0000 7f983c0c3700 1 Processor -- start 2024-02-26T04:47:23.662 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.993+0000 7f983c0c3700 1 -- start start 2024-02-26T04:47:23.662 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.994+0000 7f983c0c3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9834108070 0x7f983410a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:23.662 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.994+0000 7f983c0c3700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983407ae70 con 0x7f9834108070 2024-02-26T04:47:23.662 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.994+0000 7f9839e5f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9834108070 0x7f983410a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:23.662 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.994+0000 7f9839e5f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9834108070 0x7f983410a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45418/0 (socket says 172.21.15.60:45418) 2024-02-26T04:47:23.662 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.994+0000 7f9839e5f700 1 -- 172.21.15.60:0/1951176108 learned_addr learned my addr 172.21.15.60:0/1951176108 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:23.662 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.994+0000 7f9839e5f700 1 -- 172.21.15.60:0/1951176108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983410a9a0 con 0x7f9834108070 2024-02-26T04:47:23.662 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.994+0000 7f9839e5f700 1 --2- 172.21.15.60:0/1951176108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9834108070 0x7f983410a460 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f982c0098a0 tx=0x7f982c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711d5b3eb82f17d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:23.662 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.995+0000 7f9838e5d700 1 -- 172.21.15.60:0/1951176108 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f982c004160 con 0x7f9834108070 2024-02-26T04:47:23.662 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.995+0000 7f9838e5d700 1 -- 172.21.15.60:0/1951176108 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c0042c0 con 0x7f9834108070 2024-02-26T04:47:23.662 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.995+0000 7f9838e5d700 1 -- 172.21.15.60:0/1951176108 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f982c0045a0 con 0x7f9834108070 2024-02-26T04:47:23.662 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.995+0000 7f983c0c3700 1 -- 172.21.15.60:0/1951176108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9834108070 msgr2=0x7f983410a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:23.662 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.995+0000 7f983c0c3700 1 --2- 172.21.15.60:0/1951176108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9834108070 0x7f983410a460 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f982c0098a0 tx=0x7f982c009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.995+0000 7f983c0c3700 1 -- 172.21.15.60:0/1951176108 shutdown_connections 2024-02-26T04:47:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.995+0000 7f983c0c3700 1 --2- 172.21.15.60:0/1951176108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9834108070 0x7f983410a460 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.995+0000 7f983c0c3700 1 -- 172.21.15.60:0/1951176108 >> 172.21.15.60:0/1951176108 conn(0x7f9834101a80 msgr2=0x7f9834103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.995+0000 7f983c0c3700 1 -- 172.21.15.60:0/1951176108 shutdown_connections 2024-02-26T04:47:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.995+0000 7f983c0c3700 1 -- 172.21.15.60:0/1951176108 wait complete. 2024-02-26T04:47:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.996+0000 7f983c0c3700 1 Processor -- start 2024-02-26T04:47:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.996+0000 7f983c0c3700 1 -- start start 2024-02-26T04:47:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.996+0000 7f983c0c3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9834108070 0x7f983419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.996+0000 7f983c0c3700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983419af80 con 0x7f9834108070 2024-02-26T04:47:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.996+0000 7f9839e5f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9834108070 0x7f983419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.996+0000 7f9839e5f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9834108070 0x7f983419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45428/0 (socket says 172.21.15.60:45428) 2024-02-26T04:47:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.996+0000 7f9839e5f700 1 -- 172.21.15.60:0/356732348 learned_addr learned my addr 172.21.15.60:0/356732348 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.997+0000 7f9839e5f700 1 -- 172.21.15.60:0/356732348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c009550 con 0x7f9834108070 2024-02-26T04:47:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.997+0000 7f9839e5f700 1 --2- 172.21.15.60:0/356732348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9834108070 0x7f983419aa40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f983407a2b0 tx=0x7f982c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.997+0000 7f9832ffd700 1 -- 172.21.15.60:0/356732348 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f982c011630 con 0x7f9834108070 2024-02-26T04:47:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.997+0000 7f9832ffd700 1 -- 172.21.15.60:0/356732348 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c011790 con 0x7f9834108070 2024-02-26T04:47:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.997+0000 7f983c0c3700 1 -- 172.21.15.60:0/356732348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983419b180 con 0x7f9834108070 2024-02-26T04:47:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.997+0000 7f9832ffd700 1 -- 172.21.15.60:0/356732348 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f982c011a30 con 0x7f9834108070 2024-02-26T04:47:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.997+0000 7f983c0c3700 1 -- 172.21.15.60:0/356732348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983419b620 con 0x7f9834108070 2024-02-26T04:47:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.997+0000 7f9832ffd700 1 -- 172.21.15.60:0/356732348 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f982c022070 con 0x7f9834108070 2024-02-26T04:47:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.997+0000 7f9832ffd700 1 -- 172.21.15.60:0/356732348 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f982c01c450 con 0x7f9834108070 2024-02-26T04:47:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:22.998+0000 7f983c0c3700 1 -- 172.21.15.60:0/356732348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98341948a0 con 0x7f9834108070 2024-02-26T04:47:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:23.001+0000 7f9832ffd700 1 -- 172.21.15.60:0/356732348 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f982c011b90 con 0x7f9834108070 2024-02-26T04:47:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:23.073+0000 7f983c0c3700 1 -- 172.21.15.60:0/356732348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f983410ae60 con 0x7f9834108070 2024-02-26T04:47:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:23.073+0000 7f9832ffd700 1 -- 172.21.15.60:0/356732348 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f982c011d30 con 0x7f9834108070 2024-02-26T04:47:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:23.074+0000 7f983c0c3700 1 -- 172.21.15.60:0/356732348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9834108070 msgr2=0x7f983419aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:23.074+0000 7f983c0c3700 1 --2- 172.21.15.60:0/356732348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9834108070 0x7f983419aa40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f983407a2b0 tx=0x7f982c004750 comp rx=0 tx=0).stop 2024-02-26T04:47:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:23.074+0000 7f983c0c3700 1 -- 172.21.15.60:0/356732348 shutdown_connections 2024-02-26T04:47:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:23.074+0000 7f983c0c3700 1 --2- 172.21.15.60:0/356732348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9834108070 0x7f983419aa40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:23.074+0000 7f983c0c3700 1 -- 172.21.15.60:0/356732348 >> 172.21.15.60:0/356732348 conn(0x7f9834101a80 msgr2=0x7f9834103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:23.074+0000 7f983c0c3700 1 -- 172.21.15.60:0/356732348 shutdown_connections 2024-02-26T04:47:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:23.074+0000 7f983c0c3700 1 -- 172.21.15.60:0/356732348 wait complete. 2024-02-26T04:47:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:mon is available 2024-02-26T04:47:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:Assimilating anything we can from ceph.conf... 2024-02-26T04:47:24.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:24 smithi060 ceph-mon[32008]: from='client.? 172.21.15.60:0/2929034763' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-26T04:47:24.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:24 smithi060 ceph-mon[32008]: from='client.? 172.21.15.60:0/2929034763' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-26T04:47:24.791 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T04:47:24.791 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [global] 2024-02-26T04:47:24.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout fsid = f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:47:24.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-02-26T04:47:24.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-26T04:47:24.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-26T04:47:24.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-26T04:47:24.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-26T04:47:24.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T04:47:24.792 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [osd] 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.070+0000 7f0b8e1c5700 1 Processor -- start 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.070+0000 7f0b8e1c5700 1 -- start start 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.070+0000 7f0b8e1c5700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b8807ce90 0x7f0b8807b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.070+0000 7f0b8e1c5700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8807b910 con 0x7f0b8807ce90 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.071+0000 7f0b877fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b8807ce90 0x7f0b8807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.071+0000 7f0b877fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b8807ce90 0x7f0b8807b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45438/0 (socket says 172.21.15.60:45438) 2024-02-26T04:47:24.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.071+0000 7f0b877fe700 1 -- 172.21.15.60:0/2492857528 learned_addr learned my addr 172.21.15.60:0/2492857528 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:24.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.071+0000 7f0b877fe700 1 -- 172.21.15.60:0/2492857528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8807ba50 con 0x7f0b8807ce90 2024-02-26T04:47:24.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.071+0000 7f0b877fe700 1 --2- 172.21.15.60:0/2492857528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b8807ce90 0x7f0b8807b340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0b780098a0 tx=0x7f0b78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a3860ad0dd176c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:24.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.071+0000 7f0b867fc700 1 -- 172.21.15.60:0/2492857528 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b78004160 con 0x7f0b8807ce90 2024-02-26T04:47:24.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.071+0000 7f0b867fc700 1 -- 172.21.15.60:0/2492857528 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b780042c0 con 0x7f0b8807ce90 2024-02-26T04:47:24.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.071+0000 7f0b867fc700 1 -- 172.21.15.60:0/2492857528 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b780045a0 con 0x7f0b8807ce90 2024-02-26T04:47:24.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.072+0000 7f0b8e1c5700 1 -- 172.21.15.60:0/2492857528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b8807ce90 msgr2=0x7f0b8807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:24.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.072+0000 7f0b8e1c5700 1 --2- 172.21.15.60:0/2492857528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b8807ce90 0x7f0b8807b340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0b780098a0 tx=0x7f0b78009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:24.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.072+0000 7f0b8e1c5700 1 -- 172.21.15.60:0/2492857528 shutdown_connections 2024-02-26T04:47:24.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.072+0000 7f0b8e1c5700 1 --2- 172.21.15.60:0/2492857528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b8807ce90 0x7f0b8807b340 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:24.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.072+0000 7f0b8e1c5700 1 -- 172.21.15.60:0/2492857528 >> 172.21.15.60:0/2492857528 conn(0x7f0b88102b50 msgr2=0x7f0b88104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:24.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.072+0000 7f0b8e1c5700 1 -- 172.21.15.60:0/2492857528 shutdown_connections 2024-02-26T04:47:24.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.072+0000 7f0b8e1c5700 1 -- 172.21.15.60:0/2492857528 wait complete. 2024-02-26T04:47:24.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.073+0000 7f0b8e1c5700 1 Processor -- start 2024-02-26T04:47:24.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.073+0000 7f0b8e1c5700 1 -- start start 2024-02-26T04:47:24.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.073+0000 7f0b8e1c5700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b8807ce90 0x7f0b8819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:24.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.073+0000 7f0b8e1c5700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8819d200 con 0x7f0b8807ce90 2024-02-26T04:47:24.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.073+0000 7f0b877fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b8807ce90 0x7f0b8819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:24.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.073+0000 7f0b877fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b8807ce90 0x7f0b8819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45444/0 (socket says 172.21.15.60:45444) 2024-02-26T04:47:24.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.073+0000 7f0b877fe700 1 -- 172.21.15.60:0/2929034763 learned_addr learned my addr 172.21.15.60:0/2929034763 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:24.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.073+0000 7f0b877fe700 1 -- 172.21.15.60:0/2929034763 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b78009550 con 0x7f0b8807ce90 2024-02-26T04:47:24.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.074+0000 7f0b877fe700 1 --2- 172.21.15.60:0/2929034763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b8807ce90 0x7f0b8819ccc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0b8807c3b0 tx=0x7f0b78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:24.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.074+0000 7f0b84ff9700 1 -- 172.21.15.60:0/2929034763 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b78011630 con 0x7f0b8807ce90 2024-02-26T04:47:24.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.074+0000 7f0b84ff9700 1 -- 172.21.15.60:0/2929034763 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b78011790 con 0x7f0b8807ce90 2024-02-26T04:47:24.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.074+0000 7f0b84ff9700 1 -- 172.21.15.60:0/2929034763 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b78011a30 con 0x7f0b8807ce90 2024-02-26T04:47:24.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.074+0000 7f0b8e1c5700 1 -- 172.21.15.60:0/2929034763 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8819d400 con 0x7f0b8807ce90 2024-02-26T04:47:24.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.074+0000 7f0b8e1c5700 1 -- 172.21.15.60:0/2929034763 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8819d8a0 con 0x7f0b8807ce90 2024-02-26T04:47:24.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.074+0000 7f0b84ff9700 1 -- 172.21.15.60:0/2929034763 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0b78021070 con 0x7f0b8807ce90 2024-02-26T04:47:24.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.074+0000 7f0b84ff9700 1 -- 172.21.15.60:0/2929034763 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0b7801b7d0 con 0x7f0b8807ce90 2024-02-26T04:47:24.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.075+0000 7f0b8e1c5700 1 -- 172.21.15.60:0/2929034763 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b88196a30 con 0x7f0b8807ce90 2024-02-26T04:47:24.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.077+0000 7f0b84ff9700 1 -- 172.21.15.60:0/2929034763 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0b7801ba30 con 0x7f0b8807ce90 2024-02-26T04:47:24.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.147+0000 7f0b8e1c5700 1 -- 172.21.15.60:0/2929034763 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0b8807bd60 con 0x7f0b8807ce90 2024-02-26T04:47:24.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.150+0000 7f0b84ff9700 1 -- 172.21.15.60:0/2929034763 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f0b7801bc80 con 0x7f0b8807ce90 2024-02-26T04:47:24.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.151+0000 7f0b84ff9700 1 -- 172.21.15.60:0/2929034763 <== mon.0 v2:172.21.15.60:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b7801be20 con 0x7f0b8807ce90 2024-02-26T04:47:24.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.152+0000 7f0b8e1c5700 1 -- 172.21.15.60:0/2929034763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b8807ce90 msgr2=0x7f0b8819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:24.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.152+0000 7f0b8e1c5700 1 --2- 172.21.15.60:0/2929034763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b8807ce90 0x7f0b8819ccc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0b8807c3b0 tx=0x7f0b78004750 comp rx=0 tx=0).stop 2024-02-26T04:47:24.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.152+0000 7f0b8e1c5700 1 -- 172.21.15.60:0/2929034763 shutdown_connections 2024-02-26T04:47:24.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.152+0000 7f0b8e1c5700 1 --2- 172.21.15.60:0/2929034763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b8807ce90 0x7f0b8819ccc0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:24.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.152+0000 7f0b8e1c5700 1 -- 172.21.15.60:0/2929034763 >> 172.21.15.60:0/2929034763 conn(0x7f0b88102b50 msgr2=0x7f0b88104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:24.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.152+0000 7f0b8e1c5700 1 -- 172.21.15.60:0/2929034763 shutdown_connections 2024-02-26T04:47:24.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:24.152+0000 7f0b8e1c5700 1 -- 172.21.15.60:0/2929034763 wait complete. 2024-02-26T04:47:24.796 INFO:teuthology.orchestra.run.smithi060.stdout:Generating new minimal ceph.conf... 2024-02-26T04:47:25.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:25 smithi060 ceph-mon[32008]: from='client.? 172.21.15.60:0/972361486' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:47:25.862 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.196+0000 7fb9b3328700 1 Processor -- start 2024-02-26T04:47:25.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.196+0000 7fb9b3328700 1 -- start start 2024-02-26T04:47:25.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.196+0000 7fb9b3328700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9ac07ce30 0x7fb9ac07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:25.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.196+0000 7fb9b3328700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac07b820 con 0x7fb9ac07ce30 2024-02-26T04:47:25.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.196+0000 7fb9b10c4700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9ac07ce30 0x7fb9ac07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:25.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.196+0000 7fb9b10c4700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9ac07ce30 0x7fb9ac07b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45456/0 (socket says 172.21.15.60:45456) 2024-02-26T04:47:25.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.196+0000 7fb9b10c4700 1 -- 172.21.15.60:0/1739478694 learned_addr learned my addr 172.21.15.60:0/1739478694 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:25.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.197+0000 7fb9b10c4700 1 -- 172.21.15.60:0/1739478694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac07b960 con 0x7fb9ac07ce30 2024-02-26T04:47:25.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.197+0000 7fb9b10c4700 1 --2- 172.21.15.60:0/1739478694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9ac07ce30 0x7fb9ac07b2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80098a0 tx=0x7fb9a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3495a149c4ff0a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:25.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.197+0000 7fb9a3fff700 1 -- 172.21.15.60:0/1739478694 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9a8004160 con 0x7fb9ac07ce30 2024-02-26T04:47:25.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.197+0000 7fb9a3fff700 1 -- 172.21.15.60:0/1739478694 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9a80042c0 con 0x7fb9ac07ce30 2024-02-26T04:47:25.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.197+0000 7fb9a3fff700 1 -- 172.21.15.60:0/1739478694 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9a8011ac0 con 0x7fb9ac07ce30 2024-02-26T04:47:25.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.198+0000 7fb9b3328700 1 -- 172.21.15.60:0/1739478694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9ac07ce30 msgr2=0x7fb9ac07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:25.863 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.198+0000 7fb9b3328700 1 --2- 172.21.15.60:0/1739478694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9ac07ce30 0x7fb9ac07b2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80098a0 tx=0x7fb9a8009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:25.864 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.198+0000 7fb9b3328700 1 -- 172.21.15.60:0/1739478694 shutdown_connections 2024-02-26T04:47:25.864 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.198+0000 7fb9b3328700 1 --2- 172.21.15.60:0/1739478694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9ac07ce30 0x7fb9ac07b2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:25.864 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.198+0000 7fb9b3328700 1 -- 172.21.15.60:0/1739478694 >> 172.21.15.60:0/1739478694 conn(0x7fb9ac102be0 msgr2=0x7fb9ac105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:25.864 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.198+0000 7fb9b3328700 1 -- 172.21.15.60:0/1739478694 shutdown_connections 2024-02-26T04:47:25.864 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.198+0000 7fb9b3328700 1 -- 172.21.15.60:0/1739478694 wait complete. 2024-02-26T04:47:25.864 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.198+0000 7fb9b3328700 1 Processor -- start 2024-02-26T04:47:25.864 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.198+0000 7fb9b3328700 1 -- start start 2024-02-26T04:47:25.864 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.199+0000 7fb9b3328700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9ac07ce30 0x7fb9ac19d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:25.864 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.199+0000 7fb9b3328700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac19da90 con 0x7fb9ac07ce30 2024-02-26T04:47:25.864 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.199+0000 7fb9b10c4700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9ac07ce30 0x7fb9ac19d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:25.864 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.199+0000 7fb9b10c4700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9ac07ce30 0x7fb9ac19d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45468/0 (socket says 172.21.15.60:45468) 2024-02-26T04:47:25.864 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.199+0000 7fb9b10c4700 1 -- 172.21.15.60:0/972361486 learned_addr learned my addr 172.21.15.60:0/972361486 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:25.864 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.199+0000 7fb9b10c4700 1 -- 172.21.15.60:0/972361486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a8009550 con 0x7fb9ac07ce30 2024-02-26T04:47:25.865 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.199+0000 7fb9b10c4700 1 --2- 172.21.15.60:0/972361486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9ac07ce30 0x7fb9ac19d550 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009fb0 tx=0x7fb9a8011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:25.865 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.199+0000 7fb9a27fc700 1 -- 172.21.15.60:0/972361486 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9a80038b0 con 0x7fb9ac07ce30 2024-02-26T04:47:25.865 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.199+0000 7fb9a27fc700 1 -- 172.21.15.60:0/972361486 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9a8003a10 con 0x7fb9ac07ce30 2024-02-26T04:47:25.865 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.199+0000 7fb9a27fc700 1 -- 172.21.15.60:0/972361486 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9a80195e0 con 0x7fb9ac07ce30 2024-02-26T04:47:25.865 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.199+0000 7fb9b3328700 1 -- 172.21.15.60:0/972361486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ac19dc90 con 0x7fb9ac07ce30 2024-02-26T04:47:25.865 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.199+0000 7fb9b3328700 1 -- 172.21.15.60:0/972361486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac19e130 con 0x7fb9ac07ce30 2024-02-26T04:47:25.865 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.199+0000 7fb9a27fc700 1 -- 172.21.15.60:0/972361486 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb9a8019be0 con 0x7fb9ac07ce30 2024-02-26T04:47:25.865 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.200+0000 7fb9a27fc700 1 -- 172.21.15.60:0/972361486 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb9a8024a70 con 0x7fb9ac07ce30 2024-02-26T04:47:25.865 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.200+0000 7fb9b3328700 1 -- 172.21.15.60:0/972361486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ac10d970 con 0x7fb9ac07ce30 2024-02-26T04:47:25.865 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.203+0000 7fb9a27fc700 1 -- 172.21.15.60:0/972361486 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb9a8003b80 con 0x7fb9ac07ce30 2024-02-26T04:47:25.865 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.274+0000 7fb9b3328700 1 -- 172.21.15.60:0/972361486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb9ac1a0ca0 con 0x7fb9ac07ce30 2024-02-26T04:47:25.866 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.274+0000 7fb9a27fc700 1 -- 172.21.15.60:0/972361486 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fb9a801a9f0 con 0x7fb9ac07ce30 2024-02-26T04:47:25.866 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.275+0000 7fb9b3328700 1 -- 172.21.15.60:0/972361486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9ac07ce30 msgr2=0x7fb9ac19d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:25.866 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.275+0000 7fb9b3328700 1 --2- 172.21.15.60:0/972361486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9ac07ce30 0x7fb9ac19d550 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009fb0 tx=0x7fb9a8011b20 comp rx=0 tx=0).stop 2024-02-26T04:47:25.866 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.275+0000 7fb9b3328700 1 -- 172.21.15.60:0/972361486 shutdown_connections 2024-02-26T04:47:25.866 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.275+0000 7fb9b3328700 1 --2- 172.21.15.60:0/972361486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9ac07ce30 0x7fb9ac19d550 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:25.866 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.275+0000 7fb9b3328700 1 -- 172.21.15.60:0/972361486 >> 172.21.15.60:0/972361486 conn(0x7fb9ac102be0 msgr2=0x7fb9ac103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:25.866 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.275+0000 7fb9b3328700 1 -- 172.21.15.60:0/972361486 shutdown_connections 2024-02-26T04:47:25.866 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:25.275+0000 7fb9b3328700 1 -- 172.21.15.60:0/972361486 wait complete. 2024-02-26T04:47:25.866 INFO:teuthology.orchestra.run.smithi060.stdout:Restarting the monitor... 2024-02-26T04:47:26.143 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:25 smithi060 systemd[1]: Stopping Ceph mon.a for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T04:47:26.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:26 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a[32004]: 2024-02-26T04:47:26.140+0000 7f6cf1cea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T04:47:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:26 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a[32004]: 2024-02-26T04:47:26.140+0000 7f6cf1cea700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-26T04:47:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:26 smithi060 podman[32253]: 2024-02-26 04:47:26.335028414 +0000 UTC m=+0.310872529 container died 4b706d85372c3928bcd8d9c1f9d613352703db5e84f666fa7d7bdaa2932b9315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-26T04:47:28.159 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:27 smithi060 podman[32253]: 2024-02-26 04:47:27.948100318 +0000 UTC m=+1.923944432 container cleanup 4b706d85372c3928bcd8d9c1f9d613352703db5e84f666fa7d7bdaa2932b9315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True) 2024-02-26T04:47:28.159 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:27 smithi060 bash[32253]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a 2024-02-26T04:47:28.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 podman[32267]: 2024-02-26 04:47:28.157224004 +0000 UTC m=+1.816423593 container remove 4b706d85372c3928bcd8d9c1f9d613352703db5e84f666fa7d7bdaa2932b9315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240214) 2024-02-26T04:47:28.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 systemd[1]: ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-26T04:47:28.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 systemd[1]: Stopped Ceph mon.a for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:47:28.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 systemd[1]: Starting Ceph mon.a for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T04:47:28.724 INFO:teuthology.orchestra.run.smithi060.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-26T04:47:28.756 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 podman[32363]: 2024-02-26 04:47:28.483003824 +0000 UTC m=+0.102212149 container create 5b57dca8a03365e72efd9db9c4e871bf8c64c8eede3518e5747bac26991619b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214) 2024-02-26T04:47:28.756 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 podman[32363]: 2024-02-26 04:47:28.398072881 +0000 UTC m=+0.017281222 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 2024-02-26T04:47:28.756 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 podman[32363]: 2024-02-26 04:47:28.652022451 +0000 UTC m=+0.271230777 container init 5b57dca8a03365e72efd9db9c4e871bf8c64c8eede3518e5747bac26991619b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T04:47:28.756 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 podman[32363]: 2024-02-26 04:47:28.654784937 +0000 UTC m=+0.273993270 container start 5b57dca8a03365e72efd9db9c4e871bf8c64c8eede3518e5747bac26991619b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-26T04:47:28.756 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: set uid:gid to 167:167 (ceph:ceph) 2024-02-26T04:47:28.756 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev), process ceph-mon, pid 2 2024-02-26T04:47:28.756 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: pidfile_write: ignore empty --pid-file 2024-02-26T04:47:28.756 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: load: jerasure load: lrc 2024-02-26T04:47:28.756 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: RocksDB version: 7.9.2 2024-02-26T04:47:28.756 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Git sha 0 2024-02-26T04:47:28.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Compile date 2024-02-16 06:18:30 2024-02-26T04:47:28.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: DB SUMMARY 2024-02-26T04:47:28.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: DB Session ID: 862Q01CMCOMIXGY0RVNJ 2024-02-26T04:47:28.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: CURRENT file: CURRENT 2024-02-26T04:47:28.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: IDENTITY file: IDENTITY 2024-02-26T04:47:28.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-26T04:47:28.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-26T04:47:28.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99842 ; 2024-02-26T04:47:28.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.error_if_exists: 0 2024-02-26T04:47:28.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.create_if_missing: 0 2024-02-26T04:47:28.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.paranoid_checks: 1 2024-02-26T04:47:28.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-26T04:47:28.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.env: 0x55fc322c9ae0 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.fs: PosixFileSystem 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.info_log: 0x55fc32de78c0 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_file_opening_threads: 16 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.statistics: (nil) 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.use_fsync: 0 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_log_file_size: 0 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.keep_log_file_num: 1000 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.recycle_log_file_num: 0 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.allow_fallocate: 1 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.allow_mmap_reads: 0 2024-02-26T04:47:28.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.allow_mmap_writes: 0 2024-02-26T04:47:28.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.use_direct_reads: 0 2024-02-26T04:47:28.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-26T04:47:28.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.create_missing_column_families: 0 2024-02-26T04:47:28.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.db_log_dir: 2024-02-26T04:47:28.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.wal_dir: 2024-02-26T04:47:28.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-26T04:47:28.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-26T04:47:28.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-26T04:47:28.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-26T04:47:28.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-26T04:47:28.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-26T04:47:28.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.advise_random_on_open: 1 2024-02-26T04:47:28.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.db_write_buffer_size: 0 2024-02-26T04:47:28.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.write_buffer_manager: 0x55fc32dbb400 2024-02-26T04:47:28.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-26T04:47:28.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-26T04:47:28.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-26T04:47:28.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.rate_limiter: (nil) 2024-02-26T04:47:28.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-26T04:47:28.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.wal_recovery_mode: 2 2024-02-26T04:47:28.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.enable_thread_tracking: 0 2024-02-26T04:47:28.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.enable_pipelined_write: 0 2024-02-26T04:47:28.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.unordered_write: 0 2024-02-26T04:47:28.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-26T04:47:28.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-26T04:47:28.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-26T04:47:28.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-26T04:47:28.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.row_cache: None 2024-02-26T04:47:28.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.wal_filter: None 2024-02-26T04:47:28.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-26T04:47:28.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.allow_ingest_behind: 0 2024-02-26T04:47:28.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.two_write_queues: 0 2024-02-26T04:47:28.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.manual_wal_flush: 0 2024-02-26T04:47:28.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.wal_compression: 0 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.atomic_flush: 0 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.log_readahead_size: 0 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.best_efforts_recovery: 0 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.allow_data_in_errors: 0 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.db_host_id: __hostname__ 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_background_jobs: 2 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_background_compactions: -1 2024-02-26T04:47:28.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_subcompactions: 1 2024-02-26T04:47:28.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-26T04:47:28.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-26T04:47:28.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-26T04:47:28.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_total_wal_size: 0 2024-02-26T04:47:28.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-26T04:47:28.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-26T04:47:28.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-26T04:47:28.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-26T04:47:28.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_open_files: -1 2024-02-26T04:47:28.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.bytes_per_sync: 0 2024-02-26T04:47:28.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-26T04:47:28.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-26T04:47:28.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compaction_readahead_size: 0 2024-02-26T04:47:28.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_background_flushes: -1 2024-02-26T04:47:28.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Compression algorithms supported: 2024-02-26T04:47:28.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-26T04:47:28.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: kZSTD supported: 0 2024-02-26T04:47:28.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: kXpressCompression supported: 0 2024-02-26T04:47:28.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: kLZ4HCCompression supported: 1 2024-02-26T04:47:28.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: kZlibCompression supported: 1 2024-02-26T04:47:28.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: kSnappyCompression supported: 1 2024-02-26T04:47:28.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: kLZ4Compression supported: 1 2024-02-26T04:47:28.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: kBZip2Compression supported: 0 2024-02-26T04:47:28.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-26T04:47:28.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-26T04:47:28.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-26T04:47:28.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.merge_operator: 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compaction_filter: None 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compaction_filter_factory: None 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.sst_partitioner_factory: None 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fc32de79c0) 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout: cache_index_and_filter_blocks: 1 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout: pin_top_level_index_and_filter: 1 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_type: 0 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout: data_block_index_type: 0 2024-02-26T04:47:28.766 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_shortening: 1 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: checksum: 4 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: no_block_cache: 0 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache: 0x55fc32deb090 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_name: BinnedLRUCache 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_options: 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: capacity : 536870912 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: num_shard_bits : 4 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: strict_capacity_limit : 0 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: high_pri_pool_ratio: 0.000 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_compressed: (nil) 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: persistent_cache: (nil) 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_size: 4096 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_size_deviation: 10 2024-02-26T04:47:28.767 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_restart_interval: 16 2024-02-26T04:47:28.768 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_block_restart_interval: 1 2024-02-26T04:47:28.771 INFO:journalctl@ceph.mon.a.smithi060.stdout: metadata_block_size: 4096 2024-02-26T04:47:28.771 INFO:journalctl@ceph.mon.a.smithi060.stdout: partition_filters: 0 2024-02-26T04:47:28.771 INFO:journalctl@ceph.mon.a.smithi060.stdout: use_delta_encoding: 1 2024-02-26T04:47:28.771 INFO:journalctl@ceph.mon.a.smithi060.stdout: filter_policy: bloomfilter 2024-02-26T04:47:28.771 INFO:journalctl@ceph.mon.a.smithi060.stdout: whole_key_filtering: 1 2024-02-26T04:47:28.771 INFO:journalctl@ceph.mon.a.smithi060.stdout: verify_compression: 0 2024-02-26T04:47:28.771 INFO:journalctl@ceph.mon.a.smithi060.stdout: read_amp_bytes_per_bit: 0 2024-02-26T04:47:28.771 INFO:journalctl@ceph.mon.a.smithi060.stdout: format_version: 5 2024-02-26T04:47:28.771 INFO:journalctl@ceph.mon.a.smithi060.stdout: enable_index_compression: 1 2024-02-26T04:47:28.772 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_align: 0 2024-02-26T04:47:28.772 INFO:journalctl@ceph.mon.a.smithi060.stdout: max_auto_readahead_size: 262144 2024-02-26T04:47:28.772 INFO:journalctl@ceph.mon.a.smithi060.stdout: prepopulate_block_cache: 0 2024-02-26T04:47:28.772 INFO:journalctl@ceph.mon.a.smithi060.stdout: initial_auto_readahead_size: 8192 2024-02-26T04:47:28.772 INFO:journalctl@ceph.mon.a.smithi060.stdout: num_file_reads_for_auto_readahead: 2 2024-02-26T04:47:28.772 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.write_buffer_size: 33554432 2024-02-26T04:47:28.772 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_write_buffer_number: 2 2024-02-26T04:47:28.772 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compression: NoCompression 2024-02-26T04:47:28.772 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.bottommost_compression: Disabled 2024-02-26T04:47:28.772 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.prefix_extractor: nullptr 2024-02-26T04:47:28.772 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-26T04:47:28.772 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.num_levels: 7 2024-02-26T04:47:28.772 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compression_opts.level: 32767 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compression_opts.strategy: 0 2024-02-26T04:47:28.773 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compression_opts.enabled: false 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.target_file_size_base: 67108864 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-26T04:47:28.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.arena_block_size: 1048576 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.disable_auto_compactions: 0 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-26T04:47:28.775 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-26T04:47:28.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-26T04:47:28.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-26T04:47:28.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-26T04:47:28.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-26T04:47:28.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-26T04:47:28.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-26T04:47:28.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-26T04:47:28.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.table_properties_collectors: 2024-02-26T04:47:28.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.inplace_update_support: 0 2024-02-26T04:47:28.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-26T04:47:28.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-26T04:47:28.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-26T04:47:28.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.bloom_locality: 0 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.max_successive_merges: 0 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.paranoid_file_checks: 0 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.force_consistency_checks: 1 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.report_bg_io_stats: 0 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.ttl: 2592000 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.enable_blob_files: false 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.min_blob_size: 0 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.blob_file_size: 268435456 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-26T04:47:28.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-26T04:47:28.778 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-26T04:47:28.778 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-26T04:47:28.778 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-26T04:47:28.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.blob_file_starting_level: 0 2024-02-26T04:47:28.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-26T04:47:28.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-26T04:47:28.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-26T04:47:28.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a607ada3-5132-4477-bcfe-e2d9f5b60946 2024-02-26T04:47:28.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708922848683251, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-26T04:47:28.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-26T04:47:28.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708922848684240, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95011, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 93009, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13793, "raw_average_key_size": 50, "raw_value_size": 85510, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708922848, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a607ada3-5132-4477-bcfe-e2d9f5b60946", "db_session_id": "862Q01CMCOMIXGY0RVNJ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-26T04:47:28.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708922848684324, "job": 1, "event": "recovery_finished"} 2024-02-26T04:47:28.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-26T04:47:28.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-26T04:47:28.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fc32edc000 2024-02-26T04:47:28.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: DB pointer 0x55fc32ec6000 2024-02-26T04:47:28.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-26T04:47:28.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-26T04:47:28.780 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** DB Stats ** 2024-02-26T04:47:28.780 INFO:journalctl@ceph.mon.a.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-26T04:47:28.780 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-26T04:47:28.780 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-26T04:47:28.780 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-26T04:47:28.780 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-26T04:47:28.780 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-26T04:47:28.781 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-26T04:47:28.781 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-02-26T04:47:28.781 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** Compaction Stats [default] ** 2024-02-26T04:47:28.781 INFO:journalctl@ceph.mon.a.smithi060.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-26T04:47:28.781 INFO:journalctl@ceph.mon.a.smithi060.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-26T04:47:28.781 INFO:journalctl@ceph.mon.a.smithi060.stdout: L0 2/0 94.62 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T04:47:28.781 INFO:journalctl@ceph.mon.a.smithi060.stdout: Sum 2/0 94.62 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T04:47:28.781 INFO:journalctl@ceph.mon.a.smithi060.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T04:47:28.781 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-02-26T04:47:28.781 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** Compaction Stats [default] ** 2024-02-26T04:47:28.781 INFO:journalctl@ceph.mon.a.smithi060.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-26T04:47:28.782 INFO:journalctl@ceph.mon.a.smithi060.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-26T04:47:28.782 INFO:journalctl@ceph.mon.a.smithi060.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-26T04:47:28.782 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-02-26T04:47:28.782 INFO:journalctl@ceph.mon.a.smithi060.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-26T04:47:28.782 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-02-26T04:47:28.782 INFO:journalctl@ceph.mon.a.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-26T04:47:28.782 INFO:journalctl@ceph.mon.a.smithi060.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-26T04:47:28.782 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-26T04:47:28.782 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-26T04:47:28.782 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-26T04:47:28.782 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-26T04:47:28.782 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative compaction: 0.00 GB write, 30.80 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-26T04:47:28.782 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval compaction: 0.00 GB write, 30.80 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-26T04:47:28.782 INFO:journalctl@ceph.mon.a.smithi060.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-26T04:47:28.783 INFO:journalctl@ceph.mon.a.smithi060.stdout: Block cache BinnedLRUCache@0x55fc32deb090#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-26T04:47:28.783 INFO:journalctl@ceph.mon.a.smithi060.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-02-26T04:47:28.783 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-02-26T04:47:28.783 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-26T04:47:28.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: starting mon.a rank 0 at public addrs [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] at bind addrs [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:47:28.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: mon.a@-1(???) e1 preinit fsid f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:47:28.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: mon.a@-1(???).mds e1 new map 2024-02-26T04:47:28.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: mon.a@-1(???).mds e1 print_map 2024-02-26T04:47:28.783 INFO:journalctl@ceph.mon.a.smithi060.stdout: e1 2024-02-26T04:47:28.783 INFO:journalctl@ceph.mon.a.smithi060.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-26T04:47:28.783 INFO:journalctl@ceph.mon.a.smithi060.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T04:47:28.783 INFO:journalctl@ceph.mon.a.smithi060.stdout: legacy client fscid: -1 2024-02-26T04:47:28.783 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-02-26T04:47:28.783 INFO:journalctl@ceph.mon.a.smithi060.stdout: No filesystems configured 2024-02-26T04:47:28.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-26T04:47:28.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T04:47:28.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T04:47:28.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T04:47:28.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-26T04:47:28.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 bash[32363]: 5b57dca8a03365e72efd9db9c4e871bf8c64c8eede3518e5747bac26991619b2 2024-02-26T04:47:28.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 systemd[1]: Started Ceph mon.a for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:47:28.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-26T04:47:28.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: monmap e1: 1 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T04:47:28.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: fsmap 2024-02-26T04:47:28.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: osdmap e1: 0 total, 0 up, 0 in 2024-02-26T04:47:28.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:28 smithi060 ceph-mon[32382]: mgrmap e1: no daemons active 2024-02-26T04:47:29.811 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.183+0000 7f190d3ac700 1 Processor -- start 2024-02-26T04:47:29.811 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.183+0000 7f190d3ac700 1 -- start start 2024-02-26T04:47:29.811 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.183+0000 7f190d3ac700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1908108080 0x7f190810a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:29.811 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.183+0000 7f190d3ac700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190807ae90 con 0x7f1908108080 2024-02-26T04:47:29.811 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.183+0000 7f1906ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1908108080 0x7f190810a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:29.811 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.183+0000 7f1906ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1908108080 0x7f190810a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45484/0 (socket says 172.21.15.60:45484) 2024-02-26T04:47:29.811 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.183+0000 7f1906ffd700 1 -- 172.21.15.60:0/801384494 learned_addr learned my addr 172.21.15.60:0/801384494 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:29.811 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.183+0000 7f1906ffd700 1 -- 172.21.15.60:0/801384494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190810a9b0 con 0x7f1908108080 2024-02-26T04:47:29.811 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.184+0000 7f1906ffd700 1 --2- 172.21.15.60:0/801384494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1908108080 0x7f190810a470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f18f80098a0 tx=0x7f18f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20f3819421f0e4d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:29.812 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.184+0000 7f1905ffb700 1 -- 172.21.15.60:0/801384494 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18f8004160 con 0x7f1908108080 2024-02-26T04:47:29.812 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.184+0000 7f1905ffb700 1 -- 172.21.15.60:0/801384494 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18f80042c0 con 0x7f1908108080 2024-02-26T04:47:29.812 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.184+0000 7f1905ffb700 1 -- 172.21.15.60:0/801384494 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18f8011ac0 con 0x7f1908108080 2024-02-26T04:47:29.812 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.184+0000 7f190d3ac700 1 -- 172.21.15.60:0/801384494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1908108080 msgr2=0x7f190810a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:29.812 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.184+0000 7f190d3ac700 1 --2- 172.21.15.60:0/801384494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1908108080 0x7f190810a470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f18f80098a0 tx=0x7f18f8009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:29.812 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.184+0000 7f190d3ac700 1 -- 172.21.15.60:0/801384494 shutdown_connections 2024-02-26T04:47:29.812 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.184+0000 7f190d3ac700 1 --2- 172.21.15.60:0/801384494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1908108080 0x7f190810a470 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:29.812 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.184+0000 7f190d3ac700 1 -- 172.21.15.60:0/801384494 >> 172.21.15.60:0/801384494 conn(0x7f1908101ad0 msgr2=0x7f1908103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:29.812 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.184+0000 7f190d3ac700 1 -- 172.21.15.60:0/801384494 shutdown_connections 2024-02-26T04:47:29.812 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.184+0000 7f190d3ac700 1 -- 172.21.15.60:0/801384494 wait complete. 2024-02-26T04:47:29.812 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.185+0000 7f190d3ac700 1 Processor -- start 2024-02-26T04:47:29.813 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.185+0000 7f190d3ac700 1 -- start start 2024-02-26T04:47:29.813 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.185+0000 7f190d3ac700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1908108080 0x7f190819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:29.813 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.185+0000 7f190d3ac700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190819db00 con 0x7f1908108080 2024-02-26T04:47:29.813 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.185+0000 7f1906ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1908108080 0x7f190819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:29.813 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.186+0000 7f1906ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1908108080 0x7f190819d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45496/0 (socket says 172.21.15.60:45496) 2024-02-26T04:47:29.813 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.186+0000 7f1906ffd700 1 -- 172.21.15.60:0/456338487 learned_addr learned my addr 172.21.15.60:0/456338487 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:29.813 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.186+0000 7f1906ffd700 1 -- 172.21.15.60:0/456338487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f8009550 con 0x7f1908108080 2024-02-26T04:47:29.813 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.186+0000 7f1906ffd700 1 --2- 172.21.15.60:0/456338487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1908108080 0x7f190819d5c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f18f8011a70 tx=0x7f18f8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:29.813 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.186+0000 7f18effff700 1 -- 172.21.15.60:0/456338487 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18f80039c0 con 0x7f1908108080 2024-02-26T04:47:29.813 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.186+0000 7f18effff700 1 -- 172.21.15.60:0/456338487 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18f8003b20 con 0x7f1908108080 2024-02-26T04:47:29.813 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.186+0000 7f190d3ac700 1 -- 172.21.15.60:0/456338487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190819dd00 con 0x7f1908108080 2024-02-26T04:47:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.186+0000 7f18effff700 1 -- 172.21.15.60:0/456338487 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18f8024610 con 0x7f1908108080 2024-02-26T04:47:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.186+0000 7f190d3ac700 1 -- 172.21.15.60:0/456338487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190819e1a0 con 0x7f1908108080 2024-02-26T04:47:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.186+0000 7f18effff700 1 -- 172.21.15.60:0/456338487 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f18f8024bb0 con 0x7f1908108080 2024-02-26T04:47:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.186+0000 7f18effff700 1 -- 172.21.15.60:0/456338487 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f18f80198d0 con 0x7f1908108080 2024-02-26T04:47:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.187+0000 7f190d3ac700 1 -- 172.21.15.60:0/456338487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190810b7b0 con 0x7f1908108080 2024-02-26T04:47:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.190+0000 7f18effff700 1 -- 172.21.15.60:0/456338487 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f18f801aa10 con 0x7f1908108080 2024-02-26T04:47:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.260+0000 7f190d3ac700 1 -- 172.21.15.60:0/456338487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f190810b9c0 con 0x7f1908108080 2024-02-26T04:47:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.262+0000 7f18effff700 1 -- 172.21.15.60:0/456338487 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f18f801abb0 con 0x7f1908108080 2024-02-26T04:47:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.263+0000 7f190d3ac700 1 -- 172.21.15.60:0/456338487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1908108080 msgr2=0x7f190819d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.263+0000 7f190d3ac700 1 --2- 172.21.15.60:0/456338487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1908108080 0x7f190819d5c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f18f8011a70 tx=0x7f18f8011b50 comp rx=0 tx=0).stop 2024-02-26T04:47:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.263+0000 7f190d3ac700 1 -- 172.21.15.60:0/456338487 shutdown_connections 2024-02-26T04:47:29.815 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.263+0000 7f190d3ac700 1 --2- 172.21.15.60:0/456338487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1908108080 0x7f190819d5c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:29.815 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.263+0000 7f190d3ac700 1 -- 172.21.15.60:0/456338487 >> 172.21.15.60:0/456338487 conn(0x7f1908101ad0 msgr2=0x7f1908102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:29.815 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.263+0000 7f190d3ac700 1 -- 172.21.15.60:0/456338487 shutdown_connections 2024-02-26T04:47:29.815 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:29.263+0000 7f190d3ac700 1 -- 172.21.15.60:0/456338487 wait complete. 2024-02-26T04:47:29.815 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-26T04:47:29.835 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-26T04:47:29.835 INFO:teuthology.orchestra.run.smithi060.stdout:Creating mgr... 2024-02-26T04:47:29.835 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-26T04:47:29.836 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-26T04:47:30.078 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f12e2206-d461-11ee-95c3-87774f69a715@mgr.x 2024-02-26T04:47:30.078 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f12e2206-d461-11ee-95c3-87774f69a715@mgr.x.service: Unit ceph-f12e2206-d461-11ee-95c3-87774f69a715@mgr.x.service not loaded. 2024-02-26T04:47:30.225 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f12e2206-d461-11ee-95c3-87774f69a715.target.wants/ceph-f12e2206-d461-11ee-95c3-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-f12e2206-d461-11ee-95c3-87774f69a715@.service. 2024-02-26T04:47:30.454 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:30 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/456338487' entity='client.admin' 2024-02-26T04:47:30.694 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T04:47:30.695 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-02-26T04:47:30.702 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T04:47:30.702 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-02-26T04:47:30.702 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-02-26T04:47:30.702 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to enable service . firewalld.service is not available 2024-02-26T04:47:30.710 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T04:47:30.710 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-02-26T04:47:30.717 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T04:47:30.717 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-02-26T04:47:30.718 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-02-26T04:47:30.718 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-26T04:47:30.718 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr to start... 2024-02-26T04:47:30.718 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr... 2024-02-26T04:47:30.730 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:30 smithi060 systemd[1]: Started Ceph mgr.x for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:47:31.468 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:31 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:31.185+0000 7f5f7cc77200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T04:47:31.469 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:31 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:31.282+0000 7f5f7cc77200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T04:47:31.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:31 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3348574646' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T04:47:32.051 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:31 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:31.953+0000 7f5f7cc77200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T04:47:32.051 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:32 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:32.048+0000 7f5f7cc77200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T04:47:32.087 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T04:47:32.087 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T04:47:32.087 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "f12e2206-d461-11ee-95c3-87774f69a715", 2024-02-26T04:47:32.087 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T04:47:32.087 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T04:47:32.087 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T04:47:32.087 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T04:47:32.087 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:32.087 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T04:47:32.087 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T04:47:32.087 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-02-26T04:47:32.088 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T04:47:32.088 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T04:47:32.088 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T04:47:32.088 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T04:47:32.088 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-26T04:47:32.088 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T04:47:32.088 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:32.088 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T04:47:32.088 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T04:47:32.088 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:32.088 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T04:47:32.088 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:32.088 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T04:47:32.089 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T04:47:32.089 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T04:47:32.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T04:47:32.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T04:47:32.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T04:47:32.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:32.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T04:47:32.097 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T04:47:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T04:47:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T04:47:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T04:47:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T04:47:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T04:47:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T04:47:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T04:47:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T04:47:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T04:47:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T04:47:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T04:47:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-26T04:47:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T04:47:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T04:47:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T04:47:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T04:47:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T04:47:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T04:47:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T04:47:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T04:47:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T04:47:22.522314+0000", 2024-02-26T04:47:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T04:47:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:32.100 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T04:47:32.100 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T04:47:32.100 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.385+0000 7f597c270700 1 Processor -- start 2024-02-26T04:47:32.100 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.385+0000 7f597c270700 1 -- start start 2024-02-26T04:47:32.100 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.385+0000 7f597c270700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5974106450 0x7f5974106870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:32.100 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.385+0000 7f597c270700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5974106e40 con 0x7f5974106450 2024-02-26T04:47:32.100 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.386+0000 7f597a00c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5974106450 0x7f5974106870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:32.100 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.386+0000 7f597a00c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5974106450 0x7f5974106870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36606/0 (socket says 172.21.15.60:36606) 2024-02-26T04:47:32.100 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.386+0000 7f597a00c700 1 -- 172.21.15.60:0/4118683284 learned_addr learned my addr 172.21.15.60:0/4118683284 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:32.100 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.386+0000 7f597a00c700 1 -- 172.21.15.60:0/4118683284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5974107650 con 0x7f5974106450 2024-02-26T04:47:32.100 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.386+0000 7f597a00c700 1 --2- 172.21.15.60:0/4118683284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5974106450 0x7f5974106870 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f59700098a0 tx=0x7f5970009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a664b11b9fca868 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:32.100 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.386+0000 7f597900a700 1 -- 172.21.15.60:0/4118683284 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5970004160 con 0x7f5974106450 2024-02-26T04:47:32.101 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.387+0000 7f597900a700 1 -- 172.21.15.60:0/4118683284 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59700042c0 con 0x7f5974106450 2024-02-26T04:47:32.101 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.387+0000 7f597900a700 1 -- 172.21.15.60:0/4118683284 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5970011ac0 con 0x7f5974106450 2024-02-26T04:47:32.101 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.387+0000 7f597c270700 1 -- 172.21.15.60:0/4118683284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5974106450 msgr2=0x7f5974106870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:32.101 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.387+0000 7f597c270700 1 --2- 172.21.15.60:0/4118683284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5974106450 0x7f5974106870 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f59700098a0 tx=0x7f5970009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:32.101 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.387+0000 7f597c270700 1 -- 172.21.15.60:0/4118683284 shutdown_connections 2024-02-26T04:47:32.101 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.387+0000 7f597c270700 1 --2- 172.21.15.60:0/4118683284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5974106450 0x7f5974106870 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:32.101 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.387+0000 7f597c270700 1 -- 172.21.15.60:0/4118683284 >> 172.21.15.60:0/4118683284 conn(0x7f5974101a10 msgr2=0x7f5974103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:32.101 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.387+0000 7f597c270700 1 -- 172.21.15.60:0/4118683284 shutdown_connections 2024-02-26T04:47:32.101 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.387+0000 7f597c270700 1 -- 172.21.15.60:0/4118683284 wait complete. 2024-02-26T04:47:32.102 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.388+0000 7f597c270700 1 Processor -- start 2024-02-26T04:47:32.102 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.388+0000 7f597c270700 1 -- start start 2024-02-26T04:47:32.102 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.388+0000 7f597c270700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5974106450 0x7f597419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:32.102 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.388+0000 7f597c270700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597419b980 con 0x7f5974106450 2024-02-26T04:47:32.102 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.388+0000 7f597a00c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5974106450 0x7f597419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:32.102 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.388+0000 7f597a00c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5974106450 0x7f597419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36614/0 (socket says 172.21.15.60:36614) 2024-02-26T04:47:32.102 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.388+0000 7f597a00c700 1 -- 172.21.15.60:0/3348574646 learned_addr learned my addr 172.21.15.60:0/3348574646 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:32.102 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.389+0000 7f597a00c700 1 -- 172.21.15.60:0/3348574646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5970009550 con 0x7f5974106450 2024-02-26T04:47:32.102 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.389+0000 7f597a00c700 1 --2- 172.21.15.60:0/3348574646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5974106450 0x7f597419b440 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5970011780 tx=0x7f5970011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:32.102 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.389+0000 7f59677fe700 1 -- 172.21.15.60:0/3348574646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5970003af0 con 0x7f5974106450 2024-02-26T04:47:32.103 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.389+0000 7f59677fe700 1 -- 172.21.15.60:0/3348574646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5970003c50 con 0x7f5974106450 2024-02-26T04:47:32.103 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.389+0000 7f597c270700 1 -- 172.21.15.60:0/3348574646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597419bb80 con 0x7f5974106450 2024-02-26T04:47:32.103 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.389+0000 7f59677fe700 1 -- 172.21.15.60:0/3348574646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59700197e0 con 0x7f5974106450 2024-02-26T04:47:32.103 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.389+0000 7f597c270700 1 -- 172.21.15.60:0/3348574646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597419c020 con 0x7f5974106450 2024-02-26T04:47:32.103 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.389+0000 7f59677fe700 1 -- 172.21.15.60:0/3348574646 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5970025460 con 0x7f5974106450 2024-02-26T04:47:32.103 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.389+0000 7f59677fe700 1 -- 172.21.15.60:0/3348574646 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f597000bb70 con 0x7f5974106450 2024-02-26T04:47:32.103 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.390+0000 7f597c270700 1 -- 172.21.15.60:0/3348574646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59741954f0 con 0x7f5974106450 2024-02-26T04:47:32.103 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.392+0000 7f59677fe700 1 -- 172.21.15.60:0/3348574646 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f597002a060 con 0x7f5974106450 2024-02-26T04:47:32.103 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.465+0000 7f597c270700 1 -- 172.21.15.60:0/3348574646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f597410ab10 con 0x7f5974106450 2024-02-26T04:47:32.103 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.466+0000 7f59677fe700 1 -- 172.21.15.60:0/3348574646 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f5970019db0 con 0x7f5974106450 2024-02-26T04:47:32.103 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.467+0000 7f597c270700 1 -- 172.21.15.60:0/3348574646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5974106450 msgr2=0x7f597419b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:32.103 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.467+0000 7f597c270700 1 --2- 172.21.15.60:0/3348574646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5974106450 0x7f597419b440 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5970011780 tx=0x7f5970011b40 comp rx=0 tx=0).stop 2024-02-26T04:47:32.103 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.467+0000 7f597c270700 1 -- 172.21.15.60:0/3348574646 shutdown_connections 2024-02-26T04:47:32.104 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.467+0000 7f597c270700 1 --2- 172.21.15.60:0/3348574646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5974106450 0x7f597419b440 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:32.104 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.467+0000 7f597c270700 1 -- 172.21.15.60:0/3348574646 >> 172.21.15.60:0/3348574646 conn(0x7f5974101a10 msgr2=0x7f5974102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:32.104 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.467+0000 7f597c270700 1 -- 172.21.15.60:0/3348574646 shutdown_connections 2024-02-26T04:47:32.104 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:31.467+0000 7f597c270700 1 -- 172.21.15.60:0/3348574646 wait complete. 2024-02-26T04:47:32.104 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (1/15)... 2024-02-26T04:47:32.381 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:32 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:32.197+0000 7f5f7cc77200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T04:47:32.382 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:32 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:32.294+0000 7f5f7cc77200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T04:47:32.382 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:32 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:32.378+0000 7f5f7cc77200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T04:47:32.949 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:32 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:32.647+0000 7f5f7cc77200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T04:47:32.949 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:32 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:32.759+0000 7f5f7cc77200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T04:47:32.949 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:32 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:32.848+0000 7f5f7cc77200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T04:47:32.949 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:32 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:32.946+0000 7f5f7cc77200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T04:47:33.229 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:33 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:33.123+0000 7f5f7cc77200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T04:47:34.458 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:34 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:34.125+0000 7f5f7cc77200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T04:47:34.458 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:34 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:34.262+0000 7f5f7cc77200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T04:47:34.728 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:34 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:34.637+0000 7f5f7cc77200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T04:47:34.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:34 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3476738183' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T04:47:34.980 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:34 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:34.753+0000 7f5f7cc77200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T04:47:34.980 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:34 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:34.959+0000 7f5f7cc77200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T04:47:35.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T04:47:35.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T04:47:35.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "f12e2206-d461-11ee-95c3-87774f69a715", 2024-02-26T04:47:35.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T04:47:35.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T04:47:35.284 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T04:47:35.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T04:47:35.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:35.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T04:47:35.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T04:47:35.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-02-26T04:47:35.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T04:47:35.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T04:47:35.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T04:47:35.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T04:47:35.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-26T04:47:35.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T04:47:35.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:35.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T04:47:35.285 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T04:47:35.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:35.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T04:47:35.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:35.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T04:47:35.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T04:47:35.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T04:47:35.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T04:47:35.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T04:47:35.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T04:47:35.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:35.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T04:47:35.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T04:47:35.286 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T04:47:35.287 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T04:47:35.287 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T04:47:35.287 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T04:47:35.287 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T04:47:35.287 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T04:47:35.287 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T04:47:35.287 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:35.287 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T04:47:35.287 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:35.287 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T04:47:35.287 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T04:47:35.287 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:35.287 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T04:47:35.287 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-26T04:47:35.288 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T04:47:35.288 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T04:47:35.288 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T04:47:35.288 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T04:47:35.288 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T04:47:35.288 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T04:47:35.288 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T04:47:35.288 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:35.288 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T04:47:35.288 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:35.288 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T04:47:22.522314+0000", 2024-02-26T04:47:35.288 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T04:47:35.288 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:35.289 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T04:47:35.289 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T04:47:35.289 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.560+0000 7f6db763f700 1 Processor -- start 2024-02-26T04:47:35.289 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.560+0000 7f6db763f700 1 -- start start 2024-02-26T04:47:35.289 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.560+0000 7f6db763f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6db0105850 0x7f6db0105c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:35.289 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.560+0000 7f6db763f700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db0106240 con 0x7f6db0105850 2024-02-26T04:47:35.289 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.560+0000 7f6db53db700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6db0105850 0x7f6db0105c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:35.289 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.561+0000 7f6db53db700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6db0105850 0x7f6db0105c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36626/0 (socket says 172.21.15.60:36626) 2024-02-26T04:47:35.289 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.561+0000 7f6db53db700 1 -- 172.21.15.60:0/3251506973 learned_addr learned my addr 172.21.15.60:0/3251506973 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:35.289 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.561+0000 7f6db53db700 1 -- 172.21.15.60:0/3251506973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db0106380 con 0x7f6db0105850 2024-02-26T04:47:35.289 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.561+0000 7f6db53db700 1 --2- 172.21.15.60:0/3251506973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6db0105850 0x7f6db0105c70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0098a0 tx=0x7f6dac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e17b9f6b70b1ad server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:35.289 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.561+0000 7f6da7fff700 1 -- 172.21.15.60:0/3251506973 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dac004160 con 0x7f6db0105850 2024-02-26T04:47:35.289 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.561+0000 7f6da7fff700 1 -- 172.21.15.60:0/3251506973 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dac0042c0 con 0x7f6db0105850 2024-02-26T04:47:35.290 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.561+0000 7f6da7fff700 1 -- 172.21.15.60:0/3251506973 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dac011ac0 con 0x7f6db0105850 2024-02-26T04:47:35.290 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.562+0000 7f6db763f700 1 -- 172.21.15.60:0/3251506973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6db0105850 msgr2=0x7f6db0105c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:35.290 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.562+0000 7f6db763f700 1 --2- 172.21.15.60:0/3251506973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6db0105850 0x7f6db0105c70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0098a0 tx=0x7f6dac009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:35.290 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.562+0000 7f6db763f700 1 -- 172.21.15.60:0/3251506973 shutdown_connections 2024-02-26T04:47:35.290 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.562+0000 7f6db763f700 1 --2- 172.21.15.60:0/3251506973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6db0105850 0x7f6db0105c70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:35.290 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.562+0000 7f6db763f700 1 -- 172.21.15.60:0/3251506973 >> 172.21.15.60:0/3251506973 conn(0x7f6db01011e0 msgr2=0x7f6db0103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:35.290 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.562+0000 7f6db763f700 1 -- 172.21.15.60:0/3251506973 shutdown_connections 2024-02-26T04:47:35.290 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.562+0000 7f6db763f700 1 -- 172.21.15.60:0/3251506973 wait complete. 2024-02-26T04:47:35.290 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.563+0000 7f6db763f700 1 Processor -- start 2024-02-26T04:47:35.290 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.563+0000 7f6db763f700 1 -- start start 2024-02-26T04:47:35.290 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.563+0000 7f6db763f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6db0105850 0x7f6db0108100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:35.290 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.563+0000 7f6db763f700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db0108640 con 0x7f6db0105850 2024-02-26T04:47:35.290 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.563+0000 7f6db53db700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6db0105850 0x7f6db0108100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:35.290 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.563+0000 7f6db53db700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6db0105850 0x7f6db0108100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36628/0 (socket says 172.21.15.60:36628) 2024-02-26T04:47:35.291 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.563+0000 7f6db53db700 1 -- 172.21.15.60:0/3476738183 learned_addr learned my addr 172.21.15.60:0/3476738183 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:35.291 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.564+0000 7f6db53db700 1 -- 172.21.15.60:0/3476738183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac009550 con 0x7f6db0105850 2024-02-26T04:47:35.291 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.564+0000 7f6db53db700 1 --2- 172.21.15.60:0/3476738183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6db0105850 0x7f6db0108100 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009fb0 tx=0x7f6dac011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:35.291 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.564+0000 7f6da67fc700 1 -- 172.21.15.60:0/3476738183 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dac0038b0 con 0x7f6db0105850 2024-02-26T04:47:35.291 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.564+0000 7f6db763f700 1 -- 172.21.15.60:0/3476738183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db0106830 con 0x7f6db0105850 2024-02-26T04:47:35.291 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.564+0000 7f6da67fc700 1 -- 172.21.15.60:0/3476738183 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dac003a10 con 0x7f6db0105850 2024-02-26T04:47:35.291 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.564+0000 7f6da67fc700 1 -- 172.21.15.60:0/3476738183 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dac0195e0 con 0x7f6db0105850 2024-02-26T04:47:35.291 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.564+0000 7f6db763f700 1 -- 172.21.15.60:0/3476738183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db0106cd0 con 0x7f6db0105850 2024-02-26T04:47:35.291 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.565+0000 7f6da67fc700 1 -- 172.21.15.60:0/3476738183 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6dac019c70 con 0x7f6db0105850 2024-02-26T04:47:35.291 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.565+0000 7f6da67fc700 1 -- 172.21.15.60:0/3476738183 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6dac0249f0 con 0x7f6db0105850 2024-02-26T04:47:35.291 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.565+0000 7f6db763f700 1 -- 172.21.15.60:0/3476738183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d94005360 con 0x7f6db0105850 2024-02-26T04:47:35.291 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.569+0000 7f6da67fc700 1 -- 172.21.15.60:0/3476738183 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6dac003b80 con 0x7f6db0105850 2024-02-26T04:47:35.291 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.675+0000 7f6db763f700 1 -- 172.21.15.60:0/3476738183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6d940058e0 con 0x7f6db0105850 2024-02-26T04:47:35.292 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.675+0000 7f6da67fc700 1 -- 172.21.15.60:0/3476738183 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f6dac035090 con 0x7f6db0105850 2024-02-26T04:47:35.292 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.677+0000 7f6db763f700 1 -- 172.21.15.60:0/3476738183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6db0105850 msgr2=0x7f6db0108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:35.292 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.677+0000 7f6db763f700 1 --2- 172.21.15.60:0/3476738183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6db0105850 0x7f6db0108100 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009fb0 tx=0x7f6dac011b20 comp rx=0 tx=0).stop 2024-02-26T04:47:35.292 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.677+0000 7f6db763f700 1 -- 172.21.15.60:0/3476738183 shutdown_connections 2024-02-26T04:47:35.292 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.677+0000 7f6db763f700 1 --2- 172.21.15.60:0/3476738183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6db0105850 0x7f6db0108100 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:35.292 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.677+0000 7f6db763f700 1 -- 172.21.15.60:0/3476738183 >> 172.21.15.60:0/3476738183 conn(0x7f6db01011e0 msgr2=0x7f6db0101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:35.292 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.677+0000 7f6db763f700 1 -- 172.21.15.60:0/3476738183 shutdown_connections 2024-02-26T04:47:35.292 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:34.677+0000 7f6db763f700 1 -- 172.21.15.60:0/3476738183 wait complete. 2024-02-26T04:47:35.292 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (2/15)... 2024-02-26T04:47:35.459 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:35 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:35.151+0000 7f5f7cc77200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T04:47:35.722 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:35 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:35.457+0000 7f5f7cc77200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T04:47:35.723 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:35 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:35.720+0000 7f5f7cc77200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T04:47:35.979 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:35 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:35.960+0000 7f5f7cc77200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T04:47:37.229 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:36 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:36.785+0000 7f5f7cc77200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T04:47:37.721 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:37.393+0000 7f5f7cc77200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T04:47:37.722 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:37.493+0000 7f5f7cc77200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T04:47:37.722 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:37.719+0000 7f5f7cc77200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T04:47:37.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-mon[32382]: Activating manager daemon x 2024-02-26T04:47:37.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-mon[32382]: mgrmap e2: x(active, starting, since 0.00340971s) 2024-02-26T04:47:37.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-mon[32382]: from='mgr.14100 172.21.15.60:0/4053441334' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T04:47:37.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-mon[32382]: from='mgr.14100 172.21.15.60:0/4053441334' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T04:47:37.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-mon[32382]: from='mgr.14100 172.21.15.60:0/4053441334' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T04:47:37.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-mon[32382]: from='mgr.14100 172.21.15.60:0/4053441334' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:47:37.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-mon[32382]: from='mgr.14100 172.21.15.60:0/4053441334' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T04:47:37.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-mon[32382]: Manager daemon x is now available 2024-02-26T04:47:37.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-mon[32382]: from='mgr.14100 172.21.15.60:0/4053441334' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T04:47:37.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-mon[32382]: from='mgr.14100 172.21.15.60:0/4053441334' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T04:47:37.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-mon[32382]: from='mgr.14100 172.21.15.60:0/4053441334' entity='mgr.x' 2024-02-26T04:47:37.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-mon[32382]: from='mgr.14100 172.21.15.60:0/4053441334' entity='mgr.x' 2024-02-26T04:47:37.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:37 smithi060 ceph-mon[32382]: from='mgr.14100 172.21.15.60:0/4053441334' entity='mgr.x' 2024-02-26T04:47:38.263 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T04:47:38.264 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T04:47:38.264 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "f12e2206-d461-11ee-95c3-87774f69a715", 2024-02-26T04:47:38.264 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T04:47:38.264 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T04:47:38.264 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T04:47:38.264 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T04:47:38.264 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:38.264 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T04:47:38.264 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T04:47:38.264 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-02-26T04:47:38.264 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T04:47:38.264 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T04:47:38.264 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T04:47:38.265 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T04:47:38.265 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-02-26T04:47:38.265 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T04:47:38.265 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:38.265 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T04:47:38.265 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T04:47:38.265 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:38.265 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T04:47:38.265 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:38.265 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T04:47:38.265 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T04:47:38.265 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T04:47:38.265 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T04:47:38.265 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:38.267 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T04:47:22.522314+0000", 2024-02-26T04:47:38.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T04:47:38.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:38.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T04:47:38.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T04:47:38.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.739+0000 7f5fb3bb3700 1 Processor -- start 2024-02-26T04:47:38.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.739+0000 7f5fb3bb3700 1 -- start start 2024-02-26T04:47:38.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.740+0000 7f5fb3bb3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fac106480 0x7f5fac1068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:38.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.740+0000 7f5fb3bb3700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac106e70 con 0x7f5fac106480 2024-02-26T04:47:38.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.740+0000 7f5fb194f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fac106480 0x7f5fac1068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:38.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.740+0000 7f5fb194f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fac106480 0x7f5fac1068a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36662/0 (socket says 172.21.15.60:36662) 2024-02-26T04:47:38.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.740+0000 7f5fb194f700 1 -- 172.21.15.60:0/3999387101 learned_addr learned my addr 172.21.15.60:0/3999387101 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:38.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.740+0000 7f5fb194f700 1 -- 172.21.15.60:0/3999387101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac107680 con 0x7f5fac106480 2024-02-26T04:47:38.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.741+0000 7f5fb194f700 1 --2- 172.21.15.60:0/3999387101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fac106480 0x7f5fac1068a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800bb40 tx=0x7f5fa800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37436547f64fdacc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:38.268 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.741+0000 7f5fb094d700 1 -- 172.21.15.60:0/3999387101 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fa800dbe0 con 0x7f5fac106480 2024-02-26T04:47:38.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.741+0000 7f5fb094d700 1 -- 172.21.15.60:0/3999387101 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fa80044f0 con 0x7f5fac106480 2024-02-26T04:47:38.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.741+0000 7f5fb094d700 1 -- 172.21.15.60:0/3999387101 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fa8010470 con 0x7f5fac106480 2024-02-26T04:47:38.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.741+0000 7f5fb3bb3700 1 -- 172.21.15.60:0/3999387101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fac106480 msgr2=0x7f5fac1068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:38.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.741+0000 7f5fb3bb3700 1 --2- 172.21.15.60:0/3999387101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fac106480 0x7f5fac1068a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800bb40 tx=0x7f5fa800be50 comp rx=0 tx=0).stop 2024-02-26T04:47:38.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.742+0000 7f5fb3bb3700 1 -- 172.21.15.60:0/3999387101 shutdown_connections 2024-02-26T04:47:38.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.742+0000 7f5fb3bb3700 1 --2- 172.21.15.60:0/3999387101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fac106480 0x7f5fac1068a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:38.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.742+0000 7f5fb3bb3700 1 -- 172.21.15.60:0/3999387101 >> 172.21.15.60:0/3999387101 conn(0x7f5fac101a00 msgr2=0x7f5fac103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:38.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.742+0000 7f5fb3bb3700 1 -- 172.21.15.60:0/3999387101 shutdown_connections 2024-02-26T04:47:38.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.742+0000 7f5fb3bb3700 1 -- 172.21.15.60:0/3999387101 wait complete. 2024-02-26T04:47:38.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.742+0000 7f5fb3bb3700 1 Processor -- start 2024-02-26T04:47:38.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.742+0000 7f5fb3bb3700 1 -- start start 2024-02-26T04:47:38.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.742+0000 7f5fb3bb3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fac106480 0x7f5fac199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:38.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.742+0000 7f5fb3bb3700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac199750 con 0x7f5fac106480 2024-02-26T04:47:38.269 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.743+0000 7f5fb194f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fac106480 0x7f5fac199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:38.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.743+0000 7f5fb194f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fac106480 0x7f5fac199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36668/0 (socket says 172.21.15.60:36668) 2024-02-26T04:47:38.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.743+0000 7f5fb194f700 1 -- 172.21.15.60:0/1647010293 learned_addr learned my addr 172.21.15.60:0/1647010293 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:38.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.743+0000 7f5fb194f700 1 -- 172.21.15.60:0/1647010293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa800b7f0 con 0x7f5fac106480 2024-02-26T04:47:38.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.743+0000 7f5fb194f700 1 --2- 172.21.15.60:0/1647010293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fac106480 0x7f5fac199210 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800b340 tx=0x7f5fa8010520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:38.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.743+0000 7f5fa2ffd700 1 -- 172.21.15.60:0/1647010293 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fa801e070 con 0x7f5fac106480 2024-02-26T04:47:38.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.743+0000 7f5fa2ffd700 1 -- 172.21.15.60:0/1647010293 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fa8004250 con 0x7f5fac106480 2024-02-26T04:47:38.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.743+0000 7f5fb3bb3700 1 -- 172.21.15.60:0/1647010293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fac199950 con 0x7f5fac106480 2024-02-26T04:47:38.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.743+0000 7f5fa2ffd700 1 -- 172.21.15.60:0/1647010293 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fa8022a80 con 0x7f5fac106480 2024-02-26T04:47:38.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.743+0000 7f5fb3bb3700 1 -- 172.21.15.60:0/1647010293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fac199df0 con 0x7f5fac106480 2024-02-26T04:47:38.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.744+0000 7f5fa2ffd700 1 -- 172.21.15.60:0/1647010293 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 2) v1 ==== 45688+0+0 (secure 0 0 0) 0x7f5fa802c440 con 0x7f5fac106480 2024-02-26T04:47:38.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.744+0000 7f5fb3bb3700 1 -- 172.21.15.60:0/1647010293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fac079ff0 con 0x7f5fac106480 2024-02-26T04:47:38.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.744+0000 7f5fa2ffd700 1 -- 172.21.15.60:0/1647010293 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5fa804eb80 con 0x7f5fac106480 2024-02-26T04:47:38.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.746+0000 7f5fa2ffd700 1 -- 172.21.15.60:0/1647010293 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5fa802a720 con 0x7f5fac106480 2024-02-26T04:47:38.270 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.826+0000 7f5fb3bb3700 1 -- 172.21.15.60:0/1647010293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5fac1928e0 con 0x7f5fac106480 2024-02-26T04:47:38.271 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.827+0000 7f5fa2ffd700 1 -- 172.21.15.60:0/1647010293 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f5fa8027070 con 0x7f5fac106480 2024-02-26T04:47:38.271 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.828+0000 7f5fb3bb3700 1 -- 172.21.15.60:0/1647010293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fac106480 msgr2=0x7f5fac199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:38.271 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.828+0000 7f5fb3bb3700 1 --2- 172.21.15.60:0/1647010293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fac106480 0x7f5fac199210 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800b340 tx=0x7f5fa8010520 comp rx=0 tx=0).stop 2024-02-26T04:47:38.271 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.828+0000 7f5fb3bb3700 1 -- 172.21.15.60:0/1647010293 shutdown_connections 2024-02-26T04:47:38.271 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.828+0000 7f5fb3bb3700 1 --2- 172.21.15.60:0/1647010293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fac106480 0x7f5fac199210 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:38.271 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.828+0000 7f5fb3bb3700 1 -- 172.21.15.60:0/1647010293 >> 172.21.15.60:0/1647010293 conn(0x7f5fac101a00 msgr2=0x7f5fac18f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:38.271 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.828+0000 7f5fb3bb3700 1 -- 172.21.15.60:0/1647010293 shutdown_connections 2024-02-26T04:47:38.271 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:37.828+0000 7f5fb3bb3700 1 -- 172.21.15.60:0/1647010293 wait complete. 2024-02-26T04:47:38.271 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (3/15)... 2024-02-26T04:47:39.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:38 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1647010293' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T04:47:39.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:38 smithi060 ceph-mon[32382]: mgrmap e3: x(active, since 1.00768s) 2024-02-26T04:47:40.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:40 smithi060 ceph-mon[32382]: mgrmap e4: x(active, since 2s) 2024-02-26T04:47:41.596 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T04:47:41.596 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T04:47:41.597 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "f12e2206-d461-11ee-95c3-87774f69a715", 2024-02-26T04:47:41.597 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-02-26T04:47:41.597 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-26T04:47:41.597 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-26T04:47:41.597 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-26T04:47:41.597 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:41.597 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-26T04:47:41.597 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-26T04:47:41.597 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-02-26T04:47:41.597 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T04:47:41.597 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-26T04:47:41.597 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-02-26T04:47:41.597 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T04:47:41.598 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-02-26T04:47:41.598 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-26T04:47:41.598 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:41.598 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-26T04:47:41.598 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-26T04:47:41.598 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:41.598 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-26T04:47:41.598 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:41.598 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-26T04:47:41.599 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-26T04:47:41.599 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-26T04:47:41.608 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-26T04:47:41.608 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-26T04:47:41.608 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-26T04:47:41.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-26T04:47:41.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:41.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-26T04:47:41.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-26T04:47:41.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-26T04:47:41.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-26T04:47:41.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-26T04:47:41.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-26T04:47:41.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-02-26T04:47:41.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-02-26T04:47:41.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T04:47:41.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:41.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-26T04:47:41.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-26T04:47:41.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-02-26T04:47:22.522314+0000", 2024-02-26T04:47:41.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-26T04:47:41.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-02-26T04:47:41.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-26T04:47:41.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T04:47:41.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.693+0000 7f8a55ddf700 1 Processor -- start 2024-02-26T04:47:41.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.693+0000 7f8a55ddf700 1 -- start start 2024-02-26T04:47:41.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.693+0000 7f8a55ddf700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a501080c0 0x7f8a5010a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:41.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.693+0000 7f8a55ddf700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5007adf0 con 0x7f8a501080c0 2024-02-26T04:47:41.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.693+0000 7f8a4f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a501080c0 0x7f8a5010a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:41.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.693+0000 7f8a4f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a501080c0 0x7f8a5010a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35070/0 (socket says 172.21.15.60:35070) 2024-02-26T04:47:41.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.693+0000 7f8a4f7fe700 1 -- 172.21.15.60:0/986628200 learned_addr learned my addr 172.21.15.60:0/986628200 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:41.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.694+0000 7f8a4f7fe700 1 -- 172.21.15.60:0/986628200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5007af30 con 0x7f8a501080c0 2024-02-26T04:47:41.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.694+0000 7f8a4f7fe700 1 --2- 172.21.15.60:0/986628200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a501080c0 0x7f8a5010a4b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8a400098a0 tx=0x7f8a40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7f49b29f85bbe1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:41.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.694+0000 7f8a4e7fc700 1 -- 172.21.15.60:0/986628200 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a40004160 con 0x7f8a501080c0 2024-02-26T04:47:41.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.694+0000 7f8a4e7fc700 1 -- 172.21.15.60:0/986628200 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a400042c0 con 0x7f8a501080c0 2024-02-26T04:47:41.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.694+0000 7f8a4e7fc700 1 -- 172.21.15.60:0/986628200 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a40011ac0 con 0x7f8a501080c0 2024-02-26T04:47:41.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.695+0000 7f8a55ddf700 1 -- 172.21.15.60:0/986628200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a501080c0 msgr2=0x7f8a5010a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:41.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.695+0000 7f8a55ddf700 1 --2- 172.21.15.60:0/986628200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a501080c0 0x7f8a5010a4b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8a400098a0 tx=0x7f8a40009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:41.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.695+0000 7f8a55ddf700 1 -- 172.21.15.60:0/986628200 shutdown_connections 2024-02-26T04:47:41.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.695+0000 7f8a55ddf700 1 --2- 172.21.15.60:0/986628200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a501080c0 0x7f8a5010a4b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:41.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.695+0000 7f8a55ddf700 1 -- 172.21.15.60:0/986628200 >> 172.21.15.60:0/986628200 conn(0x7f8a50101a70 msgr2=0x7f8a50103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:41.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.695+0000 7f8a55ddf700 1 -- 172.21.15.60:0/986628200 shutdown_connections 2024-02-26T04:47:41.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.695+0000 7f8a55ddf700 1 -- 172.21.15.60:0/986628200 wait complete. 2024-02-26T04:47:41.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.695+0000 7f8a55ddf700 1 Processor -- start 2024-02-26T04:47:41.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.696+0000 7f8a55ddf700 1 -- start start 2024-02-26T04:47:41.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.696+0000 7f8a55ddf700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a501080c0 0x7f8a5019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:41.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.696+0000 7f8a55ddf700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5019b8c0 con 0x7f8a501080c0 2024-02-26T04:47:41.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.696+0000 7f8a4f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a501080c0 0x7f8a5019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:41.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.696+0000 7f8a4f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a501080c0 0x7f8a5019b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35072/0 (socket says 172.21.15.60:35072) 2024-02-26T04:47:41.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.696+0000 7f8a4f7fe700 1 -- 172.21.15.60:0/268084330 learned_addr learned my addr 172.21.15.60:0/268084330 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:41.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.696+0000 7f8a4f7fe700 1 -- 172.21.15.60:0/268084330 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a40009550 con 0x7f8a501080c0 2024-02-26T04:47:41.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.696+0000 7f8a4f7fe700 1 --2- 172.21.15.60:0/268084330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a501080c0 0x7f8a5019b380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009fb0 tx=0x7f8a40011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:41.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.697+0000 7f8a4cff9700 1 -- 172.21.15.60:0/268084330 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a400038b0 con 0x7f8a501080c0 2024-02-26T04:47:41.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.697+0000 7f8a4cff9700 1 -- 172.21.15.60:0/268084330 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a40003a10 con 0x7f8a501080c0 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.697+0000 7f8a4cff9700 1 -- 172.21.15.60:0/268084330 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a400195e0 con 0x7f8a501080c0 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.697+0000 7f8a55ddf700 1 -- 172.21.15.60:0/268084330 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5019bac0 con 0x7f8a501080c0 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.697+0000 7f8a55ddf700 1 -- 172.21.15.60:0/268084330 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5019bf60 con 0x7f8a501080c0 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.697+0000 7f8a4cff9700 1 -- 172.21.15.60:0/268084330 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f8a400198a0 con 0x7f8a501080c0 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.698+0000 7f8a4cff9700 1 --2- 172.21.15.60:0/268084330 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f8a38038e40 0x7f8a3803b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.698+0000 7f8a4cff9700 1 -- 172.21.15.60:0/268084330 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8a4004d070 con 0x7f8a501080c0 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.698+0000 7f8a4effd700 1 --2- 172.21.15.60:0/268084330 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f8a38038e40 0x7f8a3803b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.698+0000 7f8a55ddf700 1 -- 172.21.15.60:0/268084330 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5010b780 con 0x7f8a501080c0 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.698+0000 7f8a4effd700 1 --2- 172.21.15.60:0/268084330 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f8a38038e40 0x7f8a3803b300 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8a44006fd0 tx=0x7f8a44006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.703+0000 7f8a4cff9700 1 -- 172.21.15.60:0/268084330 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8a4002b340 con 0x7f8a501080c0 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.991+0000 7f8a55ddf700 1 -- 172.21.15.60:0/268084330 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8a5019ea90 con 0x7f8a501080c0 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.992+0000 7f8a4cff9700 1 -- 172.21.15.60:0/268084330 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f8a40023390 con 0x7f8a501080c0 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.994+0000 7f8a55ddf700 1 -- 172.21.15.60:0/268084330 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f8a38038e40 msgr2=0x7f8a3803b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.994+0000 7f8a55ddf700 1 --2- 172.21.15.60:0/268084330 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f8a38038e40 0x7f8a3803b300 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8a44006fd0 tx=0x7f8a44006e40 comp rx=0 tx=0).stop 2024-02-26T04:47:41.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.994+0000 7f8a55ddf700 1 -- 172.21.15.60:0/268084330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a501080c0 msgr2=0x7f8a5019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:41.615 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.994+0000 7f8a55ddf700 1 --2- 172.21.15.60:0/268084330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a501080c0 0x7f8a5019b380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009fb0 tx=0x7f8a40011b20 comp rx=0 tx=0).stop 2024-02-26T04:47:41.615 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.994+0000 7f8a55ddf700 1 -- 172.21.15.60:0/268084330 shutdown_connections 2024-02-26T04:47:41.615 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.994+0000 7f8a55ddf700 1 --2- 172.21.15.60:0/268084330 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f8a38038e40 0x7f8a3803b300 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:41.615 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.994+0000 7f8a55ddf700 1 --2- 172.21.15.60:0/268084330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a501080c0 0x7f8a5019b380 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:41.615 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.994+0000 7f8a55ddf700 1 -- 172.21.15.60:0/268084330 >> 172.21.15.60:0/268084330 conn(0x7f8a50101a70 msgr2=0x7f8a50102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:41.615 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.994+0000 7f8a55ddf700 1 -- 172.21.15.60:0/268084330 shutdown_connections 2024-02-26T04:47:41.615 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:40.994+0000 7f8a55ddf700 1 -- 172.21.15.60:0/268084330 wait complete. 2024-02-26T04:47:41.615 INFO:teuthology.orchestra.run.smithi060.stdout:mgr is available 2024-02-26T04:47:41.978 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:41 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/268084330' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-26T04:47:42.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T04:47:42.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [global] 2024-02-26T04:47:42.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout fsid = f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:47:42.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-26T04:47:42.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-26T04:47:42.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-26T04:47:42.793 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-26T04:47:42.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T04:47:42.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-26T04:47:42.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-26T04:47:42.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-02-26T04:47:42.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [osd] 2024-02-26T04:47:42.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-26T04:47:42.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-26T04:47:42.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-26T04:47:42.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-26T04:47:42.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-26T04:47:42.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.996+0000 7f3285616700 1 Processor -- start 2024-02-26T04:47:42.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.996+0000 7f3285616700 1 -- start start 2024-02-26T04:47:42.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.996+0000 7f3285616700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3280105e90 0x7f32801062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:42.794 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.996+0000 7f3285616700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328007ae30 con 0x7f3280105e90 2024-02-26T04:47:42.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.996+0000 7f327effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3280105e90 0x7f32801062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:42.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.997+0000 7f327effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3280105e90 0x7f32801062b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35074/0 (socket says 172.21.15.60:35074) 2024-02-26T04:47:42.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.997+0000 7f327effd700 1 -- 172.21.15.60:0/263173157 learned_addr learned my addr 172.21.15.60:0/263173157 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:42.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.997+0000 7f327effd700 1 -- 172.21.15.60:0/263173157 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328007af70 con 0x7f3280105e90 2024-02-26T04:47:42.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.997+0000 7f327effd700 1 --2- 172.21.15.60:0/263173157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3280105e90 0x7f32801062b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f32780098a0 tx=0x7f3278009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4d80ff416a787a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:42.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.997+0000 7f327dffb700 1 -- 172.21.15.60:0/263173157 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3278004160 con 0x7f3280105e90 2024-02-26T04:47:42.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.997+0000 7f327dffb700 1 -- 172.21.15.60:0/263173157 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32780042c0 con 0x7f3280105e90 2024-02-26T04:47:42.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.997+0000 7f327dffb700 1 -- 172.21.15.60:0/263173157 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3278011ac0 con 0x7f3280105e90 2024-02-26T04:47:42.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.998+0000 7f3285616700 1 -- 172.21.15.60:0/263173157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3280105e90 msgr2=0x7f32801062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:42.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.998+0000 7f3285616700 1 --2- 172.21.15.60:0/263173157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3280105e90 0x7f32801062b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f32780098a0 tx=0x7f3278009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:42.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.998+0000 7f3285616700 1 -- 172.21.15.60:0/263173157 shutdown_connections 2024-02-26T04:47:42.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.998+0000 7f3285616700 1 --2- 172.21.15.60:0/263173157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3280105e90 0x7f32801062b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:42.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.998+0000 7f3285616700 1 -- 172.21.15.60:0/263173157 >> 172.21.15.60:0/263173157 conn(0x7f3280101a70 msgr2=0x7f3280103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:42.795 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.998+0000 7f3285616700 1 -- 172.21.15.60:0/263173157 shutdown_connections 2024-02-26T04:47:42.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.998+0000 7f3285616700 1 -- 172.21.15.60:0/263173157 wait complete. 2024-02-26T04:47:42.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.998+0000 7f3285616700 1 Processor -- start 2024-02-26T04:47:42.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.999+0000 7f3285616700 1 -- start start 2024-02-26T04:47:42.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.999+0000 7f3285616700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3280105e90 0x7f328019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:42.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.999+0000 7f3285616700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328019dae0 con 0x7f3280105e90 2024-02-26T04:47:42.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.999+0000 7f327effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3280105e90 0x7f328019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:42.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.999+0000 7f327effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3280105e90 0x7f328019d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35082/0 (socket says 172.21.15.60:35082) 2024-02-26T04:47:42.796 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.999+0000 7f327effd700 1 -- 172.21.15.60:0/1911163811 learned_addr learned my addr 172.21.15.60:0/1911163811 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:42.797 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.999+0000 7f327effd700 1 -- 172.21.15.60:0/1911163811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3278009550 con 0x7f3280105e90 2024-02-26T04:47:42.797 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.999+0000 7f327effd700 1 --2- 172.21.15.60:0/1911163811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3280105e90 0x7f328019d5a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3278009fb0 tx=0x7f3278011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:42.797 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.999+0000 7f3277fff700 1 -- 172.21.15.60:0/1911163811 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32780038b0 con 0x7f3280105e90 2024-02-26T04:47:42.797 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.999+0000 7f3277fff700 1 -- 172.21.15.60:0/1911163811 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3278003a10 con 0x7f3280105e90 2024-02-26T04:47:42.797 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.999+0000 7f3285616700 1 -- 172.21.15.60:0/1911163811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328019dce0 con 0x7f3280105e90 2024-02-26T04:47:42.797 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:41.999+0000 7f3277fff700 1 -- 172.21.15.60:0/1911163811 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32780195e0 con 0x7f3280105e90 2024-02-26T04:47:42.797 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.000+0000 7f3285616700 1 -- 172.21.15.60:0/1911163811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328019e180 con 0x7f3280105e90 2024-02-26T04:47:42.797 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.000+0000 7f3277fff700 1 -- 172.21.15.60:0/1911163811 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f32780198c0 con 0x7f3280105e90 2024-02-26T04:47:42.797 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.000+0000 7f3277fff700 1 --2- 172.21.15.60:0/1911163811 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f326c038e40 0x7f326c03b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:42.797 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.000+0000 7f3277fff700 1 -- 172.21.15.60:0/1911163811 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f327804d280 con 0x7f3280105e90 2024-02-26T04:47:42.797 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.000+0000 7f3285616700 1 -- 172.21.15.60:0/1911163811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32801972b0 con 0x7f3280105e90 2024-02-26T04:47:42.797 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.001+0000 7f327e7fc700 1 --2- 172.21.15.60:0/1911163811 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f326c038e40 0x7f326c03b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:42.797 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.001+0000 7f327e7fc700 1 --2- 172.21.15.60:0/1911163811 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f326c038e40 0x7f326c03b300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3268006fd0 tx=0x7f3268006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:42.797 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.006+0000 7f3277fff700 1 -- 172.21.15.60:0/1911163811 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f327802b340 con 0x7f3280105e90 2024-02-26T04:47:42.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.206+0000 7f3285616700 1 -- 172.21.15.60:0/1911163811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f3280107530 con 0x7f3280105e90 2024-02-26T04:47:42.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.207+0000 7f3277fff700 1 -- 172.21.15.60:0/1911163811 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f3278018be0 con 0x7f3280105e90 2024-02-26T04:47:42.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.209+0000 7f3285616700 1 -- 172.21.15.60:0/1911163811 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f326c038e40 msgr2=0x7f326c03b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:42.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.209+0000 7f3285616700 1 --2- 172.21.15.60:0/1911163811 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f326c038e40 0x7f326c03b300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3268006fd0 tx=0x7f3268006e40 comp rx=0 tx=0).stop 2024-02-26T04:47:42.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.209+0000 7f3285616700 1 -- 172.21.15.60:0/1911163811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3280105e90 msgr2=0x7f328019d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:42.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.209+0000 7f3285616700 1 --2- 172.21.15.60:0/1911163811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3280105e90 0x7f328019d5a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3278009fb0 tx=0x7f3278011b20 comp rx=0 tx=0).stop 2024-02-26T04:47:42.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.209+0000 7f3285616700 1 -- 172.21.15.60:0/1911163811 shutdown_connections 2024-02-26T04:47:42.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.209+0000 7f3285616700 1 --2- 172.21.15.60:0/1911163811 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f326c038e40 0x7f326c03b300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:42.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.209+0000 7f3285616700 1 --2- 172.21.15.60:0/1911163811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3280105e90 0x7f328019d5a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:42.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.209+0000 7f3285616700 1 -- 172.21.15.60:0/1911163811 >> 172.21.15.60:0/1911163811 conn(0x7f3280101a70 msgr2=0x7f32801943f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:42.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.209+0000 7f3285616700 1 -- 172.21.15.60:0/1911163811 shutdown_connections 2024-02-26T04:47:42.798 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:42.209+0000 7f3285616700 1 -- 172.21.15.60:0/1911163811 wait complete. 2024-02-26T04:47:42.798 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling cephadm module... 2024-02-26T04:47:42.899 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:42 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1911163811' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-26T04:47:44.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:43 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/409311601' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-26T04:47:44.046 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:43 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: ignoring --setuser ceph since I am not root 2024-02-26T04:47:44.046 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:43 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: ignoring --setgroup ceph since I am not root 2024-02-26T04:47:44.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.189+0000 7fc6b1740700 1 Processor -- start 2024-02-26T04:47:44.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.189+0000 7fc6b1740700 1 -- start start 2024-02-26T04:47:44.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.189+0000 7fc6b1740700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac1058e0 0x7fc6ac105d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:44.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.189+0000 7fc6b1740700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac1062d0 con 0x7fc6ac1058e0 2024-02-26T04:47:44.346 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.189+0000 7fc6aaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac1058e0 0x7fc6ac105d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:44.347 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.189+0000 7fc6aaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac1058e0 0x7fc6ac105d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35088/0 (socket says 172.21.15.60:35088) 2024-02-26T04:47:44.347 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.189+0000 7fc6aaffd700 1 -- 172.21.15.60:0/710352512 learned_addr learned my addr 172.21.15.60:0/710352512 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:44.347 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.189+0000 7fc6aaffd700 1 -- 172.21.15.60:0/710352512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac108870 con 0x7fc6ac1058e0 2024-02-26T04:47:44.347 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.190+0000 7fc6aaffd700 1 --2- 172.21.15.60:0/710352512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac1058e0 0x7fc6ac105d00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0098a0 tx=0x7fc69c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779acb65c8d5c06a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:44.347 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.190+0000 7fc6a9ffb700 1 -- 172.21.15.60:0/710352512 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc69c004160 con 0x7fc6ac1058e0 2024-02-26T04:47:44.347 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.190+0000 7fc6a9ffb700 1 -- 172.21.15.60:0/710352512 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc69c0042c0 con 0x7fc6ac1058e0 2024-02-26T04:47:44.347 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.190+0000 7fc6a9ffb700 1 -- 172.21.15.60:0/710352512 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc69c011ac0 con 0x7fc6ac1058e0 2024-02-26T04:47:44.348 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.190+0000 7fc6b1740700 1 -- 172.21.15.60:0/710352512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac1058e0 msgr2=0x7fc6ac105d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:44.348 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.190+0000 7fc6b1740700 1 --2- 172.21.15.60:0/710352512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac1058e0 0x7fc6ac105d00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0098a0 tx=0x7fc69c009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:44.348 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.190+0000 7fc6b1740700 1 -- 172.21.15.60:0/710352512 shutdown_connections 2024-02-26T04:47:44.348 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.190+0000 7fc6b1740700 1 --2- 172.21.15.60:0/710352512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac1058e0 0x7fc6ac105d00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:44.348 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.190+0000 7fc6b1740700 1 -- 172.21.15.60:0/710352512 >> 172.21.15.60:0/710352512 conn(0x7fc6ac101250 msgr2=0x7fc6ac103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:44.348 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.190+0000 7fc6b1740700 1 -- 172.21.15.60:0/710352512 shutdown_connections 2024-02-26T04:47:44.348 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.190+0000 7fc6b1740700 1 -- 172.21.15.60:0/710352512 wait complete. 2024-02-26T04:47:44.348 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.191+0000 7fc6b1740700 1 Processor -- start 2024-02-26T04:47:44.348 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.191+0000 7fc6b1740700 1 -- start start 2024-02-26T04:47:44.348 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.191+0000 7fc6b1740700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac1058e0 0x7fc6ac19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:44.348 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.191+0000 7fc6b1740700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac19b960 con 0x7fc6ac1058e0 2024-02-26T04:47:44.348 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.192+0000 7fc6aaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac1058e0 0x7fc6ac19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:44.348 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.192+0000 7fc6aaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac1058e0 0x7fc6ac19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35100/0 (socket says 172.21.15.60:35100) 2024-02-26T04:47:44.349 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.192+0000 7fc6aaffd700 1 -- 172.21.15.60:0/409311601 learned_addr learned my addr 172.21.15.60:0/409311601 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:44.349 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.192+0000 7fc6aaffd700 1 -- 172.21.15.60:0/409311601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c009550 con 0x7fc6ac1058e0 2024-02-26T04:47:44.349 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.192+0000 7fc6aaffd700 1 --2- 172.21.15.60:0/409311601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac1058e0 0x7fc6ac19b420 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc69c011780 tx=0x7fc69c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:44.349 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.192+0000 7fc693fff700 1 -- 172.21.15.60:0/409311601 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc69c003af0 con 0x7fc6ac1058e0 2024-02-26T04:47:44.349 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.192+0000 7fc6b1740700 1 -- 172.21.15.60:0/409311601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ac19bb60 con 0x7fc6ac1058e0 2024-02-26T04:47:44.349 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.192+0000 7fc693fff700 1 -- 172.21.15.60:0/409311601 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc69c003c50 con 0x7fc6ac1058e0 2024-02-26T04:47:44.349 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.192+0000 7fc693fff700 1 -- 172.21.15.60:0/409311601 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc69c0197e0 con 0x7fc6ac1058e0 2024-02-26T04:47:44.349 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.192+0000 7fc6b1740700 1 -- 172.21.15.60:0/409311601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ac19c000 con 0x7fc6ac1058e0 2024-02-26T04:47:44.349 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.193+0000 7fc693fff700 1 -- 172.21.15.60:0/409311601 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7fc69c019a20 con 0x7fc6ac1058e0 2024-02-26T04:47:44.349 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.193+0000 7fc6b1740700 1 -- 172.21.15.60:0/409311601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ac10baa0 con 0x7fc6ac1058e0 2024-02-26T04:47:44.349 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.193+0000 7fc693fff700 1 --2- 172.21.15.60:0/409311601 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fc694038e90 0x7fc69403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:44.349 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.193+0000 7fc693fff700 1 -- 172.21.15.60:0/409311601 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc69c04d400 con 0x7fc6ac1058e0 2024-02-26T04:47:44.349 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.193+0000 7fc6aa7fc700 1 --2- 172.21.15.60:0/409311601 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fc694038e90 0x7fc69403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:44.350 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.194+0000 7fc6aa7fc700 1 --2- 172.21.15.60:0/409311601 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fc694038e90 0x7fc69403b350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0006fd0 tx=0x7fc6a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:44.350 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.198+0000 7fc693fff700 1 -- 172.21.15.60:0/409311601 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc69c018c20 con 0x7fc6ac1058e0 2024-02-26T04:47:44.350 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.458+0000 7fc6b1740700 1 -- 172.21.15.60:0/409311601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fc6ac19e940 con 0x7fc6ac1058e0 2024-02-26T04:47:44.350 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.789+0000 7fc693fff700 1 -- 172.21.15.60:0/409311601 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fc69c018350 con 0x7fc6ac1058e0 2024-02-26T04:47:44.350 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.790+0000 7fc693fff700 1 -- 172.21.15.60:0/409311601 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fc69c019cd0 con 0x7fc6ac1058e0 2024-02-26T04:47:44.350 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.791+0000 7fc6b1740700 1 -- 172.21.15.60:0/409311601 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fc694038e90 msgr2=0x7fc69403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:44.350 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.791+0000 7fc6b1740700 1 --2- 172.21.15.60:0/409311601 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fc694038e90 0x7fc69403b350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0006fd0 tx=0x7fc6a0006e40 comp rx=0 tx=0).stop 2024-02-26T04:47:44.350 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.791+0000 7fc6b1740700 1 -- 172.21.15.60:0/409311601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac1058e0 msgr2=0x7fc6ac19b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:44.350 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.791+0000 7fc6b1740700 1 --2- 172.21.15.60:0/409311601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac1058e0 0x7fc6ac19b420 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc69c011780 tx=0x7fc69c011b40 comp rx=0 tx=0).stop 2024-02-26T04:47:44.350 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.792+0000 7fc6b1740700 1 -- 172.21.15.60:0/409311601 shutdown_connections 2024-02-26T04:47:44.350 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.792+0000 7fc6b1740700 1 --2- 172.21.15.60:0/409311601 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fc694038e90 0x7fc69403b350 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:44.350 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.792+0000 7fc6b1740700 1 --2- 172.21.15.60:0/409311601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac1058e0 0x7fc6ac19b420 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:44.350 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.792+0000 7fc6b1740700 1 -- 172.21.15.60:0/409311601 >> 172.21.15.60:0/409311601 conn(0x7fc6ac101250 msgr2=0x7fc6ac101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:44.351 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.792+0000 7fc6b1740700 1 -- 172.21.15.60:0/409311601 shutdown_connections 2024-02-26T04:47:44.351 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:43.792+0000 7fc6b1740700 1 -- 172.21.15.60:0/409311601 wait complete. 2024-02-26T04:47:44.669 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:44 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:44.361+0000 7f337eff1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T04:47:44.669 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:44 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:44.465+0000 7f337eff1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T04:47:44.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:44 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/409311601' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-26T04:47:44.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:44 smithi060 ceph-mon[32382]: mgrmap e5: x(active, since 6s) 2024-02-26T04:47:45.229 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:45 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:45.138+0000 7f337eff1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T04:47:45.568 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:45 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:45.233+0000 7f337eff1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T04:47:45.568 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:45 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:45.380+0000 7f337eff1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T04:47:45.568 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:45 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:45.478+0000 7f337eff1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T04:47:45.569 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:45 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:45.566+0000 7f337eff1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T04:47:45.629 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T04:47:45.629 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-26T04:47:45.629 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-26T04:47:45.629 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-26T04:47:45.630 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-26T04:47:45.630 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T04:47:45.630 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.763+0000 7f8492b34700 1 Processor -- start 2024-02-26T04:47:45.630 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.763+0000 7f8492b34700 1 -- start start 2024-02-26T04:47:45.630 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.764+0000 7f8492b34700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f848c108010 0x7f848c10a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:45.630 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.764+0000 7f8492b34700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c07ae90 con 0x7f848c108010 2024-02-26T04:47:45.630 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.764+0000 7f84908d0700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f848c108010 0x7f848c10a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:45.630 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.764+0000 7f84908d0700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f848c108010 0x7f848c10a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35112/0 (socket says 172.21.15.60:35112) 2024-02-26T04:47:45.630 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.764+0000 7f84908d0700 1 -- 172.21.15.60:0/1334365292 learned_addr learned my addr 172.21.15.60:0/1334365292 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:45.630 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.764+0000 7f84908d0700 1 -- 172.21.15.60:0/1334365292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c10a940 con 0x7f848c108010 2024-02-26T04:47:45.630 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.764+0000 7f84908d0700 1 --2- 172.21.15.60:0/1334365292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f848c108010 0x7f848c10a400 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84800098a0 tx=0x7f8480009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca8bf49a2aa8871 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:45.630 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.764+0000 7f848b7fe700 1 -- 172.21.15.60:0/1334365292 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8480004160 con 0x7f848c108010 2024-02-26T04:47:45.630 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.764+0000 7f848b7fe700 1 -- 172.21.15.60:0/1334365292 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84800042c0 con 0x7f848c108010 2024-02-26T04:47:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.764+0000 7f848b7fe700 1 -- 172.21.15.60:0/1334365292 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8480011ac0 con 0x7f848c108010 2024-02-26T04:47:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.765+0000 7f8492b34700 1 -- 172.21.15.60:0/1334365292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f848c108010 msgr2=0x7f848c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.765+0000 7f8492b34700 1 --2- 172.21.15.60:0/1334365292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f848c108010 0x7f848c10a400 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84800098a0 tx=0x7f8480009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.765+0000 7f8492b34700 1 -- 172.21.15.60:0/1334365292 shutdown_connections 2024-02-26T04:47:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.765+0000 7f8492b34700 1 --2- 172.21.15.60:0/1334365292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f848c108010 0x7f848c10a400 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.765+0000 7f8492b34700 1 -- 172.21.15.60:0/1334365292 >> 172.21.15.60:0/1334365292 conn(0x7f848c101a00 msgr2=0x7f848c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.765+0000 7f8492b34700 1 -- 172.21.15.60:0/1334365292 shutdown_connections 2024-02-26T04:47:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.765+0000 7f8492b34700 1 -- 172.21.15.60:0/1334365292 wait complete. 2024-02-26T04:47:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.766+0000 7f8492b34700 1 Processor -- start 2024-02-26T04:47:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.766+0000 7f8492b34700 1 -- start start 2024-02-26T04:47:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.766+0000 7f8492b34700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f848c108010 0x7f848c107890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.766+0000 7f8492b34700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c107dd0 con 0x7f848c108010 2024-02-26T04:47:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.766+0000 7f84908d0700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f848c108010 0x7f848c107890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.766+0000 7f84908d0700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f848c108010 0x7f848c107890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35118/0 (socket says 172.21.15.60:35118) 2024-02-26T04:47:45.632 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.766+0000 7f84908d0700 1 -- 172.21.15.60:0/3408231993 learned_addr learned my addr 172.21.15.60:0/3408231993 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:45.632 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.766+0000 7f84908d0700 1 -- 172.21.15.60:0/3408231993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8480009550 con 0x7f848c108010 2024-02-26T04:47:45.632 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.766+0000 7f84908d0700 1 --2- 172.21.15.60:0/3408231993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f848c108010 0x7f848c107890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8480003f70 tx=0x7f8480011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:45.632 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.767+0000 7f8489ffb700 1 -- 172.21.15.60:0/3408231993 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84800039f0 con 0x7f848c108010 2024-02-26T04:47:45.632 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.767+0000 7f8489ffb700 1 -- 172.21.15.60:0/3408231993 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8480003b50 con 0x7f848c108010 2024-02-26T04:47:45.632 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.767+0000 7f8492b34700 1 -- 172.21.15.60:0/3408231993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848c105fc0 con 0x7f848c108010 2024-02-26T04:47:45.632 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.767+0000 7f8489ffb700 1 -- 172.21.15.60:0/3408231993 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f848001a760 con 0x7f848c108010 2024-02-26T04:47:45.632 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.767+0000 7f8492b34700 1 -- 172.21.15.60:0/3408231993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848c106460 con 0x7f848c108010 2024-02-26T04:47:45.632 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.768+0000 7f8489ffb700 1 -- 172.21.15.60:0/3408231993 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f848001a900 con 0x7f848c108010 2024-02-26T04:47:45.632 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.768+0000 7f8492b34700 1 -- 172.21.15.60:0/3408231993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848c10b740 con 0x7f848c108010 2024-02-26T04:47:45.632 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.768+0000 7f8489ffb700 1 --2- 172.21.15.60:0/3408231993 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f847c038ee0 0x7f847c03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:45.632 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.768+0000 7f8489ffb700 1 -- 172.21.15.60:0/3408231993 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f848004d060 con 0x7f848c108010 2024-02-26T04:47:45.632 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.768+0000 7f848bfff700 1 -- 172.21.15.60:0/3408231993 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f847c038ee0 msgr2=0x7f847c03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/114766175 2024-02-26T04:47:45.633 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.768+0000 7f848bfff700 1 --2- 172.21.15.60:0/3408231993 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f847c038ee0 0x7f847c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T04:47:45.633 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.773+0000 7f8489ffb700 1 -- 172.21.15.60:0/3408231993 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f848001abb0 con 0x7f848c108010 2024-02-26T04:47:45.633 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.969+0000 7f848bfff700 1 -- 172.21.15.60:0/3408231993 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f847c038ee0 msgr2=0x7f847c03b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/114766175 2024-02-26T04:47:45.633 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:44.969+0000 7f848bfff700 1 --2- 172.21.15.60:0/3408231993 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f847c038ee0 0x7f847c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T04:47:45.633 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:45.033+0000 7f8492b34700 1 -- 172.21.15.60:0/3408231993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f848c106b50 con 0x7f848c108010 2024-02-26T04:47:45.633 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:45.033+0000 7f8489ffb700 1 -- 172.21.15.60:0/3408231993 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f8480018be0 con 0x7f848c108010 2024-02-26T04:47:45.633 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:45.035+0000 7f8492b34700 1 -- 172.21.15.60:0/3408231993 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f847c038ee0 msgr2=0x7f847c03b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T04:47:45.633 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:45.035+0000 7f8492b34700 1 --2- 172.21.15.60:0/3408231993 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f847c038ee0 0x7f847c03b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:45.633 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:45.035+0000 7f8492b34700 1 -- 172.21.15.60:0/3408231993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f848c108010 msgr2=0x7f848c107890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:45.634 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:45.035+0000 7f8492b34700 1 --2- 172.21.15.60:0/3408231993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f848c108010 0x7f848c107890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8480003f70 tx=0x7f8480011b40 comp rx=0 tx=0).stop 2024-02-26T04:47:45.634 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:45.035+0000 7f8492b34700 1 -- 172.21.15.60:0/3408231993 shutdown_connections 2024-02-26T04:47:45.634 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:45.035+0000 7f8492b34700 1 --2- 172.21.15.60:0/3408231993 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7f847c038ee0 0x7f847c03b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:45.634 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:45.035+0000 7f8492b34700 1 --2- 172.21.15.60:0/3408231993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f848c108010 0x7f848c107890 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:45.634 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:45.035+0000 7f8492b34700 1 -- 172.21.15.60:0/3408231993 >> 172.21.15.60:0/3408231993 conn(0x7f848c101a00 msgr2=0x7f848c1025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:45.634 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:45.036+0000 7f8492b34700 1 -- 172.21.15.60:0/3408231993 shutdown_connections 2024-02-26T04:47:45.634 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:45.036+0000 7f8492b34700 1 -- 172.21.15.60:0/3408231993 wait complete. 2024-02-26T04:47:45.634 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for the mgr to restart... 2024-02-26T04:47:45.634 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr epoch 5... 2024-02-26T04:47:45.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:45 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3408231993' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-26T04:47:46.149 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:45 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:45.846+0000 7f337eff1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T04:47:46.149 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:45 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:45.959+0000 7f337eff1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T04:47:46.149 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:46 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:46.050+0000 7f337eff1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T04:47:46.480 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:46 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:46.147+0000 7f337eff1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T04:47:46.480 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:46 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:46.325+0000 7f337eff1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T04:47:47.729 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:47 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:47.307+0000 7f337eff1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T04:47:47.729 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:47 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:47.441+0000 7f337eff1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T04:47:48.075 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:47 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:47.779+0000 7f337eff1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T04:47:48.075 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:47 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:47.871+0000 7f337eff1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T04:47:48.075 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:48 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:48.072+0000 7f337eff1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T04:47:48.479 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:48 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:48.261+0000 7f337eff1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T04:47:48.817 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:48 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:48.553+0000 7f337eff1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T04:47:48.818 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:48 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:48.815+0000 7f337eff1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T04:47:49.229 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:49 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:49.058+0000 7f337eff1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T04:47:50.229 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:49 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:49.882+0000 7f337eff1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T04:47:50.729 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:50.464+0000 7f337eff1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T04:47:50.729 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:50.565+0000 7f337eff1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T04:47:51.229 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:47:50.786+0000 7f337eff1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T04:47:51.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: Active manager daemon x restarted 2024-02-26T04:47:51.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: Activating manager daemon x 2024-02-26T04:47:51.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: osdmap e2: 0 total, 0 up, 0 in 2024-02-26T04:47:51.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: mgrmap e6: x(active, starting, since 0.00332704s) 2024-02-26T04:47:51.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:47:51.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T04:47:51.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T04:47:51.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T04:47:51.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T04:47:51.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: Manager daemon x is now available 2024-02-26T04:47:51.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: Found migration_current of "None". Setting to last migration. 2024-02-26T04:47:51.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' 2024-02-26T04:47:51.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' 2024-02-26T04:47:51.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:47:51.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:50 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:47:52.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:51 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:47:52.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:51 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T04:47:52.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:51 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T04:47:52.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:51 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' 2024-02-26T04:47:52.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:51 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' 2024-02-26T04:47:52.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:51 smithi060 ceph-mon[32382]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-26T04:47:52.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:51 smithi060 ceph-mon[32382]: mgrmap e7: x(active, since 1.00547s) 2024-02-26T04:47:52.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:51 smithi060 ceph-mon[32382]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-26T04:47:52.413 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T04:47:52.413 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-26T04:47:52.413 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-26T04:47:52.413 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T04:47:52.413 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.004+0000 7fb58ad2b700 1 Processor -- start 2024-02-26T04:47:52.413 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.004+0000 7fb58ad2b700 1 -- start start 2024-02-26T04:47:52.414 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.005+0000 7fb58ad2b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb584108080 0x7fb58410a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:52.414 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.005+0000 7fb58ad2b700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58407ae10 con 0x7fb584108080 2024-02-26T04:47:52.414 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.005+0000 7fb588ac7700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb584108080 0x7fb58410a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:52.414 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.005+0000 7fb588ac7700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb584108080 0x7fb58410a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35130/0 (socket says 172.21.15.60:35130) 2024-02-26T04:47:52.414 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.005+0000 7fb588ac7700 1 -- 172.21.15.60:0/4136539864 learned_addr learned my addr 172.21.15.60:0/4136539864 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:52.414 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.005+0000 7fb588ac7700 1 -- 172.21.15.60:0/4136539864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58407af50 con 0x7fb584108080 2024-02-26T04:47:52.414 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.005+0000 7fb588ac7700 1 --2- 172.21.15.60:0/4136539864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb584108080 0x7fb58410a470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb5780098a0 tx=0x7fb578009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7095e3a9bed588df server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:52.414 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.005+0000 7fb5837fe700 1 -- 172.21.15.60:0/4136539864 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb578004160 con 0x7fb584108080 2024-02-26T04:47:52.414 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.005+0000 7fb5837fe700 1 -- 172.21.15.60:0/4136539864 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5780042c0 con 0x7fb584108080 2024-02-26T04:47:52.414 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.005+0000 7fb5837fe700 1 -- 172.21.15.60:0/4136539864 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb578011ac0 con 0x7fb584108080 2024-02-26T04:47:52.414 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.006+0000 7fb58ad2b700 1 -- 172.21.15.60:0/4136539864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb584108080 msgr2=0x7fb58410a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:52.414 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.006+0000 7fb58ad2b700 1 --2- 172.21.15.60:0/4136539864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb584108080 0x7fb58410a470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb5780098a0 tx=0x7fb578009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:52.414 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.006+0000 7fb58ad2b700 1 -- 172.21.15.60:0/4136539864 shutdown_connections 2024-02-26T04:47:52.415 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.006+0000 7fb58ad2b700 1 --2- 172.21.15.60:0/4136539864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb584108080 0x7fb58410a470 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:52.415 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.006+0000 7fb58ad2b700 1 -- 172.21.15.60:0/4136539864 >> 172.21.15.60:0/4136539864 conn(0x7fb584101a10 msgr2=0x7fb584103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:52.415 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.006+0000 7fb58ad2b700 1 -- 172.21.15.60:0/4136539864 shutdown_connections 2024-02-26T04:47:52.415 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.006+0000 7fb58ad2b700 1 -- 172.21.15.60:0/4136539864 wait complete. 2024-02-26T04:47:52.415 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.007+0000 7fb58ad2b700 1 Processor -- start 2024-02-26T04:47:52.415 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.007+0000 7fb58ad2b700 1 -- start start 2024-02-26T04:47:52.415 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.007+0000 7fb58ad2b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb584108080 0x7fb584107900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:52.415 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.007+0000 7fb58ad2b700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb584107e40 con 0x7fb584108080 2024-02-26T04:47:52.415 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.007+0000 7fb588ac7700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb584108080 0x7fb584107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:52.415 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.007+0000 7fb588ac7700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb584108080 0x7fb584107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35146/0 (socket says 172.21.15.60:35146) 2024-02-26T04:47:52.415 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.007+0000 7fb588ac7700 1 -- 172.21.15.60:0/856116666 learned_addr learned my addr 172.21.15.60:0/856116666 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:52.415 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.007+0000 7fb588ac7700 1 -- 172.21.15.60:0/856116666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb578009550 con 0x7fb584108080 2024-02-26T04:47:52.416 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.007+0000 7fb588ac7700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb584108080 0x7fb584107900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb578011a70 tx=0x7fb578011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:52.416 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.008+0000 7fb581ffb700 1 -- 172.21.15.60:0/856116666 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5780039c0 con 0x7fb584108080 2024-02-26T04:47:52.416 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.008+0000 7fb581ffb700 1 -- 172.21.15.60:0/856116666 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb578003b20 con 0x7fb584108080 2024-02-26T04:47:52.416 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.008+0000 7fb58ad2b700 1 -- 172.21.15.60:0/856116666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb584106030 con 0x7fb584108080 2024-02-26T04:47:52.416 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.008+0000 7fb581ffb700 1 -- 172.21.15.60:0/856116666 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb578024610 con 0x7fb584108080 2024-02-26T04:47:52.416 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.008+0000 7fb58ad2b700 1 -- 172.21.15.60:0/856116666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5841064d0 con 0x7fb584108080 2024-02-26T04:47:52.416 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.009+0000 7fb581ffb700 1 -- 172.21.15.60:0/856116666 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fb57802d3f0 con 0x7fb584108080 2024-02-26T04:47:52.416 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.009+0000 7fb581ffb700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fb574038f30 0x7fb57403b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:52.416 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.009+0000 7fb581ffb700 1 -- 172.21.15.60:0/856116666 --> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb57403bb00 con 0x7fb574038f30 2024-02-26T04:47:52.417 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.009+0000 7fb581ffb700 1 -- 172.21.15.60:0/856116666 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb57804ced0 con 0x7fb584108080 2024-02-26T04:47:52.417 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.009+0000 7fb583fff700 1 -- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fb574038f30 msgr2=0x7fb57403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/114766175 2024-02-26T04:47:52.417 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.009+0000 7fb583fff700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fb574038f30 0x7fb57403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T04:47:52.417 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.210+0000 7fb583fff700 1 -- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fb574038f30 msgr2=0x7fb57403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/114766175 2024-02-26T04:47:52.417 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.210+0000 7fb583fff700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fb574038f30 0x7fb57403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T04:47:52.417 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.611+0000 7fb583fff700 1 -- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fb574038f30 msgr2=0x7fb57403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/114766175 2024-02-26T04:47:52.417 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:46.611+0000 7fb583fff700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fb574038f30 0x7fb57403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-26T04:47:52.417 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:47.412+0000 7fb583fff700 1 -- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fb574038f30 msgr2=0x7fb57403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/114766175 2024-02-26T04:47:52.417 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:47.412+0000 7fb583fff700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fb574038f30 0x7fb57403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-26T04:47:52.417 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:49.013+0000 7fb583fff700 1 -- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fb574038f30 msgr2=0x7fb57403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/114766175 2024-02-26T04:47:52.417 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:49.013+0000 7fb583fff700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fb574038f30 0x7fb57403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-26T04:47:52.417 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:50.801+0000 7fb581ffb700 1 -- 172.21.15.60:0/856116666 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mgrmap(e 6) v1 ==== 45699+0+0 (secure 0 0 0) 0x7fb578021d30 con 0x7fb584108080 2024-02-26T04:47:52.417 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:50.801+0000 7fb581ffb700 1 -- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fb574038f30 msgr2=0x7fb57403b3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T04:47:52.418 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:50.801+0000 7fb581ffb700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fb574038f30 0x7fb57403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:52.418 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.802+0000 7fb581ffb700 1 -- 172.21.15.60:0/856116666 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 7) v1 ==== 45826+0+0 (secure 0 0 0) 0x7fb57804e850 con 0x7fb584108080 2024-02-26T04:47:52.418 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.802+0000 7fb581ffb700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fb57403c820 0x7fb57403ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:52.418 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.802+0000 7fb581ffb700 1 -- 172.21.15.60:0/856116666 --> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb57403bb00 con 0x7fb57403c820 2024-02-26T04:47:52.418 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.803+0000 7fb583fff700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fb57403c820 0x7fb57403ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:52.418 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.803+0000 7fb583fff700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fb57403c820 0x7fb57403ec10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb570003a10 tx=0x7fb5700092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:52.418 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.804+0000 7fb581ffb700 1 -- 172.21.15.60:0/856116666 <== mgr.14120 v2:172.21.15.60:6800/3694881479 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fb57403bb00 con 0x7fb57403c820 2024-02-26T04:47:52.418 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.810+0000 7fb58ad2b700 1 -- 172.21.15.60:0/856116666 --> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fb584068d30 con 0x7fb57403c820 2024-02-26T04:47:52.418 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.810+0000 7fb581ffb700 1 -- 172.21.15.60:0/856116666 <== mgr.14120 v2:172.21.15.60:6800/3694881479 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fb584068d30 con 0x7fb57403c820 2024-02-26T04:47:52.418 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.811+0000 7fb58ad2b700 1 -- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fb57403c820 msgr2=0x7fb57403ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:52.418 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.811+0000 7fb58ad2b700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fb57403c820 0x7fb57403ec10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb570003a10 tx=0x7fb5700092b0 comp rx=0 tx=0).stop 2024-02-26T04:47:52.418 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.811+0000 7fb58ad2b700 1 -- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb584108080 msgr2=0x7fb584107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:52.418 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.811+0000 7fb58ad2b700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb584108080 0x7fb584107900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb578011a70 tx=0x7fb578011b50 comp rx=0 tx=0).stop 2024-02-26T04:47:52.419 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.811+0000 7fb58ad2b700 1 -- 172.21.15.60:0/856116666 shutdown_connections 2024-02-26T04:47:52.419 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.811+0000 7fb58ad2b700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fb57403c820 0x7fb57403ec10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:52.419 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.811+0000 7fb58ad2b700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb584108080 0x7fb584107900 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:52.419 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.811+0000 7fb58ad2b700 1 --2- 172.21.15.60:0/856116666 >> [v2:172.21.15.60:6800/114766175,v1:172.21.15.60:6801/114766175] conn(0x7fb574038f30 0x7fb57403b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:52.419 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.811+0000 7fb58ad2b700 1 -- 172.21.15.60:0/856116666 >> 172.21.15.60:0/856116666 conn(0x7fb584101a10 msgr2=0x7fb5841026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:52.419 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.811+0000 7fb58ad2b700 1 -- 172.21.15.60:0/856116666 shutdown_connections 2024-02-26T04:47:52.419 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:51.811+0000 7fb58ad2b700 1 -- 172.21.15.60:0/856116666 wait complete. 2024-02-26T04:47:52.419 INFO:teuthology.orchestra.run.smithi060.stdout:mgr epoch 5 is available 2024-02-26T04:47:52.419 INFO:teuthology.orchestra.run.smithi060.stdout:Setting orchestrator backend to cephadm... 2024-02-26T04:47:53.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:52 smithi060 ceph-mon[32382]: [26/Feb/2024:04:47:52] ENGINE Bus STARTING 2024-02-26T04:47:53.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:52 smithi060 ceph-mon[32382]: [26/Feb/2024:04:47:52] ENGINE Serving on http://172.21.15.60:8765 2024-02-26T04:47:53.090 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:52 smithi060 ceph-mon[32382]: [26/Feb/2024:04:47:52] ENGINE Serving on https://172.21.15.60:7150 2024-02-26T04:47:53.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:52 smithi060 ceph-mon[32382]: [26/Feb/2024:04:47:52] ENGINE Bus STARTED 2024-02-26T04:47:53.091 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:52 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:47:53.649 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.855+0000 7f744d14f700 1 Processor -- start 2024-02-26T04:47:53.650 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.855+0000 7f744d14f700 1 -- start start 2024-02-26T04:47:53.650 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.855+0000 7f744d14f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74481075c0 0x7f74481079e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:53.650 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.855+0000 7f744d14f700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7448107fb0 con 0x7f74481075c0 2024-02-26T04:47:53.650 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.855+0000 7f7446d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74481075c0 0x7f74481079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:53.650 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.855+0000 7f7446d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74481075c0 0x7f74481079e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46198/0 (socket says 172.21.15.60:46198) 2024-02-26T04:47:53.650 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.855+0000 7f7446d9d700 1 -- 172.21.15.60:0/1858849920 learned_addr learned my addr 172.21.15.60:0/1858849920 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:53.650 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.855+0000 7f7446d9d700 1 -- 172.21.15.60:0/1858849920 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7448108770 con 0x7f74481075c0 2024-02-26T04:47:53.650 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.856+0000 7f7446d9d700 1 --2- 172.21.15.60:0/1858849920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74481075c0 0x7f74481079e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f74380098a0 tx=0x7f7438009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87a6778f658a4fd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:53.650 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.857+0000 7f7445d9b700 1 -- 172.21.15.60:0/1858849920 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7438004160 con 0x7f74481075c0 2024-02-26T04:47:53.650 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.857+0000 7f7445d9b700 1 -- 172.21.15.60:0/1858849920 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74380042c0 con 0x7f74481075c0 2024-02-26T04:47:53.650 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.857+0000 7f7445d9b700 1 -- 172.21.15.60:0/1858849920 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7438011ac0 con 0x7f74481075c0 2024-02-26T04:47:53.650 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.857+0000 7f744d14f700 1 -- 172.21.15.60:0/1858849920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74481075c0 msgr2=0x7f74481079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:53.651 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.857+0000 7f744d14f700 1 --2- 172.21.15.60:0/1858849920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74481075c0 0x7f74481079e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f74380098a0 tx=0x7f7438009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:53.651 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.857+0000 7f744d14f700 1 -- 172.21.15.60:0/1858849920 shutdown_connections 2024-02-26T04:47:53.651 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.857+0000 7f744d14f700 1 --2- 172.21.15.60:0/1858849920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74481075c0 0x7f74481079e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:53.651 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.857+0000 7f744d14f700 1 -- 172.21.15.60:0/1858849920 >> 172.21.15.60:0/1858849920 conn(0x7f7448102be0 msgr2=0x7f7448105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:53.651 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.857+0000 7f744d14f700 1 -- 172.21.15.60:0/1858849920 shutdown_connections 2024-02-26T04:47:53.651 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.857+0000 7f744d14f700 1 -- 172.21.15.60:0/1858849920 wait complete. 2024-02-26T04:47:53.651 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.858+0000 7f744d14f700 1 Processor -- start 2024-02-26T04:47:53.651 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.858+0000 7f744d14f700 1 -- start start 2024-02-26T04:47:53.651 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.858+0000 7f744d14f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74481075c0 0x7f744819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:53.651 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.858+0000 7f744d14f700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744819b8d0 con 0x7f74481075c0 2024-02-26T04:47:53.651 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.859+0000 7f7446d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74481075c0 0x7f744819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:53.651 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.859+0000 7f7446d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74481075c0 0x7f744819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46202/0 (socket says 172.21.15.60:46202) 2024-02-26T04:47:53.651 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.859+0000 7f7446d9d700 1 -- 172.21.15.60:0/815468064 learned_addr learned my addr 172.21.15.60:0/815468064 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:53.651 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.859+0000 7f7446d9d700 1 -- 172.21.15.60:0/815468064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7438009550 con 0x7f74481075c0 2024-02-26T04:47:53.652 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.859+0000 7f7446d9d700 1 --2- 172.21.15.60:0/815468064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74481075c0 0x7f744819b390 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7438009fb0 tx=0x7f7438011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:53.652 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.859+0000 7f742ffff700 1 -- 172.21.15.60:0/815468064 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74380038b0 con 0x7f74481075c0 2024-02-26T04:47:53.652 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.859+0000 7f742ffff700 1 -- 172.21.15.60:0/815468064 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7438003a10 con 0x7f74481075c0 2024-02-26T04:47:53.652 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.859+0000 7f744d14f700 1 -- 172.21.15.60:0/815468064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744819bad0 con 0x7f74481075c0 2024-02-26T04:47:53.652 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.859+0000 7f742ffff700 1 -- 172.21.15.60:0/815468064 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74380195e0 con 0x7f74481075c0 2024-02-26T04:47:53.652 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.859+0000 7f744d14f700 1 -- 172.21.15.60:0/815468064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744819bf70 con 0x7f74481075c0 2024-02-26T04:47:53.652 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.860+0000 7f742ffff700 1 -- 172.21.15.60:0/815468064 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f7438019850 con 0x7f74481075c0 2024-02-26T04:47:53.652 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.860+0000 7f742ffff700 1 --2- 172.21.15.60:0/815468064 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f7430038ee0 0x7f743003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:53.652 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.860+0000 7f744d14f700 1 -- 172.21.15.60:0/815468064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744810b9a0 con 0x7f74481075c0 2024-02-26T04:47:53.652 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.860+0000 7f742ffff700 1 -- 172.21.15.60:0/815468064 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f743804d170 con 0x7f74481075c0 2024-02-26T04:47:53.652 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.860+0000 7f744659c700 1 --2- 172.21.15.60:0/815468064 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f7430038ee0 0x7f743003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:53.652 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.861+0000 7f744659c700 1 --2- 172.21.15.60:0/815468064 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f7430038ee0 0x7f743003b3a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f743c006fd0 tx=0x7f743c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:53.652 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:52.865+0000 7f742ffff700 1 -- 172.21.15.60:0/815468064 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f743802b340 con 0x7f74481075c0 2024-02-26T04:47:53.652 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:53.088+0000 7f744d14f700 1 -- 172.21.15.60:0/815468064 --> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f7448103e60 con 0x7f7430038ee0 2024-02-26T04:47:53.653 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:53.092+0000 7f742ffff700 1 -- 172.21.15.60:0/815468064 <== mgr.14120 v2:172.21.15.60:6800/3694881479 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7448103e60 con 0x7f7430038ee0 2024-02-26T04:47:53.653 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:53.094+0000 7f744d14f700 1 -- 172.21.15.60:0/815468064 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f7430038ee0 msgr2=0x7f743003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:53.653 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:53.094+0000 7f744d14f700 1 --2- 172.21.15.60:0/815468064 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f7430038ee0 0x7f743003b3a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f743c006fd0 tx=0x7f743c006e40 comp rx=0 tx=0).stop 2024-02-26T04:47:53.653 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:53.094+0000 7f744d14f700 1 -- 172.21.15.60:0/815468064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74481075c0 msgr2=0x7f744819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:53.653 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:53.094+0000 7f744d14f700 1 --2- 172.21.15.60:0/815468064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74481075c0 0x7f744819b390 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7438009fb0 tx=0x7f7438011b20 comp rx=0 tx=0).stop 2024-02-26T04:47:53.653 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:53.095+0000 7f744d14f700 1 -- 172.21.15.60:0/815468064 shutdown_connections 2024-02-26T04:47:53.653 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:53.095+0000 7f744d14f700 1 --2- 172.21.15.60:0/815468064 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f7430038ee0 0x7f743003b3a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:53.653 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:53.095+0000 7f744d14f700 1 --2- 172.21.15.60:0/815468064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74481075c0 0x7f744819b390 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:53.653 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:53.095+0000 7f744d14f700 1 -- 172.21.15.60:0/815468064 >> 172.21.15.60:0/815468064 conn(0x7f7448102be0 msgr2=0x7f7448103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:53.653 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:53.095+0000 7f744d14f700 1 -- 172.21.15.60:0/815468064 shutdown_connections 2024-02-26T04:47:53.653 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:53.095+0000 7f744d14f700 1 -- 172.21.15.60:0/815468064 wait complete. 2024-02-26T04:47:53.969 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:53 smithi060 ceph-mon[32382]: mgrmap e8: x(active, since 2s) 2024-02-26T04:47:53.970 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:53 smithi060 ceph-mon[32382]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:47:53.970 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:53 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' 2024-02-26T04:47:53.970 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:53 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:47:54.848 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-26T04:47:54.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.068+0000 7fa08ef11700 1 Processor -- start 2024-02-26T04:47:54.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.068+0000 7fa08ef11700 1 -- start start 2024-02-26T04:47:54.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.068+0000 7fa08ef11700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa088105880 0x7fa088105ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:54.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.068+0000 7fa08ef11700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa088106270 con 0x7fa088105880 2024-02-26T04:47:54.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.068+0000 7fa08ccad700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa088105880 0x7fa088105ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:54.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.068+0000 7fa08ccad700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa088105880 0x7fa088105ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46204/0 (socket says 172.21.15.60:46204) 2024-02-26T04:47:54.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.068+0000 7fa08ccad700 1 -- 172.21.15.60:0/3155626291 learned_addr learned my addr 172.21.15.60:0/3155626291 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:54.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.069+0000 7fa08ccad700 1 -- 172.21.15.60:0/3155626291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa088108810 con 0x7fa088105880 2024-02-26T04:47:54.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.069+0000 7fa08ccad700 1 --2- 172.21.15.60:0/3155626291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa088105880 0x7fa088105ca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa0840098a0 tx=0x7fa084009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccaea3c48882e5e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:54.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.069+0000 7fa07f7fe700 1 -- 172.21.15.60:0/3155626291 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa084004160 con 0x7fa088105880 2024-02-26T04:47:54.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.069+0000 7fa07f7fe700 1 -- 172.21.15.60:0/3155626291 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0840042c0 con 0x7fa088105880 2024-02-26T04:47:54.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.069+0000 7fa07f7fe700 1 -- 172.21.15.60:0/3155626291 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa084011ac0 con 0x7fa088105880 2024-02-26T04:47:54.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.069+0000 7fa08ef11700 1 -- 172.21.15.60:0/3155626291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa088105880 msgr2=0x7fa088105ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:54.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.069+0000 7fa08ef11700 1 --2- 172.21.15.60:0/3155626291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa088105880 0x7fa088105ca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa0840098a0 tx=0x7fa084009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:54.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.069+0000 7fa08ef11700 1 -- 172.21.15.60:0/3155626291 shutdown_connections 2024-02-26T04:47:54.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.069+0000 7fa08ef11700 1 --2- 172.21.15.60:0/3155626291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa088105880 0x7fa088105ca0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:54.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.069+0000 7fa08ef11700 1 -- 172.21.15.60:0/3155626291 >> 172.21.15.60:0/3155626291 conn(0x7fa088101210 msgr2=0x7fa088103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:54.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.070+0000 7fa08ef11700 1 -- 172.21.15.60:0/3155626291 shutdown_connections 2024-02-26T04:47:54.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.070+0000 7fa08ef11700 1 -- 172.21.15.60:0/3155626291 wait complete. 2024-02-26T04:47:54.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.070+0000 7fa08ef11700 1 Processor -- start 2024-02-26T04:47:54.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.070+0000 7fa08ef11700 1 -- start start 2024-02-26T04:47:54.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.070+0000 7fa08ef11700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa088105880 0x7fa08819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:54.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.070+0000 7fa08ef11700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08819b900 con 0x7fa088105880 2024-02-26T04:47:54.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.070+0000 7fa08ccad700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa088105880 0x7fa08819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:54.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.070+0000 7fa08ccad700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa088105880 0x7fa08819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46210/0 (socket says 172.21.15.60:46210) 2024-02-26T04:47:54.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.070+0000 7fa08ccad700 1 -- 172.21.15.60:0/2300721027 learned_addr learned my addr 172.21.15.60:0/2300721027 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:54.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.071+0000 7fa08ccad700 1 -- 172.21.15.60:0/2300721027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa084009550 con 0x7fa088105880 2024-02-26T04:47:54.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.071+0000 7fa08ccad700 1 --2- 172.21.15.60:0/2300721027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa088105880 0x7fa08819b3c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa084011780 tx=0x7fa084011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:54.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.071+0000 7fa07dffb700 1 -- 172.21.15.60:0/2300721027 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa084003af0 con 0x7fa088105880 2024-02-26T04:47:54.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.071+0000 7fa07dffb700 1 -- 172.21.15.60:0/2300721027 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa084003c50 con 0x7fa088105880 2024-02-26T04:47:54.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.071+0000 7fa08ef11700 1 -- 172.21.15.60:0/2300721027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08819bb00 con 0x7fa088105880 2024-02-26T04:47:54.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.071+0000 7fa07dffb700 1 -- 172.21.15.60:0/2300721027 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0840197e0 con 0x7fa088105880 2024-02-26T04:47:54.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.071+0000 7fa08ef11700 1 -- 172.21.15.60:0/2300721027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08819bfa0 con 0x7fa088105880 2024-02-26T04:47:54.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.072+0000 7fa07dffb700 1 -- 172.21.15.60:0/2300721027 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fa08401a430 con 0x7fa088105880 2024-02-26T04:47:54.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.072+0000 7fa07dffb700 1 --2- 172.21.15.60:0/2300721027 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fa070038f30 0x7fa07003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:54.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.072+0000 7fa07dffb700 1 -- 172.21.15.60:0/2300721027 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa08404d240 con 0x7fa088105880 2024-02-26T04:47:54.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.072+0000 7fa07ffff700 1 --2- 172.21.15.60:0/2300721027 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fa070038f30 0x7fa07003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:54.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.072+0000 7fa08ef11700 1 -- 172.21.15.60:0/2300721027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa08810ba40 con 0x7fa088105880 2024-02-26T04:47:54.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.072+0000 7fa07ffff700 1 --2- 172.21.15.60:0/2300721027 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fa070038f30 0x7fa07003b3f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa078006fd0 tx=0x7fa078006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:54.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.077+0000 7fa07dffb700 1 -- 172.21.15.60:0/2300721027 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa08402da40 con 0x7fa088105880 2024-02-26T04:47:54.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.279+0000 7fa08ef11700 1 -- 172.21.15.60:0/2300721027 --> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fa088102460 con 0x7fa070038f30 2024-02-26T04:47:54.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.280+0000 7fa07dffb700 1 -- 172.21.15.60:0/2300721027 <== mgr.14120 v2:172.21.15.60:6800/3694881479 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fa088102460 con 0x7fa070038f30 2024-02-26T04:47:54.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.281+0000 7fa08ef11700 1 -- 172.21.15.60:0/2300721027 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fa070038f30 msgr2=0x7fa07003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:54.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.281+0000 7fa08ef11700 1 --2- 172.21.15.60:0/2300721027 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fa070038f30 0x7fa07003b3f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa078006fd0 tx=0x7fa078006e40 comp rx=0 tx=0).stop 2024-02-26T04:47:54.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.281+0000 7fa08ef11700 1 -- 172.21.15.60:0/2300721027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa088105880 msgr2=0x7fa08819b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:54.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.282+0000 7fa08ef11700 1 --2- 172.21.15.60:0/2300721027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa088105880 0x7fa08819b3c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa084011780 tx=0x7fa084011b40 comp rx=0 tx=0).stop 2024-02-26T04:47:54.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.282+0000 7fa08ef11700 1 -- 172.21.15.60:0/2300721027 shutdown_connections 2024-02-26T04:47:54.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.282+0000 7fa08ef11700 1 --2- 172.21.15.60:0/2300721027 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fa070038f30 0x7fa07003b3f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:54.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.282+0000 7fa08ef11700 1 --2- 172.21.15.60:0/2300721027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa088105880 0x7fa08819b3c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:54.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.282+0000 7fa08ef11700 1 -- 172.21.15.60:0/2300721027 >> 172.21.15.60:0/2300721027 conn(0x7fa088101210 msgr2=0x7fa088101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:54.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.282+0000 7fa08ef11700 1 -- 172.21.15.60:0/2300721027 shutdown_connections 2024-02-26T04:47:54.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:54.282+0000 7fa08ef11700 1 -- 172.21.15.60:0/2300721027 wait complete. 2024-02-26T04:47:54.853 INFO:teuthology.orchestra.run.smithi060.stdout:Generating ssh key... 2024-02-26T04:47:55.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:54 smithi060 ceph-mon[32382]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:47:56.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:55 smithi060 ceph-mon[32382]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:47:56.115 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:55 smithi060 ceph-mon[32382]: Generating ssh key... 2024-02-26T04:47:56.115 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: Generating public/private rsa key pair. 2024-02-26T04:47:56.116 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: Your identification has been saved in /tmp/tmp_d4difuz/key. 2024-02-26T04:47:56.116 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: Your public key has been saved in /tmp/tmp_d4difuz/key.pub. 2024-02-26T04:47:56.116 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: The key fingerprint is: 2024-02-26T04:47:56.116 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: SHA256:dzesm2ae8bPRjEk0TvDC3uwcODOt5nwl5ZhjfME0dxQ ceph-f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:47:56.116 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: The key's randomart image is: 2024-02-26T04:47:56.116 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: +---[RSA 3072]----+ 2024-02-26T04:47:56.116 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: | . Eo| 2024-02-26T04:47:56.116 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: | . o oo| 2024-02-26T04:47:56.116 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: | o B +| 2024-02-26T04:47:56.116 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: | . @ +.| 2024-02-26T04:47:56.116 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: | S . B.@=.| 2024-02-26T04:47:56.116 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: | . . @BB+| 2024-02-26T04:47:56.116 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: | =.*=o| 2024-02-26T04:47:56.117 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: | +o*o. | 2024-02-26T04:47:56.117 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: | +B.oo | 2024-02-26T04:47:56.117 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:47:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: +----[SHA256]-----+ 2024-02-26T04:47:56.747 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.221+0000 7f45ea7b6700 1 Processor -- start 2024-02-26T04:47:56.747 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.221+0000 7f45ea7b6700 1 -- start start 2024-02-26T04:47:56.747 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.221+0000 7f45ea7b6700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e41075b0 0x7f45e41079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:56.747 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.221+0000 7f45ea7b6700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e4107fa0 con 0x7f45e41075b0 2024-02-26T04:47:56.747 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.222+0000 7f45e3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e41075b0 0x7f45e41079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:56.747 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.222+0000 7f45e3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e41075b0 0x7f45e41079d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46222/0 (socket says 172.21.15.60:46222) 2024-02-26T04:47:56.747 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.222+0000 7f45e3fff700 1 -- 172.21.15.60:0/2998347116 learned_addr learned my addr 172.21.15.60:0/2998347116 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:56.747 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.222+0000 7f45e3fff700 1 -- 172.21.15.60:0/2998347116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e4108760 con 0x7f45e41075b0 2024-02-26T04:47:56.748 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.222+0000 7f45e3fff700 1 --2- 172.21.15.60:0/2998347116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e41075b0 0x7f45e41079d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f45d40098a0 tx=0x7f45d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e768201b2c95df3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:56.748 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.222+0000 7f45e2ffd700 1 -- 172.21.15.60:0/2998347116 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45d4004160 con 0x7f45e41075b0 2024-02-26T04:47:56.748 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.222+0000 7f45e2ffd700 1 -- 172.21.15.60:0/2998347116 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45d40042c0 con 0x7f45e41075b0 2024-02-26T04:47:56.748 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.222+0000 7f45e2ffd700 1 -- 172.21.15.60:0/2998347116 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45d4011ac0 con 0x7f45e41075b0 2024-02-26T04:47:56.748 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.223+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2998347116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e41075b0 msgr2=0x7f45e41079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:56.748 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.223+0000 7f45ea7b6700 1 --2- 172.21.15.60:0/2998347116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e41075b0 0x7f45e41079d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f45d40098a0 tx=0x7f45d4009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:56.748 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.223+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2998347116 shutdown_connections 2024-02-26T04:47:56.748 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.223+0000 7f45ea7b6700 1 --2- 172.21.15.60:0/2998347116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e41075b0 0x7f45e41079d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:56.748 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.223+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2998347116 >> 172.21.15.60:0/2998347116 conn(0x7f45e4102c10 msgr2=0x7f45e4105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:56.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.223+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2998347116 shutdown_connections 2024-02-26T04:47:56.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.223+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2998347116 wait complete. 2024-02-26T04:47:56.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.223+0000 7f45ea7b6700 1 Processor -- start 2024-02-26T04:47:56.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.224+0000 7f45ea7b6700 1 -- start start 2024-02-26T04:47:56.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.224+0000 7f45ea7b6700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e41075b0 0x7f45e419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:56.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.224+0000 7f45ea7b6700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e419db40 con 0x7f45e41075b0 2024-02-26T04:47:56.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.224+0000 7f45e3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e41075b0 0x7f45e419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:56.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.224+0000 7f45e3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e41075b0 0x7f45e419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46232/0 (socket says 172.21.15.60:46232) 2024-02-26T04:47:56.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.224+0000 7f45e3fff700 1 -- 172.21.15.60:0/2799564558 learned_addr learned my addr 172.21.15.60:0/2799564558 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:56.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.224+0000 7f45e3fff700 1 -- 172.21.15.60:0/2799564558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d4009550 con 0x7f45e41075b0 2024-02-26T04:47:56.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.224+0000 7f45e3fff700 1 --2- 172.21.15.60:0/2799564558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e41075b0 0x7f45e419d600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f45d4011780 tx=0x7f45d4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:56.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.225+0000 7f45e17fa700 1 -- 172.21.15.60:0/2799564558 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45d4003af0 con 0x7f45e41075b0 2024-02-26T04:47:56.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.225+0000 7f45e17fa700 1 -- 172.21.15.60:0/2799564558 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45d4003c50 con 0x7f45e41075b0 2024-02-26T04:47:56.749 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.225+0000 7f45e17fa700 1 -- 172.21.15.60:0/2799564558 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45d40197e0 con 0x7f45e41075b0 2024-02-26T04:47:56.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.225+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2799564558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e419dd40 con 0x7f45e41075b0 2024-02-26T04:47:56.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.225+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2799564558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e419e1e0 con 0x7f45e41075b0 2024-02-26T04:47:56.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.225+0000 7f45e17fa700 1 -- 172.21.15.60:0/2799564558 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f45d401a430 con 0x7f45e41075b0 2024-02-26T04:47:56.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.226+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2799564558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e410b990 con 0x7f45e41075b0 2024-02-26T04:47:56.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.226+0000 7f45e17fa700 1 --2- 172.21.15.60:0/2799564558 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f45cc038ee0 0x7f45cc03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:56.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.226+0000 7f45e17fa700 1 -- 172.21.15.60:0/2799564558 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f45d404d290 con 0x7f45e41075b0 2024-02-26T04:47:56.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.226+0000 7f45e37fe700 1 --2- 172.21.15.60:0/2799564558 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f45cc038ee0 0x7f45cc03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:56.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.226+0000 7f45e37fe700 1 --2- 172.21.15.60:0/2799564558 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f45cc038ee0 0x7f45cc03b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f45d8006fd0 tx=0x7f45d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:56.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.231+0000 7f45e17fa700 1 -- 172.21.15.60:0/2799564558 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f45d401aba0 con 0x7f45e41075b0 2024-02-26T04:47:56.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:55.437+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2799564558 --> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f45e4103df0 con 0x7f45cc038ee0 2024-02-26T04:47:56.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:56.100+0000 7f45e17fa700 1 -- 172.21.15.60:0/2799564558 <== mgr.14120 v2:172.21.15.60:6800/3694881479 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f45e4103df0 con 0x7f45cc038ee0 2024-02-26T04:47:56.750 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:56.102+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2799564558 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f45cc038ee0 msgr2=0x7f45cc03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:56.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:56.102+0000 7f45ea7b6700 1 --2- 172.21.15.60:0/2799564558 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f45cc038ee0 0x7f45cc03b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f45d8006fd0 tx=0x7f45d8006e40 comp rx=0 tx=0).stop 2024-02-26T04:47:56.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:56.102+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2799564558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e41075b0 msgr2=0x7f45e419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:56.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:56.102+0000 7f45ea7b6700 1 --2- 172.21.15.60:0/2799564558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e41075b0 0x7f45e419d600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f45d4011780 tx=0x7f45d4011b40 comp rx=0 tx=0).stop 2024-02-26T04:47:56.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:56.102+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2799564558 shutdown_connections 2024-02-26T04:47:56.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:56.102+0000 7f45ea7b6700 1 --2- 172.21.15.60:0/2799564558 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f45cc038ee0 0x7f45cc03b3a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:56.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:56.102+0000 7f45ea7b6700 1 --2- 172.21.15.60:0/2799564558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e41075b0 0x7f45e419d600 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:56.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:56.102+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2799564558 >> 172.21.15.60:0/2799564558 conn(0x7f45e4102c10 msgr2=0x7f45e41036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:56.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:56.102+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2799564558 shutdown_connections 2024-02-26T04:47:56.751 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:56.102+0000 7f45ea7b6700 1 -- 172.21.15.60:0/2799564558 wait complete. 2024-02-26T04:47:57.395 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:57 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' 2024-02-26T04:47:57.395 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:57 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' 2024-02-26T04:47:57.978 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpAmQHZ/X4yJFbrTJkldnmCUiZH/yImKnS4aVLXUZyreKkQ8yQAyfymSz+EWpNexr/tsjyUvvxhEU36Ls8Azf7trww3lYn3y+GG1sMZWOsiWmHlJBQcHg5PtiEjxtVj8Mc+7eKnDC9yWVwgUXciJYslL4nFcoXHpLkiqeZfJC5u+em/JcQwDJUYgyAeVyX9C99BqZd2ghfd5woGcDOtnbdTqjBkZ5jwb3dvk6cGaFtbwizrldu94SSppVQINJZPzsu+4ZYVvKtncNcvkbW93URlCzYqG72vvhD8dY65TGV+/vNNYmVLjKAXl4Mv14rhdRE1oknDKabGpme5Ge7pgeP56w+BIshYWKGaO0+MOFuIjMGobVvZSqeKy6SxvGUtkEhjnTddIF8aVebY+kkoQyWZdbxmPxsQZ3o9A9GrT2f9DWZYtYo1Z1CbjowZKo6zus5TitVD2Ui2bpkhrL70Bx8m+Bb9Xi7yfSDm5PPwU99LZb/f6K5lGIdi3g1HJqOGUs= ceph-f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:47:57.978 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.179+0000 7f802474f700 1 Processor -- start 2024-02-26T04:47:57.978 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.179+0000 7f802474f700 1 -- start start 2024-02-26T04:47:57.978 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.179+0000 7f802474f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c108040 0x7f801c10a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:57.978 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.179+0000 7f802474f700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c07ae50 con 0x7f801c108040 2024-02-26T04:47:57.978 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.180+0000 7f80224eb700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c108040 0x7f801c10a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:57.978 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.180+0000 7f80224eb700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c108040 0x7f801c10a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46238/0 (socket says 172.21.15.60:46238) 2024-02-26T04:47:57.978 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.180+0000 7f80224eb700 1 -- 172.21.15.60:0/14834072 learned_addr learned my addr 172.21.15.60:0/14834072 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:57.978 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.180+0000 7f80224eb700 1 -- 172.21.15.60:0/14834072 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c10a970 con 0x7f801c108040 2024-02-26T04:47:57.978 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.180+0000 7f80224eb700 1 --2- 172.21.15.60:0/14834072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c108040 0x7f801c10a430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f80180098a0 tx=0x7f8018009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146f04cdb78d2429 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:57.978 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.180+0000 7f80214e9700 1 -- 172.21.15.60:0/14834072 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8018004160 con 0x7f801c108040 2024-02-26T04:47:57.979 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.180+0000 7f80214e9700 1 -- 172.21.15.60:0/14834072 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80180042c0 con 0x7f801c108040 2024-02-26T04:47:57.979 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.180+0000 7f80214e9700 1 -- 172.21.15.60:0/14834072 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8018011ac0 con 0x7f801c108040 2024-02-26T04:47:57.979 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.181+0000 7f802474f700 1 -- 172.21.15.60:0/14834072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c108040 msgr2=0x7f801c10a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:57.979 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.181+0000 7f802474f700 1 --2- 172.21.15.60:0/14834072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c108040 0x7f801c10a430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f80180098a0 tx=0x7f8018009bb0 comp rx=0 tx=0).stop 2024-02-26T04:47:57.979 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.181+0000 7f802474f700 1 -- 172.21.15.60:0/14834072 shutdown_connections 2024-02-26T04:47:57.979 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.181+0000 7f802474f700 1 --2- 172.21.15.60:0/14834072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c108040 0x7f801c10a430 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:57.979 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.181+0000 7f802474f700 1 -- 172.21.15.60:0/14834072 >> 172.21.15.60:0/14834072 conn(0x7f801c101a10 msgr2=0x7f801c103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:57.979 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.181+0000 7f802474f700 1 -- 172.21.15.60:0/14834072 shutdown_connections 2024-02-26T04:47:57.979 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.181+0000 7f802474f700 1 -- 172.21.15.60:0/14834072 wait complete. 2024-02-26T04:47:57.979 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.181+0000 7f802474f700 1 Processor -- start 2024-02-26T04:47:57.979 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.182+0000 7f802474f700 1 -- start start 2024-02-26T04:47:57.979 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.182+0000 7f802474f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c108040 0x7f801c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:57.979 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.182+0000 7f802474f700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c19b8d0 con 0x7f801c108040 2024-02-26T04:47:57.980 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.182+0000 7f80224eb700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c108040 0x7f801c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:57.980 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.182+0000 7f80224eb700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c108040 0x7f801c19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46242/0 (socket says 172.21.15.60:46242) 2024-02-26T04:47:57.980 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.182+0000 7f80224eb700 1 -- 172.21.15.60:0/2618625214 learned_addr learned my addr 172.21.15.60:0/2618625214 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:47:57.980 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.182+0000 7f80224eb700 1 -- 172.21.15.60:0/2618625214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8018009550 con 0x7f801c108040 2024-02-26T04:47:57.980 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.182+0000 7f80224eb700 1 --2- 172.21.15.60:0/2618625214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c108040 0x7f801c19b390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8018011a70 tx=0x7f8018011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:57.980 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.182+0000 7f80137fe700 1 -- 172.21.15.60:0/2618625214 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80180039c0 con 0x7f801c108040 2024-02-26T04:47:57.980 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.182+0000 7f80137fe700 1 -- 172.21.15.60:0/2618625214 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8018003b20 con 0x7f801c108040 2024-02-26T04:47:57.980 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.182+0000 7f802474f700 1 -- 172.21.15.60:0/2618625214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801c19bad0 con 0x7f801c108040 2024-02-26T04:47:57.980 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.183+0000 7f80137fe700 1 -- 172.21.15.60:0/2618625214 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8018024610 con 0x7f801c108040 2024-02-26T04:47:57.980 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.183+0000 7f802474f700 1 -- 172.21.15.60:0/2618625214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801c19bf70 con 0x7f801c108040 2024-02-26T04:47:57.980 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.183+0000 7f80137fe700 1 -- 172.21.15.60:0/2618625214 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f801802d3f0 con 0x7f801c108040 2024-02-26T04:47:57.980 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.183+0000 7f80137fe700 1 --2- 172.21.15.60:0/2618625214 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f8008038f30 0x7f800803b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:47:57.980 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.183+0000 7f802474f700 1 -- 172.21.15.60:0/2618625214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801c10b720 con 0x7f801c108040 2024-02-26T04:47:57.981 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.184+0000 7f80137fe700 1 -- 172.21.15.60:0/2618625214 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f801804d020 con 0x7f801c108040 2024-02-26T04:47:57.981 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.184+0000 7f8021cea700 1 --2- 172.21.15.60:0/2618625214 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f8008038f30 0x7f800803b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:47:57.981 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.184+0000 7f8021cea700 1 --2- 172.21.15.60:0/2618625214 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f8008038f30 0x7f800803b3f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f800c006fd0 tx=0x7f800c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:47:57.982 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.188+0000 7f80137fe700 1 -- 172.21.15.60:0/2618625214 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8018018ba0 con 0x7f801c108040 2024-02-26T04:47:57.982 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.393+0000 7f802474f700 1 -- 172.21.15.60:0/2618625214 --> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f801c102cb0 con 0x7f8008038f30 2024-02-26T04:47:57.982 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.394+0000 7f80137fe700 1 -- 172.21.15.60:0/2618625214 <== mgr.14120 v2:172.21.15.60:6800/3694881479 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f801c102cb0 con 0x7f8008038f30 2024-02-26T04:47:57.982 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.396+0000 7f802474f700 1 -- 172.21.15.60:0/2618625214 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f8008038f30 msgr2=0x7f800803b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:57.982 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.396+0000 7f802474f700 1 --2- 172.21.15.60:0/2618625214 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f8008038f30 0x7f800803b3f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f800c006fd0 tx=0x7f800c006e40 comp rx=0 tx=0).stop 2024-02-26T04:47:57.982 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.396+0000 7f802474f700 1 -- 172.21.15.60:0/2618625214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c108040 msgr2=0x7f801c19b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:47:57.982 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.396+0000 7f802474f700 1 --2- 172.21.15.60:0/2618625214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c108040 0x7f801c19b390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8018011a70 tx=0x7f8018011b50 comp rx=0 tx=0).stop 2024-02-26T04:47:57.983 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.396+0000 7f802474f700 1 -- 172.21.15.60:0/2618625214 shutdown_connections 2024-02-26T04:47:57.983 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.396+0000 7f802474f700 1 --2- 172.21.15.60:0/2618625214 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f8008038f30 0x7f800803b3f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:57.983 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.396+0000 7f802474f700 1 --2- 172.21.15.60:0/2618625214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c108040 0x7f801c19b390 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:47:57.983 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.396+0000 7f802474f700 1 -- 172.21.15.60:0/2618625214 >> 172.21.15.60:0/2618625214 conn(0x7f801c101a10 msgr2=0x7f801c1025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:47:57.983 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.396+0000 7f802474f700 1 -- 172.21.15.60:0/2618625214 shutdown_connections 2024-02-26T04:47:57.983 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:57.396+0000 7f802474f700 1 -- 172.21.15.60:0/2618625214 wait complete. 2024-02-26T04:47:57.983 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-26T04:47:57.983 INFO:teuthology.orchestra.run.smithi060.stdout:Adding key to root@localhost authorized_keys... 2024-02-26T04:47:57.983 INFO:teuthology.orchestra.run.smithi060.stdout:Adding host smithi060... 2024-02-26T04:47:58.223 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:58 smithi060 ceph-mon[32382]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:47:59.323 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:47:59 smithi060 ceph-mon[32382]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi060", "addr": "172.21.15.60", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:48:00.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:00 smithi060 ceph-mon[32382]: Deploying cephadm binary to smithi060 2024-02-26T04:48:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Added host 'smithi060' with addr '172.21.15.60' 2024-02-26T04:48:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.415+0000 7f0367a66700 1 Processor -- start 2024-02-26T04:48:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.415+0000 7f0367a66700 1 -- start start 2024-02-26T04:48:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.416+0000 7f0367a66700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036007ce20 0x7f036007b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.416+0000 7f0367a66700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036007b860 con 0x7f036007ce20 2024-02-26T04:48:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.416+0000 7f0365802700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036007ce20 0x7f036007b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.416+0000 7f0365802700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036007ce20 0x7f036007b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46258/0 (socket says 172.21.15.60:46258) 2024-02-26T04:48:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.416+0000 7f0365802700 1 -- 172.21.15.60:0/1423077925 learned_addr learned my addr 172.21.15.60:0/1423077925 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.416+0000 7f0365802700 1 -- 172.21.15.60:0/1423077925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036007b9a0 con 0x7f036007ce20 2024-02-26T04:48:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.416+0000 7f0365802700 1 --2- 172.21.15.60:0/1423077925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036007ce20 0x7f036007b320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0350009e80 tx=0x7f035000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4aec9fd8bf0972 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.417+0000 7f0364800700 1 -- 172.21.15.60:0/1423077925 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f035000e070 con 0x7f036007ce20 2024-02-26T04:48:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.417+0000 7f0364800700 1 -- 172.21.15.60:0/1423077925 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f035000b8c0 con 0x7f036007ce20 2024-02-26T04:48:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.417+0000 7f0364800700 1 -- 172.21.15.60:0/1423077925 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0350004550 con 0x7f036007ce20 2024-02-26T04:48:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.417+0000 7f0367a66700 1 -- 172.21.15.60:0/1423077925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036007ce20 msgr2=0x7f036007b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.417+0000 7f0367a66700 1 --2- 172.21.15.60:0/1423077925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036007ce20 0x7f036007b320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0350009e80 tx=0x7f035000b330 comp rx=0 tx=0).stop 2024-02-26T04:48:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.417+0000 7f0367a66700 1 -- 172.21.15.60:0/1423077925 shutdown_connections 2024-02-26T04:48:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.417+0000 7f0367a66700 1 --2- 172.21.15.60:0/1423077925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036007ce20 0x7f036007b320 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.417+0000 7f0367a66700 1 -- 172.21.15.60:0/1423077925 >> 172.21.15.60:0/1423077925 conn(0x7f0360102be0 msgr2=0x7f0360105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.417+0000 7f0367a66700 1 -- 172.21.15.60:0/1423077925 shutdown_connections 2024-02-26T04:48:03.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.417+0000 7f0367a66700 1 -- 172.21.15.60:0/1423077925 wait complete. 2024-02-26T04:48:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.418+0000 7f0367a66700 1 Processor -- start 2024-02-26T04:48:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.418+0000 7f0367a66700 1 -- start start 2024-02-26T04:48:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.418+0000 7f0367a66700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036007ce20 0x7f036019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.418+0000 7f0367a66700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036019d930 con 0x7f036007ce20 2024-02-26T04:48:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.418+0000 7f0365802700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036007ce20 0x7f036019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.418+0000 7f0365802700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036007ce20 0x7f036019d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46272/0 (socket says 172.21.15.60:46272) 2024-02-26T04:48:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.418+0000 7f0365802700 1 -- 172.21.15.60:0/3257847985 learned_addr learned my addr 172.21.15.60:0/3257847985 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.419+0000 7f0365802700 1 -- 172.21.15.60:0/3257847985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0350009b30 con 0x7f036007ce20 2024-02-26T04:48:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.419+0000 7f0365802700 1 --2- 172.21.15.60:0/3257847985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036007ce20 0x7f036019d3f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f03500042e0 tx=0x7f0350003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.419+0000 7f035effd700 1 -- 172.21.15.60:0/3257847985 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f035000e040 con 0x7f036007ce20 2024-02-26T04:48:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.419+0000 7f0367a66700 1 -- 172.21.15.60:0/3257847985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036019db30 con 0x7f036007ce20 2024-02-26T04:48:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.419+0000 7f035effd700 1 -- 172.21.15.60:0/3257847985 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f035000bd70 con 0x7f036007ce20 2024-02-26T04:48:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.419+0000 7f035effd700 1 -- 172.21.15.60:0/3257847985 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f035001c7f0 con 0x7f036007ce20 2024-02-26T04:48:03.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.419+0000 7f0367a66700 1 -- 172.21.15.60:0/3257847985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036019dfb0 con 0x7f036007ce20 2024-02-26T04:48:03.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.420+0000 7f035effd700 1 -- 172.21.15.60:0/3257847985 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f035001cab0 con 0x7f036007ce20 2024-02-26T04:48:03.615 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.420+0000 7f0367a66700 1 -- 172.21.15.60:0/3257847985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036010d930 con 0x7f036007ce20 2024-02-26T04:48:03.615 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.420+0000 7f035effd700 1 --2- 172.21.15.60:0/3257847985 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f0348038e90 0x7f034803b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:03.615 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.420+0000 7f035effd700 1 -- 172.21.15.60:0/3257847985 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f035004d190 con 0x7f036007ce20 2024-02-26T04:48:03.615 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.420+0000 7f0365001700 1 --2- 172.21.15.60:0/3257847985 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f0348038e90 0x7f034803b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:03.615 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.421+0000 7f0365001700 1 --2- 172.21.15.60:0/3257847985 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f0348038e90 0x7f034803b350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0354006fd0 tx=0x7f0354006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:03.615 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.425+0000 7f035effd700 1 -- 172.21.15.60:0/3257847985 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0350011390 con 0x7f036007ce20 2024-02-26T04:48:03.616 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:47:58.633+0000 7f0367a66700 1 -- 172.21.15.60:0/3257847985 --> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi060", "addr": "172.21.15.60", "target": ["mon-mgr", ""]}) v1 -- 0x7f0360103d70 con 0x7f0348038e90 2024-02-26T04:48:03.616 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:02.991+0000 7f035effd700 1 -- 172.21.15.60:0/3257847985 <== mgr.14120 v2:172.21.15.60:6800/3694881479 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f0360103d70 con 0x7f0348038e90 2024-02-26T04:48:03.616 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:02.994+0000 7f0367a66700 1 -- 172.21.15.60:0/3257847985 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f0348038e90 msgr2=0x7f034803b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:03.616 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:02.994+0000 7f0367a66700 1 --2- 172.21.15.60:0/3257847985 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f0348038e90 0x7f034803b350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0354006fd0 tx=0x7f0354006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:03.616 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:02.994+0000 7f0367a66700 1 -- 172.21.15.60:0/3257847985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036007ce20 msgr2=0x7f036019d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:03.616 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:02.994+0000 7f0367a66700 1 --2- 172.21.15.60:0/3257847985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036007ce20 0x7f036019d3f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f03500042e0 tx=0x7f0350003d00 comp rx=0 tx=0).stop 2024-02-26T04:48:03.616 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:02.994+0000 7f0367a66700 1 -- 172.21.15.60:0/3257847985 shutdown_connections 2024-02-26T04:48:03.616 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:02.994+0000 7f0367a66700 1 --2- 172.21.15.60:0/3257847985 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f0348038e90 0x7f034803b350 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:03.616 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:02.994+0000 7f0367a66700 1 --2- 172.21.15.60:0/3257847985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036007ce20 0x7f036019d3f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:03.616 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:02.994+0000 7f0367a66700 1 -- 172.21.15.60:0/3257847985 >> 172.21.15.60:0/3257847985 conn(0x7f0360102be0 msgr2=0x7f0360103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:03.616 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:02.994+0000 7f0367a66700 1 -- 172.21.15.60:0/3257847985 shutdown_connections 2024-02-26T04:48:03.616 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:02.994+0000 7f0367a66700 1 -- 172.21.15.60:0/3257847985 wait complete. 2024-02-26T04:48:03.616 INFO:teuthology.orchestra.run.smithi060.stdout:Deploying unmanaged mon service... 2024-02-26T04:48:04.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:03 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' 2024-02-26T04:48:04.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:03 smithi060 ceph-mon[32382]: Added host smithi060 2024-02-26T04:48:04.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:03 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:48:04.965 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-26T04:48:04.966 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.023+0000 7f54e76f5700 1 Processor -- start 2024-02-26T04:48:04.966 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.024+0000 7f54e76f5700 1 -- start start 2024-02-26T04:48:04.966 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.024+0000 7f54e76f5700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e007ce50 0x7f54e007b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:04.966 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.024+0000 7f54e76f5700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e007b840 con 0x7f54e007ce50 2024-02-26T04:48:04.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.024+0000 7f54e5491700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e007ce50 0x7f54e007b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:04.967 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.024+0000 7f54e5491700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e007ce50 0x7f54e007b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47064/0 (socket says 172.21.15.60:47064) 2024-02-26T04:48:04.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.024+0000 7f54e5491700 1 -- 172.21.15.60:0/3042676002 learned_addr learned my addr 172.21.15.60:0/3042676002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:04.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.024+0000 7f54e5491700 1 -- 172.21.15.60:0/3042676002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e007b980 con 0x7f54e007ce50 2024-02-26T04:48:04.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.025+0000 7f54e5491700 1 --2- 172.21.15.60:0/3042676002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e007ce50 0x7f54e007b300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0098a0 tx=0x7f54dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f626b696fa5660 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:04.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.025+0000 7f54d3fff700 1 -- 172.21.15.60:0/3042676002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54dc004160 con 0x7f54e007ce50 2024-02-26T04:48:04.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.025+0000 7f54d3fff700 1 -- 172.21.15.60:0/3042676002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54dc0042c0 con 0x7f54e007ce50 2024-02-26T04:48:04.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.025+0000 7f54d3fff700 1 -- 172.21.15.60:0/3042676002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54dc011ac0 con 0x7f54e007ce50 2024-02-26T04:48:04.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.025+0000 7f54e76f5700 1 -- 172.21.15.60:0/3042676002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e007ce50 msgr2=0x7f54e007b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:04.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.025+0000 7f54e76f5700 1 --2- 172.21.15.60:0/3042676002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e007ce50 0x7f54e007b300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0098a0 tx=0x7f54dc009bb0 comp rx=0 tx=0).stop 2024-02-26T04:48:04.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.025+0000 7f54e76f5700 1 -- 172.21.15.60:0/3042676002 shutdown_connections 2024-02-26T04:48:04.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.025+0000 7f54e76f5700 1 --2- 172.21.15.60:0/3042676002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e007ce50 0x7f54e007b300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:04.968 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.025+0000 7f54e76f5700 1 -- 172.21.15.60:0/3042676002 >> 172.21.15.60:0/3042676002 conn(0x7f54e0102c10 msgr2=0x7f54e0105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.025+0000 7f54e76f5700 1 -- 172.21.15.60:0/3042676002 shutdown_connections 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.025+0000 7f54e76f5700 1 -- 172.21.15.60:0/3042676002 wait complete. 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.026+0000 7f54e76f5700 1 Processor -- start 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.026+0000 7f54e76f5700 1 -- start start 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.026+0000 7f54e76f5700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e007ce50 0x7f54e019d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.026+0000 7f54e76f5700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e019da40 con 0x7f54e007ce50 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.026+0000 7f54e5491700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e007ce50 0x7f54e019d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.026+0000 7f54e5491700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e007ce50 0x7f54e019d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47066/0 (socket says 172.21.15.60:47066) 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.026+0000 7f54e5491700 1 -- 172.21.15.60:0/1370058442 learned_addr learned my addr 172.21.15.60:0/1370058442 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.026+0000 7f54e5491700 1 -- 172.21.15.60:0/1370058442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc009550 con 0x7f54e007ce50 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.027+0000 7f54e5491700 1 --2- 172.21.15.60:0/1370058442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e007ce50 0x7f54e019d500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f54dc003f70 tx=0x7f54dc011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.027+0000 7f54d27fc700 1 -- 172.21.15.60:0/1370058442 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54dc0039f0 con 0x7f54e007ce50 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.027+0000 7f54e76f5700 1 -- 172.21.15.60:0/1370058442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e019dc40 con 0x7f54e007ce50 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.027+0000 7f54d27fc700 1 -- 172.21.15.60:0/1370058442 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54dc003b50 con 0x7f54e007ce50 2024-02-26T04:48:04.969 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.027+0000 7f54d27fc700 1 -- 172.21.15.60:0/1370058442 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54dc01a760 con 0x7f54e007ce50 2024-02-26T04:48:04.970 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.027+0000 7f54e76f5700 1 -- 172.21.15.60:0/1370058442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e019e0e0 con 0x7f54e007ce50 2024-02-26T04:48:04.970 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.028+0000 7f54d27fc700 1 -- 172.21.15.60:0/1370058442 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f54dc01a9d0 con 0x7f54e007ce50 2024-02-26T04:48:04.970 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.028+0000 7f54e76f5700 1 -- 172.21.15.60:0/1370058442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e010d940 con 0x7f54e007ce50 2024-02-26T04:48:04.970 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.028+0000 7f54d27fc700 1 --2- 172.21.15.60:0/1370058442 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f54cc038e90 0x7f54cc03b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:04.970 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.028+0000 7f54d27fc700 1 -- 172.21.15.60:0/1370058442 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f54dc04d340 con 0x7f54e007ce50 2024-02-26T04:48:04.970 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.028+0000 7f54e4c90700 1 --2- 172.21.15.60:0/1370058442 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f54cc038e90 0x7f54cc03b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:04.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.028+0000 7f54e4c90700 1 --2- 172.21.15.60:0/1370058442 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f54cc038e90 0x7f54cc03b350 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f54d4006fd0 tx=0x7f54d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:04.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.033+0000 7f54d27fc700 1 -- 172.21.15.60:0/1370058442 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f54dc01ac80 con 0x7f54e007ce50 2024-02-26T04:48:04.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.271+0000 7f54e76f5700 1 -- 172.21.15.60:0/1370058442 --> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f54e0103df0 con 0x7f54cc038e90 2024-02-26T04:48:04.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.275+0000 7f54d27fc700 1 -- 172.21.15.60:0/1370058442 <== mgr.14120 v2:172.21.15.60:6800/3694881479 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f54e0103df0 con 0x7f54cc038e90 2024-02-26T04:48:04.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.277+0000 7f54e76f5700 1 -- 172.21.15.60:0/1370058442 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f54cc038e90 msgr2=0x7f54cc03b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:04.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.277+0000 7f54e76f5700 1 --2- 172.21.15.60:0/1370058442 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f54cc038e90 0x7f54cc03b350 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f54d4006fd0 tx=0x7f54d4006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:04.971 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.277+0000 7f54e76f5700 1 -- 172.21.15.60:0/1370058442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e007ce50 msgr2=0x7f54e019d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:04.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.277+0000 7f54e76f5700 1 --2- 172.21.15.60:0/1370058442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e007ce50 0x7f54e019d500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f54dc003f70 tx=0x7f54dc011b40 comp rx=0 tx=0).stop 2024-02-26T04:48:04.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.277+0000 7f54e76f5700 1 -- 172.21.15.60:0/1370058442 shutdown_connections 2024-02-26T04:48:04.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.277+0000 7f54e76f5700 1 --2- 172.21.15.60:0/1370058442 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f54cc038e90 0x7f54cc03b350 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:04.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.277+0000 7f54e76f5700 1 --2- 172.21.15.60:0/1370058442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e007ce50 0x7f54e019d500 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:04.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.277+0000 7f54e76f5700 1 -- 172.21.15.60:0/1370058442 >> 172.21.15.60:0/1370058442 conn(0x7f54e0102c10 msgr2=0x7f54e01036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:04.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.278+0000 7f54e76f5700 1 -- 172.21.15.60:0/1370058442 shutdown_connections 2024-02-26T04:48:04.972 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:04.278+0000 7f54e76f5700 1 -- 172.21.15.60:0/1370058442 wait complete. 2024-02-26T04:48:04.973 INFO:teuthology.orchestra.run.smithi060.stdout:Deploying unmanaged mgr service... 2024-02-26T04:48:05.541 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:05 smithi060 ceph-mon[32382]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:48:05.541 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:05 smithi060 ceph-mon[32382]: Saving service mon spec with placement count:5 2024-02-26T04:48:05.541 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:05 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' 2024-02-26T04:48:06.531 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-26T04:48:06.531 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.428+0000 7fefb1951700 1 Processor -- start 2024-02-26T04:48:06.531 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.428+0000 7fefb1951700 1 -- start start 2024-02-26T04:48:06.531 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.428+0000 7fefb1951700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefac07ceb0 0x7fefac07b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:06.531 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.428+0000 7fefb1951700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac07b930 con 0x7fefac07ceb0 2024-02-26T04:48:06.531 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.428+0000 7fefaaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefac07ceb0 0x7fefac07b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:06.531 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.428+0000 7fefaaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefac07ceb0 0x7fefac07b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47076/0 (socket says 172.21.15.60:47076) 2024-02-26T04:48:06.531 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.428+0000 7fefaaffd700 1 -- 172.21.15.60:0/1574531213 learned_addr learned my addr 172.21.15.60:0/1574531213 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:06.531 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.429+0000 7fefaaffd700 1 -- 172.21.15.60:0/1574531213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac07ba70 con 0x7fefac07ceb0 2024-02-26T04:48:06.531 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.429+0000 7fefaaffd700 1 --2- 172.21.15.60:0/1574531213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefac07ceb0 0x7fefac07b360 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0098a0 tx=0x7fef9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7bcc3e694dc8f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:06.531 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.429+0000 7fefa9ffb700 1 -- 172.21.15.60:0/1574531213 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef9c004160 con 0x7fefac07ceb0 2024-02-26T04:48:06.532 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.429+0000 7fefa9ffb700 1 -- 172.21.15.60:0/1574531213 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef9c0042c0 con 0x7fefac07ceb0 2024-02-26T04:48:06.532 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.429+0000 7fefa9ffb700 1 -- 172.21.15.60:0/1574531213 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef9c011ac0 con 0x7fefac07ceb0 2024-02-26T04:48:06.532 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.430+0000 7fefb1951700 1 -- 172.21.15.60:0/1574531213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefac07ceb0 msgr2=0x7fefac07b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:06.532 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.430+0000 7fefb1951700 1 --2- 172.21.15.60:0/1574531213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefac07ceb0 0x7fefac07b360 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0098a0 tx=0x7fef9c009bb0 comp rx=0 tx=0).stop 2024-02-26T04:48:06.532 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.430+0000 7fefb1951700 1 -- 172.21.15.60:0/1574531213 shutdown_connections 2024-02-26T04:48:06.532 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.430+0000 7fefb1951700 1 --2- 172.21.15.60:0/1574531213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefac07ceb0 0x7fefac07b360 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:06.532 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.430+0000 7fefb1951700 1 -- 172.21.15.60:0/1574531213 >> 172.21.15.60:0/1574531213 conn(0x7fefac102be0 msgr2=0x7fefac105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:06.532 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.430+0000 7fefb1951700 1 -- 172.21.15.60:0/1574531213 shutdown_connections 2024-02-26T04:48:06.532 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.430+0000 7fefb1951700 1 -- 172.21.15.60:0/1574531213 wait complete. 2024-02-26T04:48:06.532 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.431+0000 7fefb1951700 1 Processor -- start 2024-02-26T04:48:06.533 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.431+0000 7fefb1951700 1 -- start start 2024-02-26T04:48:06.533 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.431+0000 7fefb1951700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefac07ceb0 0x7fefac19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:06.533 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.431+0000 7fefb1951700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac19dae0 con 0x7fefac07ceb0 2024-02-26T04:48:06.533 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.431+0000 7fefaaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefac07ceb0 0x7fefac19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:06.533 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.431+0000 7fefaaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefac07ceb0 0x7fefac19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47090/0 (socket says 172.21.15.60:47090) 2024-02-26T04:48:06.533 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.431+0000 7fefaaffd700 1 -- 172.21.15.60:0/511499527 learned_addr learned my addr 172.21.15.60:0/511499527 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:06.533 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.432+0000 7fefaaffd700 1 -- 172.21.15.60:0/511499527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c009550 con 0x7fefac07ceb0 2024-02-26T04:48:06.533 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.432+0000 7fefaaffd700 1 --2- 172.21.15.60:0/511499527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefac07ceb0 0x7fefac19d5a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009fb0 tx=0x7fef9c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:06.533 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.432+0000 7fefb094f700 1 -- 172.21.15.60:0/511499527 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef9c0038b0 con 0x7fefac07ceb0 2024-02-26T04:48:06.533 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.432+0000 7fefb094f700 1 -- 172.21.15.60:0/511499527 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef9c003a10 con 0x7fefac07ceb0 2024-02-26T04:48:06.533 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.433+0000 7fefb1951700 1 -- 172.21.15.60:0/511499527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefac19dce0 con 0x7fefac07ceb0 2024-02-26T04:48:06.533 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.433+0000 7fefb1951700 1 -- 172.21.15.60:0/511499527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefac19e0c0 con 0x7fefac07ceb0 2024-02-26T04:48:06.533 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.433+0000 7fefb094f700 1 -- 172.21.15.60:0/511499527 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef9c0195e0 con 0x7fefac07ceb0 2024-02-26T04:48:06.533 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.434+0000 7fefb094f700 1 -- 172.21.15.60:0/511499527 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fef9c01a480 con 0x7fefac07ceb0 2024-02-26T04:48:06.534 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.435+0000 7fefb094f700 1 --2- 172.21.15.60:0/511499527 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fef94038b80 0x7fef9403b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:06.534 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.435+0000 7fefb094f700 1 -- 172.21.15.60:0/511499527 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fef9c04d0e0 con 0x7fefac07ceb0 2024-02-26T04:48:06.534 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.435+0000 7fefaa7fc700 1 --2- 172.21.15.60:0/511499527 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fef94038b80 0x7fef9403b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:06.534 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.435+0000 7fefb1951700 1 -- 172.21.15.60:0/511499527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefac10d9b0 con 0x7fefac07ceb0 2024-02-26T04:48:06.534 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.436+0000 7fefaa7fc700 1 --2- 172.21.15.60:0/511499527 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fef94038b80 0x7fef9403b040 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fefa0006fd0 tx=0x7fefa0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:06.534 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.441+0000 7fefb094f700 1 -- 172.21.15.60:0/511499527 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef9c019740 con 0x7fefac07ceb0 2024-02-26T04:48:06.534 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.659+0000 7fefb1951700 1 -- 172.21.15.60:0/511499527 --> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fefac103ec0 con 0x7fef94038b80 2024-02-26T04:48:06.534 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.663+0000 7fefb094f700 1 -- 172.21.15.60:0/511499527 <== mgr.14120 v2:172.21.15.60:6800/3694881479 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fefac103ec0 con 0x7fef94038b80 2024-02-26T04:48:06.534 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.665+0000 7fefb1951700 1 -- 172.21.15.60:0/511499527 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fef94038b80 msgr2=0x7fef9403b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:06.534 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.665+0000 7fefb1951700 1 --2- 172.21.15.60:0/511499527 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fef94038b80 0x7fef9403b040 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fefa0006fd0 tx=0x7fefa0006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:06.534 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.665+0000 7fefb1951700 1 -- 172.21.15.60:0/511499527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefac07ceb0 msgr2=0x7fefac19d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:06.534 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.665+0000 7fefb1951700 1 --2- 172.21.15.60:0/511499527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefac07ceb0 0x7fefac19d5a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009fb0 tx=0x7fef9c011b20 comp rx=0 tx=0).stop 2024-02-26T04:48:06.534 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.665+0000 7fefb1951700 1 -- 172.21.15.60:0/511499527 shutdown_connections 2024-02-26T04:48:06.535 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.665+0000 7fefb1951700 1 --2- 172.21.15.60:0/511499527 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7fef94038b80 0x7fef9403b040 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:06.535 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.665+0000 7fefb1951700 1 --2- 172.21.15.60:0/511499527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefac07ceb0 0x7fefac19d5a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:06.535 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.665+0000 7fefb1951700 1 -- 172.21.15.60:0/511499527 >> 172.21.15.60:0/511499527 conn(0x7fefac102be0 msgr2=0x7fefac1037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:06.535 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.666+0000 7fefb1951700 1 -- 172.21.15.60:0/511499527 shutdown_connections 2024-02-26T04:48:06.535 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:05.666+0000 7fefb1951700 1 -- 172.21.15.60:0/511499527 wait complete. 2024-02-26T04:48:06.901 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:06 smithi060 ceph-mon[32382]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:48:06.901 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:06 smithi060 ceph-mon[32382]: Saving service mgr spec with placement count:2 2024-02-26T04:48:06.901 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:06 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' 2024-02-26T04:48:07.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.005+0000 7f8dae0f0700 1 Processor -- start 2024-02-26T04:48:07.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.005+0000 7f8dae0f0700 1 -- start start 2024-02-26T04:48:07.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.005+0000 7f8dae0f0700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da8108030 0x7f8da810a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:07.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.005+0000 7f8dae0f0700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da807ae70 con 0x7f8da8108030 2024-02-26T04:48:07.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.005+0000 7f8da77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da8108030 0x7f8da810a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:07.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.006+0000 7f8da77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da8108030 0x7f8da810a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47100/0 (socket says 172.21.15.60:47100) 2024-02-26T04:48:07.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.006+0000 7f8da77fe700 1 -- 172.21.15.60:0/2088137203 learned_addr learned my addr 172.21.15.60:0/2088137203 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:07.849 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.006+0000 7f8da77fe700 1 -- 172.21.15.60:0/2088137203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da810a960 con 0x7f8da8108030 2024-02-26T04:48:07.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.006+0000 7f8da77fe700 1 --2- 172.21.15.60:0/2088137203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da8108030 0x7f8da810a420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009e80 tx=0x7f8d9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f74d625a15d23f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:07.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.006+0000 7f8da67fc700 1 -- 172.21.15.60:0/2088137203 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d9800e070 con 0x7f8da8108030 2024-02-26T04:48:07.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.006+0000 7f8da67fc700 1 -- 172.21.15.60:0/2088137203 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d9800b8c0 con 0x7f8da8108030 2024-02-26T04:48:07.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.006+0000 7f8da67fc700 1 -- 172.21.15.60:0/2088137203 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d98004550 con 0x7f8da8108030 2024-02-26T04:48:07.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.007+0000 7f8dae0f0700 1 -- 172.21.15.60:0/2088137203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da8108030 msgr2=0x7f8da810a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:07.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.007+0000 7f8dae0f0700 1 --2- 172.21.15.60:0/2088137203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da8108030 0x7f8da810a420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009e80 tx=0x7f8d9800b330 comp rx=0 tx=0).stop 2024-02-26T04:48:07.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.007+0000 7f8dae0f0700 1 -- 172.21.15.60:0/2088137203 shutdown_connections 2024-02-26T04:48:07.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.007+0000 7f8dae0f0700 1 --2- 172.21.15.60:0/2088137203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da8108030 0x7f8da810a420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:07.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.007+0000 7f8dae0f0700 1 -- 172.21.15.60:0/2088137203 >> 172.21.15.60:0/2088137203 conn(0x7f8da8101a00 msgr2=0x7f8da8103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:07.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.007+0000 7f8dae0f0700 1 -- 172.21.15.60:0/2088137203 shutdown_connections 2024-02-26T04:48:07.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.007+0000 7f8dae0f0700 1 -- 172.21.15.60:0/2088137203 wait complete. 2024-02-26T04:48:07.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.007+0000 7f8dae0f0700 1 Processor -- start 2024-02-26T04:48:07.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.008+0000 7f8dae0f0700 1 -- start start 2024-02-26T04:48:07.850 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.008+0000 7f8dae0f0700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da8108030 0x7f8da81078b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:07.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.008+0000 7f8dae0f0700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da8107df0 con 0x7f8da8108030 2024-02-26T04:48:07.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.008+0000 7f8da77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da8108030 0x7f8da81078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:07.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.008+0000 7f8da77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da8108030 0x7f8da81078b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47112/0 (socket says 172.21.15.60:47112) 2024-02-26T04:48:07.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.008+0000 7f8da77fe700 1 -- 172.21.15.60:0/1703887704 learned_addr learned my addr 172.21.15.60:0/1703887704 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:07.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.008+0000 7f8da77fe700 1 -- 172.21.15.60:0/1703887704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d98009b30 con 0x7f8da8108030 2024-02-26T04:48:07.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.008+0000 7f8da77fe700 1 --2- 172.21.15.60:0/1703887704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da8108030 0x7f8da81078b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8d980042e0 tx=0x7f8d98003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:07.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.008+0000 7f8da4ff9700 1 -- 172.21.15.60:0/1703887704 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d9800e040 con 0x7f8da8108030 2024-02-26T04:48:07.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.008+0000 7f8da4ff9700 1 -- 172.21.15.60:0/1703887704 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d9800bd70 con 0x7f8da8108030 2024-02-26T04:48:07.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.008+0000 7f8dae0f0700 1 -- 172.21.15.60:0/1703887704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da8105fe0 con 0x7f8da8108030 2024-02-26T04:48:07.851 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.008+0000 7f8da4ff9700 1 -- 172.21.15.60:0/1703887704 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d9801c7f0 con 0x7f8da8108030 2024-02-26T04:48:07.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.008+0000 7f8dae0f0700 1 -- 172.21.15.60:0/1703887704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da8106460 con 0x7f8da8108030 2024-02-26T04:48:07.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.009+0000 7f8da4ff9700 1 -- 172.21.15.60:0/1703887704 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f8d9801ca60 con 0x7f8da8108030 2024-02-26T04:48:07.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.009+0000 7f8da4ff9700 1 --2- 172.21.15.60:0/1703887704 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f8d94038ee0 0x7f8d9403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:07.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.009+0000 7f8da4ff9700 1 -- 172.21.15.60:0/1703887704 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8d9804cf60 con 0x7f8da8108030 2024-02-26T04:48:07.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.010+0000 7f8da6ffd700 1 --2- 172.21.15.60:0/1703887704 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f8d94038ee0 0x7f8d9403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:07.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.010+0000 7f8dae0f0700 1 -- 172.21.15.60:0/1703887704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da8068d30 con 0x7f8da8108030 2024-02-26T04:48:07.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.010+0000 7f8da6ffd700 1 --2- 172.21.15.60:0/1703887704 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f8d94038ee0 0x7f8d9403b3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8d90006fd0 tx=0x7f8d90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:07.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.015+0000 7f8da4ff9700 1 -- 172.21.15.60:0/1703887704 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8d98017070 con 0x7f8da8108030 2024-02-26T04:48:07.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.217+0000 7f8dae0f0700 1 -- 172.21.15.60:0/1703887704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f8da810b760 con 0x7f8da8108030 2024-02-26T04:48:07.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.219+0000 7f8da4ff9700 1 -- 172.21.15.60:0/1703887704 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) v1 ==== 142+0+0 (secure 0 0 0) 0x7f8d98020020 con 0x7f8da8108030 2024-02-26T04:48:07.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.221+0000 7f8dae0f0700 1 -- 172.21.15.60:0/1703887704 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f8d94038ee0 msgr2=0x7f8d9403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:07.852 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.221+0000 7f8dae0f0700 1 --2- 172.21.15.60:0/1703887704 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f8d94038ee0 0x7f8d9403b3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8d90006fd0 tx=0x7f8d90006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:07.853 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.221+0000 7f8dae0f0700 1 -- 172.21.15.60:0/1703887704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da8108030 msgr2=0x7f8da81078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:07.853 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.221+0000 7f8dae0f0700 1 --2- 172.21.15.60:0/1703887704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da8108030 0x7f8da81078b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8d980042e0 tx=0x7f8d98003d00 comp rx=0 tx=0).stop 2024-02-26T04:48:07.853 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.221+0000 7f8dae0f0700 1 -- 172.21.15.60:0/1703887704 shutdown_connections 2024-02-26T04:48:07.853 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.221+0000 7f8dae0f0700 1 --2- 172.21.15.60:0/1703887704 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f8d94038ee0 0x7f8d9403b3a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:07.853 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.221+0000 7f8dae0f0700 1 --2- 172.21.15.60:0/1703887704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da8108030 0x7f8da81078b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:07.853 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.221+0000 7f8dae0f0700 1 -- 172.21.15.60:0/1703887704 >> 172.21.15.60:0/1703887704 conn(0x7f8da8101a00 msgr2=0x7f8da81024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:07.853 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.222+0000 7f8dae0f0700 1 -- 172.21.15.60:0/1703887704 shutdown_connections 2024-02-26T04:48:07.853 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:07.222+0000 7f8dae0f0700 1 -- 172.21.15.60:0/1703887704 wait complete. 2024-02-26T04:48:08.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:08 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1703887704' entity='client.admin' 2024-02-26T04:48:08.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:08 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' 2024-02-26T04:48:08.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:08 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' 2024-02-26T04:48:09.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.312+0000 7f592cff2700 1 Processor -- start 2024-02-26T04:48:09.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.312+0000 7f592cff2700 1 -- start start 2024-02-26T04:48:09.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.312+0000 7f592cff2700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59281075e0 0x7f5928107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:09.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.312+0000 7f592cff2700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5928107fd0 con 0x7f59281075e0 2024-02-26T04:48:09.609 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.312+0000 7f592659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59281075e0 0x7f5928107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:09.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.312+0000 7f592659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59281075e0 0x7f5928107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47128/0 (socket says 172.21.15.60:47128) 2024-02-26T04:48:09.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.312+0000 7f592659c700 1 -- 172.21.15.60:0/2233324713 learned_addr learned my addr 172.21.15.60:0/2233324713 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:09.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.313+0000 7f592659c700 1 -- 172.21.15.60:0/2233324713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5928108740 con 0x7f59281075e0 2024-02-26T04:48:09.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.313+0000 7f592659c700 1 --2- 172.21.15.60:0/2233324713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59281075e0 0x7f5928107a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f59180098a0 tx=0x7f5918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cfd31fd320337a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:09.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.313+0000 7f592559a700 1 -- 172.21.15.60:0/2233324713 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5918004160 con 0x7f59281075e0 2024-02-26T04:48:09.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.313+0000 7f592559a700 1 -- 172.21.15.60:0/2233324713 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59180042c0 con 0x7f59281075e0 2024-02-26T04:48:09.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.313+0000 7f592559a700 1 -- 172.21.15.60:0/2233324713 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5918011a80 con 0x7f59281075e0 2024-02-26T04:48:09.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.313+0000 7f592cff2700 1 -- 172.21.15.60:0/2233324713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59281075e0 msgr2=0x7f5928107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:09.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.313+0000 7f592cff2700 1 --2- 172.21.15.60:0/2233324713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59281075e0 0x7f5928107a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f59180098a0 tx=0x7f5918009bb0 comp rx=0 tx=0).stop 2024-02-26T04:48:09.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.314+0000 7f592cff2700 1 -- 172.21.15.60:0/2233324713 shutdown_connections 2024-02-26T04:48:09.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.314+0000 7f592cff2700 1 --2- 172.21.15.60:0/2233324713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59281075e0 0x7f5928107a00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:09.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.314+0000 7f592cff2700 1 -- 172.21.15.60:0/2233324713 >> 172.21.15.60:0/2233324713 conn(0x7f5928102c80 msgr2=0x7f59281050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:09.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.314+0000 7f592cff2700 1 -- 172.21.15.60:0/2233324713 shutdown_connections 2024-02-26T04:48:09.610 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.314+0000 7f592cff2700 1 -- 172.21.15.60:0/2233324713 wait complete. 2024-02-26T04:48:09.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.314+0000 7f592cff2700 1 Processor -- start 2024-02-26T04:48:09.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.314+0000 7f592cff2700 1 -- start start 2024-02-26T04:48:09.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.315+0000 7f592cff2700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59281075e0 0x7f592819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:09.611 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.315+0000 7f592cff2700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592819b880 con 0x7f59281075e0 2024-02-26T04:48:09.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.315+0000 7f592659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59281075e0 0x7f592819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:09.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.315+0000 7f592659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59281075e0 0x7f592819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47134/0 (socket says 172.21.15.60:47134) 2024-02-26T04:48:09.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.315+0000 7f592659c700 1 -- 172.21.15.60:0/3814879913 learned_addr learned my addr 172.21.15.60:0/3814879913 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:09.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.315+0000 7f592659c700 1 -- 172.21.15.60:0/3814879913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5918009550 con 0x7f59281075e0 2024-02-26T04:48:09.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.315+0000 7f592659c700 1 --2- 172.21.15.60:0/3814879913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59281075e0 0x7f592819b340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5918000c00 tx=0x7f5918011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:09.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.315+0000 7f59177fe700 1 -- 172.21.15.60:0/3814879913 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59180039e0 con 0x7f59281075e0 2024-02-26T04:48:09.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.315+0000 7f59177fe700 1 -- 172.21.15.60:0/3814879913 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5918003b40 con 0x7f59281075e0 2024-02-26T04:48:09.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.315+0000 7f59177fe700 1 -- 172.21.15.60:0/3814879913 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5918024820 con 0x7f59281075e0 2024-02-26T04:48:09.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.315+0000 7f592cff2700 1 -- 172.21.15.60:0/3814879913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592819ba80 con 0x7f59281075e0 2024-02-26T04:48:09.612 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.315+0000 7f592cff2700 1 -- 172.21.15.60:0/3814879913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592819bf20 con 0x7f59281075e0 2024-02-26T04:48:09.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.316+0000 7f59177fe700 1 -- 172.21.15.60:0/3814879913 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f5918024980 con 0x7f59281075e0 2024-02-26T04:48:09.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.316+0000 7f592cff2700 1 -- 172.21.15.60:0/3814879913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592810b970 con 0x7f59281075e0 2024-02-26T04:48:09.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.317+0000 7f59177fe700 1 --2- 172.21.15.60:0/3814879913 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f5910038ee0 0x7f591003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:09.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.317+0000 7f59177fe700 1 -- 172.21.15.60:0/3814879913 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f591804d1d0 con 0x7f59281075e0 2024-02-26T04:48:09.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.317+0000 7f5925d9b700 1 --2- 172.21.15.60:0/3814879913 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f5910038ee0 0x7f591003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:09.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.317+0000 7f5925d9b700 1 --2- 172.21.15.60:0/3814879913 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f5910038ee0 0x7f591003b3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f591c006fd0 tx=0x7f591c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:09.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.322+0000 7f59177fe700 1 -- 172.21.15.60:0/3814879913 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5918018530 con 0x7f59281075e0 2024-02-26T04:48:09.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.552+0000 7f592cff2700 1 -- 172.21.15.60:0/3814879913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f592819e800 con 0x7f59281075e0 2024-02-26T04:48:09.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.554+0000 7f59177fe700 1 -- 172.21.15.60:0/3814879913 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f5918018350 con 0x7f59281075e0 2024-02-26T04:48:09.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.556+0000 7f592cff2700 1 -- 172.21.15.60:0/3814879913 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f5910038ee0 msgr2=0x7f591003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:09.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.556+0000 7f592cff2700 1 --2- 172.21.15.60:0/3814879913 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f5910038ee0 0x7f591003b3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f591c006fd0 tx=0x7f591c006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:09.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.556+0000 7f592cff2700 1 -- 172.21.15.60:0/3814879913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59281075e0 msgr2=0x7f592819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:09.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.556+0000 7f592cff2700 1 --2- 172.21.15.60:0/3814879913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59281075e0 0x7f592819b340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5918000c00 tx=0x7f5918011b00 comp rx=0 tx=0).stop 2024-02-26T04:48:09.613 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.556+0000 7f592cff2700 1 -- 172.21.15.60:0/3814879913 shutdown_connections 2024-02-26T04:48:09.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.557+0000 7f592cff2700 1 --2- 172.21.15.60:0/3814879913 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f5910038ee0 0x7f591003b3a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:09.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.557+0000 7f592cff2700 1 --2- 172.21.15.60:0/3814879913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59281075e0 0x7f592819b340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:09.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.557+0000 7f592cff2700 1 -- 172.21.15.60:0/3814879913 >> 172.21.15.60:0/3814879913 conn(0x7f5928102c80 msgr2=0x7f5928103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:09.614 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.557+0000 7f592cff2700 1 -- 172.21.15.60:0/3814879913 shutdown_connections 2024-02-26T04:48:09.615 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:08.557+0000 7f592cff2700 1 -- 172.21.15.60:0/3814879913 wait complete. 2024-02-26T04:48:09.615 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling the dashboard module... 2024-02-26T04:48:09.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:09 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3814879913' entity='client.admin' 2024-02-26T04:48:11.440 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:11 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: ignoring --setuser ceph since I am not root 2024-02-26T04:48:11.440 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:11 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: ignoring --setgroup ceph since I am not root 2024-02-26T04:48:11.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:11 smithi060 ceph-mon[32382]: from='mgr.14120 172.21.15.60:0/4016364295' entity='mgr.x' 2024-02-26T04:48:11.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:11 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2106709826' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-26T04:48:11.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:11 smithi060 ceph-mon[32382]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:48:11.740 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.046+0000 7f31181e6700 1 Processor -- start 2024-02-26T04:48:11.740 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.047+0000 7f31181e6700 1 -- start start 2024-02-26T04:48:11.740 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.047+0000 7f31181e6700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f311007cde0 0x7f311007b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:11.741 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.047+0000 7f31181e6700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311007b820 con 0x7f311007cde0 2024-02-26T04:48:11.741 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.047+0000 7f3115f82700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f311007cde0 0x7f311007b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:11.741 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.047+0000 7f3115f82700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f311007cde0 0x7f311007b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57802/0 (socket says 172.21.15.60:57802) 2024-02-26T04:48:11.741 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.047+0000 7f3115f82700 1 -- 172.21.15.60:0/3534672246 learned_addr learned my addr 172.21.15.60:0/3534672246 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:11.741 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.047+0000 7f3115f82700 1 -- 172.21.15.60:0/3534672246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311007b960 con 0x7f311007cde0 2024-02-26T04:48:11.741 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.047+0000 7f3115f82700 1 --2- 172.21.15.60:0/3534672246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f311007cde0 0x7f311007b2e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f310c0098a0 tx=0x7f310c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f72d8ce1c2b84d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:11.741 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.047+0000 7f3114f80700 1 -- 172.21.15.60:0/3534672246 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f310c004160 con 0x7f311007cde0 2024-02-26T04:48:11.741 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.048+0000 7f3114f80700 1 -- 172.21.15.60:0/3534672246 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f310c0042c0 con 0x7f311007cde0 2024-02-26T04:48:11.741 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.048+0000 7f3114f80700 1 -- 172.21.15.60:0/3534672246 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f310c011a80 con 0x7f311007cde0 2024-02-26T04:48:11.741 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.048+0000 7f31181e6700 1 -- 172.21.15.60:0/3534672246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f311007cde0 msgr2=0x7f311007b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:11.741 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.048+0000 7f31181e6700 1 --2- 172.21.15.60:0/3534672246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f311007cde0 0x7f311007b2e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f310c0098a0 tx=0x7f310c009bb0 comp rx=0 tx=0).stop 2024-02-26T04:48:11.741 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.048+0000 7f31181e6700 1 -- 172.21.15.60:0/3534672246 shutdown_connections 2024-02-26T04:48:11.741 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.048+0000 7f31181e6700 1 --2- 172.21.15.60:0/3534672246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f311007cde0 0x7f311007b2e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:11.742 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.048+0000 7f31181e6700 1 -- 172.21.15.60:0/3534672246 >> 172.21.15.60:0/3534672246 conn(0x7f3110102be0 msgr2=0x7f3110105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:11.742 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.048+0000 7f31181e6700 1 -- 172.21.15.60:0/3534672246 shutdown_connections 2024-02-26T04:48:11.742 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.048+0000 7f31181e6700 1 -- 172.21.15.60:0/3534672246 wait complete. 2024-02-26T04:48:11.742 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.049+0000 7f31181e6700 1 Processor -- start 2024-02-26T04:48:11.742 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.049+0000 7f31181e6700 1 -- start start 2024-02-26T04:48:11.742 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.049+0000 7f31181e6700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f311007cde0 0x7f311019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:11.742 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.049+0000 7f31181e6700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311019d9c0 con 0x7f311007cde0 2024-02-26T04:48:11.742 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.049+0000 7f3115f82700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f311007cde0 0x7f311019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:11.742 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.049+0000 7f3115f82700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f311007cde0 0x7f311019d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57810/0 (socket says 172.21.15.60:57810) 2024-02-26T04:48:11.742 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.049+0000 7f3115f82700 1 -- 172.21.15.60:0/2106709826 learned_addr learned my addr 172.21.15.60:0/2106709826 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:11.742 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.049+0000 7f3115f82700 1 -- 172.21.15.60:0/2106709826 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c009550 con 0x7f311007cde0 2024-02-26T04:48:11.742 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.050+0000 7f3115f82700 1 --2- 172.21.15.60:0/2106709826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f311007cde0 0x7f311019d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f310c011740 tx=0x7f310c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:11.742 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.050+0000 7f3106ffd700 1 -- 172.21.15.60:0/2106709826 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f310c003b20 con 0x7f311007cde0 2024-02-26T04:48:11.742 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.050+0000 7f3106ffd700 1 -- 172.21.15.60:0/2106709826 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f310c01a910 con 0x7f311007cde0 2024-02-26T04:48:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.050+0000 7f31181e6700 1 -- 172.21.15.60:0/2106709826 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311019dbc0 con 0x7f311007cde0 2024-02-26T04:48:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.050+0000 7f3106ffd700 1 -- 172.21.15.60:0/2106709826 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f310c0248c0 con 0x7f311007cde0 2024-02-26T04:48:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.050+0000 7f31181e6700 1 -- 172.21.15.60:0/2106709826 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311019e060 con 0x7f311007cde0 2024-02-26T04:48:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.051+0000 7f3106ffd700 1 -- 172.21.15.60:0/2106709826 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f310c02d3f0 con 0x7f311007cde0 2024-02-26T04:48:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.051+0000 7f31181e6700 1 -- 172.21.15.60:0/2106709826 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311010d980 con 0x7f311007cde0 2024-02-26T04:48:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.051+0000 7f3106ffd700 1 --2- 172.21.15.60:0/2106709826 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f30fc038ee0 0x7f30fc03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.051+0000 7f3106ffd700 1 -- 172.21.15.60:0/2106709826 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f310c04d3f0 con 0x7f311007cde0 2024-02-26T04:48:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.051+0000 7f3115781700 1 --2- 172.21.15.60:0/2106709826 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f30fc038ee0 0x7f30fc03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.051+0000 7f3115781700 1 --2- 172.21.15.60:0/2106709826 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f30fc038ee0 0x7f30fc03b3a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3100006fd0 tx=0x7f3100006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.056+0000 7f3106ffd700 1 -- 172.21.15.60:0/2106709826 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f310c01adf0 con 0x7f311007cde0 2024-02-26T04:48:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:10.336+0000 7f31181e6700 1 -- 172.21.15.60:0/2106709826 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f31101a0940 con 0x7f311007cde0 2024-02-26T04:48:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:11.130+0000 7f3106ffd700 1 -- 172.21.15.60:0/2106709826 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f310c04cd60 con 0x7f311007cde0 2024-02-26T04:48:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:11.132+0000 7f3106ffd700 1 -- 172.21.15.60:0/2106709826 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f310c02d6a0 con 0x7f311007cde0 2024-02-26T04:48:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:11.134+0000 7f31181e6700 1 -- 172.21.15.60:0/2106709826 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f30fc038ee0 msgr2=0x7f30fc03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:11.744 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:11.134+0000 7f31181e6700 1 --2- 172.21.15.60:0/2106709826 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f30fc038ee0 0x7f30fc03b3a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3100006fd0 tx=0x7f3100006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:11.744 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:11.134+0000 7f31181e6700 1 -- 172.21.15.60:0/2106709826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f311007cde0 msgr2=0x7f311019d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:11.744 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:11.134+0000 7f31181e6700 1 --2- 172.21.15.60:0/2106709826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f311007cde0 0x7f311019d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f310c011740 tx=0x7f310c011b00 comp rx=0 tx=0).stop 2024-02-26T04:48:11.744 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:11.134+0000 7f31181e6700 1 -- 172.21.15.60:0/2106709826 shutdown_connections 2024-02-26T04:48:11.744 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:11.134+0000 7f31181e6700 1 --2- 172.21.15.60:0/2106709826 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f30fc038ee0 0x7f30fc03b3a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:11.744 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:11.134+0000 7f31181e6700 1 --2- 172.21.15.60:0/2106709826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f311007cde0 0x7f311019d480 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:11.744 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:11.134+0000 7f31181e6700 1 -- 172.21.15.60:0/2106709826 >> 172.21.15.60:0/2106709826 conn(0x7f3110102be0 msgr2=0x7f31101036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:11.744 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:11.135+0000 7f31181e6700 1 -- 172.21.15.60:0/2106709826 shutdown_connections 2024-02-26T04:48:11.744 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:11.135+0000 7f31181e6700 1 -- 172.21.15.60:0/2106709826 wait complete. 2024-02-26T04:48:11.979 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:11 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:11.720+0000 7fc8b04e6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T04:48:11.979 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:11 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:11.826+0000 7fc8b04e6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T04:48:12.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:12 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2106709826' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-26T04:48:12.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:12 smithi060 ceph-mon[32382]: mgrmap e9: x(active, since 20s) 2024-02-26T04:48:12.729 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:12 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:12.500+0000 7fc8b04e6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T04:48:12.729 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:12 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:12.596+0000 7fc8b04e6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T04:48:13.056 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:12 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:12.742+0000 7fc8b04e6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T04:48:13.056 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:12 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:12.841+0000 7fc8b04e6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T04:48:13.056 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:12 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:12.926+0000 7fc8b04e6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T04:48:13.066 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T04:48:13.066 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-26T04:48:13.067 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-26T04:48:13.067 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-26T04:48:13.067 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-26T04:48:13.067 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T04:48:13.067 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.167+0000 7f6049be0700 1 Processor -- start 2024-02-26T04:48:13.067 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.167+0000 7f6049be0700 1 -- start start 2024-02-26T04:48:13.067 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.167+0000 7f6049be0700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6044106450 0x7f6044106870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:13.067 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.167+0000 7f6049be0700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6044106e40 con 0x7f6044106450 2024-02-26T04:48:13.067 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.168+0000 7f60437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6044106450 0x7f6044106870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:13.067 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.168+0000 7f60437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6044106450 0x7f6044106870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57838/0 (socket says 172.21.15.60:57838) 2024-02-26T04:48:13.067 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.168+0000 7f60437fe700 1 -- 172.21.15.60:0/1650875078 learned_addr learned my addr 172.21.15.60:0/1650875078 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:13.067 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.168+0000 7f60437fe700 1 -- 172.21.15.60:0/1650875078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6044107650 con 0x7f6044106450 2024-02-26T04:48:13.068 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.168+0000 7f60437fe700 1 --2- 172.21.15.60:0/1650875078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6044106450 0x7f6044106870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f60380098a0 tx=0x7f6038009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200a523b8b10dbdd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:13.068 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.168+0000 7f60427fc700 1 -- 172.21.15.60:0/1650875078 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6038004160 con 0x7f6044106450 2024-02-26T04:48:13.068 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.168+0000 7f60427fc700 1 -- 172.21.15.60:0/1650875078 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60380042c0 con 0x7f6044106450 2024-02-26T04:48:13.068 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.168+0000 7f60427fc700 1 -- 172.21.15.60:0/1650875078 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6038011a80 con 0x7f6044106450 2024-02-26T04:48:13.068 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.169+0000 7f6049be0700 1 -- 172.21.15.60:0/1650875078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6044106450 msgr2=0x7f6044106870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:13.068 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.169+0000 7f6049be0700 1 --2- 172.21.15.60:0/1650875078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6044106450 0x7f6044106870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f60380098a0 tx=0x7f6038009bb0 comp rx=0 tx=0).stop 2024-02-26T04:48:13.068 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.169+0000 7f6049be0700 1 -- 172.21.15.60:0/1650875078 shutdown_connections 2024-02-26T04:48:13.068 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.169+0000 7f6049be0700 1 --2- 172.21.15.60:0/1650875078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6044106450 0x7f6044106870 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:13.068 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.169+0000 7f6049be0700 1 -- 172.21.15.60:0/1650875078 >> 172.21.15.60:0/1650875078 conn(0x7f6044101a10 msgr2=0x7f6044103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:13.068 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.169+0000 7f6049be0700 1 -- 172.21.15.60:0/1650875078 shutdown_connections 2024-02-26T04:48:13.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.169+0000 7f6049be0700 1 -- 172.21.15.60:0/1650875078 wait complete. 2024-02-26T04:48:13.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.170+0000 7f6049be0700 1 Processor -- start 2024-02-26T04:48:13.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.170+0000 7f6049be0700 1 -- start start 2024-02-26T04:48:13.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.170+0000 7f6049be0700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6044106450 0x7f604419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:13.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.170+0000 7f6049be0700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604419db40 con 0x7f6044106450 2024-02-26T04:48:13.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.170+0000 7f60437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6044106450 0x7f604419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:13.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.170+0000 7f60437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6044106450 0x7f604419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57844/0 (socket says 172.21.15.60:57844) 2024-02-26T04:48:13.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.170+0000 7f60437fe700 1 -- 172.21.15.60:0/1513396086 learned_addr learned my addr 172.21.15.60:0/1513396086 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:13.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.170+0000 7f60437fe700 1 -- 172.21.15.60:0/1513396086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6038009550 con 0x7f6044106450 2024-02-26T04:48:13.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.171+0000 7f60437fe700 1 --2- 172.21.15.60:0/1513396086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6044106450 0x7f604419d600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6038000c00 tx=0x7f6038011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:13.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.171+0000 7f6040ff9700 1 -- 172.21.15.60:0/1513396086 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60380039e0 con 0x7f6044106450 2024-02-26T04:48:13.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.171+0000 7f6049be0700 1 -- 172.21.15.60:0/1513396086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604419dd40 con 0x7f6044106450 2024-02-26T04:48:13.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.171+0000 7f6040ff9700 1 -- 172.21.15.60:0/1513396086 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6038003b40 con 0x7f6044106450 2024-02-26T04:48:13.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.171+0000 7f6040ff9700 1 -- 172.21.15.60:0/1513396086 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6038024820 con 0x7f6044106450 2024-02-26T04:48:13.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.171+0000 7f6049be0700 1 -- 172.21.15.60:0/1513396086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604419e1e0 con 0x7f6044106450 2024-02-26T04:48:13.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.172+0000 7f6049be0700 1 -- 172.21.15.60:0/1513396086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60441978f0 con 0x7f6044106450 2024-02-26T04:48:13.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.172+0000 7f6040ff9700 1 -- 172.21.15.60:0/1513396086 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f6038024980 con 0x7f6044106450 2024-02-26T04:48:13.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.172+0000 7f6040ff9700 1 --2- 172.21.15.60:0/1513396086 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f602c038f30 0x7f602c03b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:13.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.172+0000 7f6040ff9700 1 -- 172.21.15.60:0/1513396086 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f603804d070 con 0x7f6044106450 2024-02-26T04:48:13.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.172+0000 7f6042ffd700 1 -- 172.21.15.60:0/1513396086 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f602c038f30 msgr2=0x7f602c03b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/3694881479 2024-02-26T04:48:13.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.172+0000 7f6042ffd700 1 --2- 172.21.15.60:0/1513396086 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f602c038f30 0x7f602c03b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T04:48:13.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.181+0000 7f6040ff9700 1 -- 172.21.15.60:0/1513396086 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6038018dc0 con 0x7f6044106450 2024-02-26T04:48:13.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.373+0000 7f6042ffd700 1 -- 172.21.15.60:0/1513396086 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f602c038f30 msgr2=0x7f602c03b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/3694881479 2024-02-26T04:48:13.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.373+0000 7f6042ffd700 1 --2- 172.21.15.60:0/1513396086 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f602c038f30 0x7f602c03b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T04:48:13.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.438+0000 7f6049be0700 1 -- 172.21.15.60:0/1513396086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f604402d940 con 0x7f6044106450 2024-02-26T04:48:13.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.439+0000 7f6040ff9700 1 -- 172.21.15.60:0/1513396086 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f603804c9e0 con 0x7f6044106450 2024-02-26T04:48:13.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.441+0000 7f6049be0700 1 -- 172.21.15.60:0/1513396086 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f602c038f30 msgr2=0x7f602c03b3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T04:48:13.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.441+0000 7f6049be0700 1 --2- 172.21.15.60:0/1513396086 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f602c038f30 0x7f602c03b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:13.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.441+0000 7f6049be0700 1 -- 172.21.15.60:0/1513396086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6044106450 msgr2=0x7f604419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:13.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.441+0000 7f6049be0700 1 --2- 172.21.15.60:0/1513396086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6044106450 0x7f604419d600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6038000c00 tx=0x7f6038011b00 comp rx=0 tx=0).stop 2024-02-26T04:48:13.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.441+0000 7f6049be0700 1 -- 172.21.15.60:0/1513396086 shutdown_connections 2024-02-26T04:48:13.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.441+0000 7f6049be0700 1 --2- 172.21.15.60:0/1513396086 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f602c038f30 0x7f602c03b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:13.073 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.441+0000 7f6049be0700 1 --2- 172.21.15.60:0/1513396086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6044106450 0x7f604419d600 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:13.074 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.441+0000 7f6049be0700 1 -- 172.21.15.60:0/1513396086 >> 172.21.15.60:0/1513396086 conn(0x7f6044101a10 msgr2=0x7f6044102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:13.074 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.441+0000 7f6049be0700 1 -- 172.21.15.60:0/1513396086 shutdown_connections 2024-02-26T04:48:13.074 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:12.441+0000 7f6049be0700 1 -- 172.21.15.60:0/1513396086 wait complete. 2024-02-26T04:48:13.074 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for the mgr to restart... 2024-02-26T04:48:13.074 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr epoch 9... 2024-02-26T04:48:13.307 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:13 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1513396086' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-26T04:48:13.307 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:13 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:13.202+0000 7fc8b04e6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T04:48:13.692 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:13 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:13.319+0000 7fc8b04e6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T04:48:13.692 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:13 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:13.407+0000 7fc8b04e6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T04:48:13.692 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:13 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:13.508+0000 7fc8b04e6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T04:48:13.692 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:13 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:13.690+0000 7fc8b04e6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T04:48:14.979 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:14 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:14.683+0000 7fc8b04e6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T04:48:14.979 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:14 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:14.818+0000 7fc8b04e6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T04:48:15.462 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:15 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:15.159+0000 7fc8b04e6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T04:48:15.462 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:15 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:15.254+0000 7fc8b04e6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T04:48:15.462 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:15 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:15.459+0000 7fc8b04e6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T04:48:15.733 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:15 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:15.652+0000 7fc8b04e6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T04:48:16.216 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:15 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:15.951+0000 7fc8b04e6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T04:48:16.216 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:16 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:16.213+0000 7fc8b04e6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T04:48:16.479 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:16 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:16.459+0000 7fc8b04e6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T04:48:17.729 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:17 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:17.297+0000 7fc8b04e6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T04:48:18.212 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:17 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:17.883+0000 7fc8b04e6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T04:48:18.212 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:17 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:17.986+0000 7fc8b04e6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T04:48:18.212 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:48:18 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:48:18.209+0000 7fc8b04e6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T04:48:18.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:18 smithi060 ceph-mon[32382]: Active manager daemon x restarted 2024-02-26T04:48:18.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:18 smithi060 ceph-mon[32382]: Activating manager daemon x 2024-02-26T04:48:18.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:18 smithi060 ceph-mon[32382]: osdmap e3: 0 total, 0 up, 0 in 2024-02-26T04:48:18.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:18 smithi060 ceph-mon[32382]: mgrmap e10: x(active, starting, since 0.00307127s) 2024-02-26T04:48:18.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:18 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:48:18.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:18 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-26T04:48:18.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:18 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-26T04:48:18.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:18 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-26T04:48:18.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:18 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-26T04:48:18.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:18 smithi060 ceph-mon[32382]: Manager daemon x is now available 2024-02-26T04:48:18.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:18 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:48:18.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:18 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:48:19.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-26T04:48:19.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-26T04:48:19.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:19 smithi060 ceph-mon[32382]: [26/Feb/2024:04:48:18] ENGINE Bus STARTING 2024-02-26T04:48:19.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:19 smithi060 ceph-mon[32382]: [26/Feb/2024:04:48:18] ENGINE Serving on https://172.21.15.60:7150 2024-02-26T04:48:19.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:19 smithi060 ceph-mon[32382]: [26/Feb/2024:04:48:19] ENGINE Serving on http://172.21.15.60:8765 2024-02-26T04:48:19.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:19 smithi060 ceph-mon[32382]: [26/Feb/2024:04:48:19] ENGINE Bus STARTED 2024-02-26T04:48:19.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:19 smithi060 ceph-mon[32382]: mgrmap e11: x(active, since 1.00599s) 2024-02-26T04:48:20.051 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-02-26T04:48:20.051 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-26T04:48:20.051 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-26T04:48:20.051 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-02-26T04:48:20.051 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.525+0000 7f67d7e74700 1 Processor -- start 2024-02-26T04:48:20.051 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.525+0000 7f67d7e74700 1 -- start start 2024-02-26T04:48:20.051 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.525+0000 7f67d7e74700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d0108070 0x7f67d010a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:20.051 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.525+0000 7f67d7e74700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d007ae70 con 0x7f67d0108070 2024-02-26T04:48:20.051 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.525+0000 7f67d5c10700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d0108070 0x7f67d010a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:20.051 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.525+0000 7f67d5c10700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d0108070 0x7f67d010a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57848/0 (socket says 172.21.15.60:57848) 2024-02-26T04:48:20.052 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.525+0000 7f67d5c10700 1 -- 172.21.15.60:0/3583863792 learned_addr learned my addr 172.21.15.60:0/3583863792 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:20.052 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.525+0000 7f67d5c10700 1 -- 172.21.15.60:0/3583863792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d010a9a0 con 0x7f67d0108070 2024-02-26T04:48:20.052 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.526+0000 7f67d5c10700 1 --2- 172.21.15.60:0/3583863792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d0108070 0x7f67d010a460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0098a0 tx=0x7f67cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1c7ed135254f54 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:20.052 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.526+0000 7f67d4c0e700 1 -- 172.21.15.60:0/3583863792 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67cc004160 con 0x7f67d0108070 2024-02-26T04:48:20.052 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.526+0000 7f67d4c0e700 1 -- 172.21.15.60:0/3583863792 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67cc0042c0 con 0x7f67d0108070 2024-02-26T04:48:20.052 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.526+0000 7f67d4c0e700 1 -- 172.21.15.60:0/3583863792 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67cc011a80 con 0x7f67d0108070 2024-02-26T04:48:20.052 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.526+0000 7f67d7e74700 1 -- 172.21.15.60:0/3583863792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d0108070 msgr2=0x7f67d010a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:20.052 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.526+0000 7f67d7e74700 1 --2- 172.21.15.60:0/3583863792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d0108070 0x7f67d010a460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0098a0 tx=0x7f67cc009bb0 comp rx=0 tx=0).stop 2024-02-26T04:48:20.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.526+0000 7f67d7e74700 1 -- 172.21.15.60:0/3583863792 shutdown_connections 2024-02-26T04:48:20.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.526+0000 7f67d7e74700 1 --2- 172.21.15.60:0/3583863792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d0108070 0x7f67d010a460 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:20.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.526+0000 7f67d7e74700 1 -- 172.21.15.60:0/3583863792 >> 172.21.15.60:0/3583863792 conn(0x7f67d0101a80 msgr2=0x7f67d0103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:20.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.527+0000 7f67d7e74700 1 -- 172.21.15.60:0/3583863792 shutdown_connections 2024-02-26T04:48:20.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.527+0000 7f67d7e74700 1 -- 172.21.15.60:0/3583863792 wait complete. 2024-02-26T04:48:20.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.527+0000 7f67d7e74700 1 Processor -- start 2024-02-26T04:48:20.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.527+0000 7f67d7e74700 1 -- start start 2024-02-26T04:48:20.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.527+0000 7f67d7e74700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d0108070 0x7f67d019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:20.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.528+0000 7f67d7e74700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d019b850 con 0x7f67d0108070 2024-02-26T04:48:20.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.528+0000 7f67d5c10700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d0108070 0x7f67d019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:20.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.528+0000 7f67d5c10700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d0108070 0x7f67d019b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57850/0 (socket says 172.21.15.60:57850) 2024-02-26T04:48:20.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.528+0000 7f67d5c10700 1 -- 172.21.15.60:0/1457709831 learned_addr learned my addr 172.21.15.60:0/1457709831 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:20.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.528+0000 7f67d5c10700 1 -- 172.21.15.60:0/1457709831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc009550 con 0x7f67d0108070 2024-02-26T04:48:20.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.528+0000 7f67d5c10700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d0108070 0x7f67d019b310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f67cc003f20 tx=0x7f67cc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:20.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.528+0000 7f67c6ffd700 1 -- 172.21.15.60:0/1457709831 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67cc0039f0 con 0x7f67d0108070 2024-02-26T04:48:20.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.528+0000 7f67c6ffd700 1 -- 172.21.15.60:0/1457709831 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67cc003b50 con 0x7f67d0108070 2024-02-26T04:48:20.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.528+0000 7f67d7e74700 1 -- 172.21.15.60:0/1457709831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d019ba50 con 0x7f67d0108070 2024-02-26T04:48:20.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.528+0000 7f67c6ffd700 1 -- 172.21.15.60:0/1457709831 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67cc01a6d0 con 0x7f67d0108070 2024-02-26T04:48:20.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.528+0000 7f67d7e74700 1 -- 172.21.15.60:0/1457709831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d019bef0 con 0x7f67d0108070 2024-02-26T04:48:20.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.529+0000 7f67c6ffd700 1 -- 172.21.15.60:0/1457709831 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f67cc01a9c0 con 0x7f67d0108070 2024-02-26T04:48:20.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.529+0000 7f67c6ffd700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f67bc038f40 0x7f67bc03b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:20.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.529+0000 7f67c6ffd700 1 -- 172.21.15.60:0/1457709831 --> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67bc03bb10 con 0x7f67bc038f40 2024-02-26T04:48:20.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.529+0000 7f67c6ffd700 1 -- 172.21.15.60:0/1457709831 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f67cc04d2a0 con 0x7f67d0108070 2024-02-26T04:48:20.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.530+0000 7f67d540f700 1 -- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f67bc038f40 msgr2=0x7f67bc03b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/3694881479 2024-02-26T04:48:20.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.530+0000 7f67d540f700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f67bc038f40 0x7f67bc03b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T04:48:20.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.731+0000 7f67d540f700 1 -- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f67bc038f40 msgr2=0x7f67bc03b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/3694881479 2024-02-26T04:48:20.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:13.731+0000 7f67d540f700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f67bc038f40 0x7f67bc03b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-26T04:48:20.055 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:14.132+0000 7f67d540f700 1 -- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f67bc038f40 msgr2=0x7f67bc03b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/3694881479 2024-02-26T04:48:20.055 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:14.132+0000 7f67d540f700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f67bc038f40 0x7f67bc03b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-26T04:48:20.055 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:14.933+0000 7f67d540f700 1 -- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f67bc038f40 msgr2=0x7f67bc03b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/3694881479 2024-02-26T04:48:20.055 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:14.933+0000 7f67d540f700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f67bc038f40 0x7f67bc03b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-26T04:48:20.055 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:16.534+0000 7f67d540f700 1 -- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f67bc038f40 msgr2=0x7f67bc03b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/3694881479 2024-02-26T04:48:20.055 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:16.534+0000 7f67d540f700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f67bc038f40 0x7f67bc03b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-26T04:48:20.055 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:18.225+0000 7f67c6ffd700 1 -- 172.21.15.60:0/1457709831 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mgrmap(e 10) v1 ==== 45712+0+0 (secure 0 0 0) 0x7f67cc02ad60 con 0x7f67d0108070 2024-02-26T04:48:20.055 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:18.225+0000 7f67c6ffd700 1 -- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f67bc038f40 msgr2=0x7f67bc03b400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T04:48:20.055 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:18.225+0000 7f67c6ffd700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f67bc038f40 0x7f67bc03b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:20.055 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.226+0000 7f67c6ffd700 1 -- 172.21.15.60:0/1457709831 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 11) v1 ==== 45839+0+0 (secure 0 0 0) 0x7f67cc04eca0 con 0x7f67d0108070 2024-02-26T04:48:20.055 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.227+0000 7f67c6ffd700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f67bc03c9f0 0x7f67bc03ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:20.055 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.227+0000 7f67c6ffd700 1 -- 172.21.15.60:0/1457709831 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67bc03bb10 con 0x7f67bc03c9f0 2024-02-26T04:48:20.055 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.228+0000 7f67d540f700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f67bc03c9f0 0x7f67bc03ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:20.056 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.228+0000 7f67d540f700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f67bc03c9f0 0x7f67bc03ede0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67c0003a10 tx=0x7f67c0009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:20.056 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.229+0000 7f67c6ffd700 1 -- 172.21.15.60:0/1457709831 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f67bc03bb10 con 0x7f67bc03c9f0 2024-02-26T04:48:20.056 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.242+0000 7f67d7e74700 1 -- 172.21.15.60:0/1457709831 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f67d010b7e0 con 0x7f67bc03c9f0 2024-02-26T04:48:20.056 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.242+0000 7f67c6ffd700 1 -- 172.21.15.60:0/1457709831 <== mgr.14152 v2:172.21.15.60:6800/3458567419 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f67d010b7e0 con 0x7f67bc03c9f0 2024-02-26T04:48:20.056 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.243+0000 7f67d7e74700 1 -- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f67bc03c9f0 msgr2=0x7f67bc03ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:20.056 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.243+0000 7f67d7e74700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f67bc03c9f0 0x7f67bc03ede0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67c0003a10 tx=0x7f67c0009270 comp rx=0 tx=0).stop 2024-02-26T04:48:20.056 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.243+0000 7f67d7e74700 1 -- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d0108070 msgr2=0x7f67d019b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:20.056 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.243+0000 7f67d7e74700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d0108070 0x7f67d019b310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f67cc003f20 tx=0x7f67cc011b00 comp rx=0 tx=0).stop 2024-02-26T04:48:20.056 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.243+0000 7f67d7e74700 1 -- 172.21.15.60:0/1457709831 shutdown_connections 2024-02-26T04:48:20.056 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.243+0000 7f67d7e74700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f67bc03c9f0 0x7f67bc03ede0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:20.056 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.243+0000 7f67d7e74700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d0108070 0x7f67d019b310 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:20.056 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.243+0000 7f67d7e74700 1 --2- 172.21.15.60:0/1457709831 >> [v2:172.21.15.60:6800/3694881479,v1:172.21.15.60:6801/3694881479] conn(0x7f67bc038f40 0x7f67bc03b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:20.056 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.243+0000 7f67d7e74700 1 -- 172.21.15.60:0/1457709831 >> 172.21.15.60:0/1457709831 conn(0x7f67d0101a80 msgr2=0x7f67d0102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:20.057 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.243+0000 7f67d7e74700 1 -- 172.21.15.60:0/1457709831 shutdown_connections 2024-02-26T04:48:20.057 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:19.243+0000 7f67d7e74700 1 -- 172.21.15.60:0/1457709831 wait complete. 2024-02-26T04:48:20.057 INFO:teuthology.orchestra.run.smithi060.stdout:mgr epoch 9 is available 2024-02-26T04:48:20.057 INFO:teuthology.orchestra.run.smithi060.stdout:Generating a dashboard self-signed certificate... 2024-02-26T04:48:20.304 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:20 smithi060 ceph-mon[32382]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-26T04:48:20.304 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:20 smithi060 ceph-mon[32382]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-26T04:48:21.481 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-26T04:48:21.481 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.508+0000 7fe8e876a700 1 Processor -- start 2024-02-26T04:48:21.481 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.508+0000 7fe8e876a700 1 -- start start 2024-02-26T04:48:21.481 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.509+0000 7fe8e876a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e0106450 0x7fe8e0106870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:21.481 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.509+0000 7fe8e876a700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e0106e40 con 0x7fe8e0106450 2024-02-26T04:48:21.481 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.509+0000 7fe8e6506700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e0106450 0x7fe8e0106870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:21.481 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.509+0000 7fe8e6506700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e0106450 0x7fe8e0106870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59538/0 (socket says 172.21.15.60:59538) 2024-02-26T04:48:21.481 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.509+0000 7fe8e6506700 1 -- 172.21.15.60:0/4067699606 learned_addr learned my addr 172.21.15.60:0/4067699606 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:21.481 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.509+0000 7fe8e6506700 1 -- 172.21.15.60:0/4067699606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e0107650 con 0x7fe8e0106450 2024-02-26T04:48:21.482 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.509+0000 7fe8e6506700 1 --2- 172.21.15.60:0/4067699606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e0106450 0x7fe8e0106870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00098a0 tx=0x7fe8d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bae586b77b7873 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:21.482 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.510+0000 7fe8e5504700 1 -- 172.21.15.60:0/4067699606 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8d0004160 con 0x7fe8e0106450 2024-02-26T04:48:21.482 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.510+0000 7fe8e5504700 1 -- 172.21.15.60:0/4067699606 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8d00042c0 con 0x7fe8e0106450 2024-02-26T04:48:21.482 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.510+0000 7fe8e5504700 1 -- 172.21.15.60:0/4067699606 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8d0011a80 con 0x7fe8e0106450 2024-02-26T04:48:21.482 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.510+0000 7fe8e876a700 1 -- 172.21.15.60:0/4067699606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e0106450 msgr2=0x7fe8e0106870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:21.482 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.510+0000 7fe8e876a700 1 --2- 172.21.15.60:0/4067699606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e0106450 0x7fe8e0106870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00098a0 tx=0x7fe8d0009bb0 comp rx=0 tx=0).stop 2024-02-26T04:48:21.482 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.510+0000 7fe8e876a700 1 -- 172.21.15.60:0/4067699606 shutdown_connections 2024-02-26T04:48:21.482 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.510+0000 7fe8e876a700 1 --2- 172.21.15.60:0/4067699606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e0106450 0x7fe8e0106870 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:21.482 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.510+0000 7fe8e876a700 1 -- 172.21.15.60:0/4067699606 >> 172.21.15.60:0/4067699606 conn(0x7fe8e0101a10 msgr2=0x7fe8e0103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:21.482 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.510+0000 7fe8e876a700 1 -- 172.21.15.60:0/4067699606 shutdown_connections 2024-02-26T04:48:21.482 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.510+0000 7fe8e876a700 1 -- 172.21.15.60:0/4067699606 wait complete. 2024-02-26T04:48:21.482 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.511+0000 7fe8e876a700 1 Processor -- start 2024-02-26T04:48:21.483 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.511+0000 7fe8e876a700 1 -- start start 2024-02-26T04:48:21.484 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.511+0000 7fe8e876a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e0106450 0x7fe8e01915c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:21.484 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.511+0000 7fe8e876a700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e0191b00 con 0x7fe8e0106450 2024-02-26T04:48:21.484 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.511+0000 7fe8e6506700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e0106450 0x7fe8e01915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:21.484 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.511+0000 7fe8e6506700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e0106450 0x7fe8e01915c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59554/0 (socket says 172.21.15.60:59554) 2024-02-26T04:48:21.484 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.511+0000 7fe8e6506700 1 -- 172.21.15.60:0/2399649226 learned_addr learned my addr 172.21.15.60:0/2399649226 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:21.484 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.511+0000 7fe8e6506700 1 -- 172.21.15.60:0/2399649226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d0009550 con 0x7fe8e0106450 2024-02-26T04:48:21.484 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.511+0000 7fe8e6506700 1 --2- 172.21.15.60:0/2399649226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e0106450 0x7fe8e01915c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0000c00 tx=0x7fe8d0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:21.485 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.512+0000 7fe8d77fe700 1 -- 172.21.15.60:0/2399649226 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8d00039e0 con 0x7fe8e0106450 2024-02-26T04:48:21.485 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.512+0000 7fe8e876a700 1 -- 172.21.15.60:0/2399649226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e018fcf0 con 0x7fe8e0106450 2024-02-26T04:48:21.485 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.512+0000 7fe8d77fe700 1 -- 172.21.15.60:0/2399649226 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8d0003b40 con 0x7fe8e0106450 2024-02-26T04:48:21.485 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.512+0000 7fe8d77fe700 1 -- 172.21.15.60:0/2399649226 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8d0024820 con 0x7fe8e0106450 2024-02-26T04:48:21.485 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.512+0000 7fe8e876a700 1 -- 172.21.15.60:0/2399649226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e0190190 con 0x7fe8e0106450 2024-02-26T04:48:21.485 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.512+0000 7fe8d77fe700 1 -- 172.21.15.60:0/2399649226 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fe8d001ab80 con 0x7fe8e0106450 2024-02-26T04:48:21.485 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.513+0000 7fe8e876a700 1 -- 172.21.15.60:0/2399649226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e0068d30 con 0x7fe8e0106450 2024-02-26T04:48:21.485 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.513+0000 7fe8d77fe700 1 --2- 172.21.15.60:0/2399649226 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe8cc038f30 0x7fe8cc03b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:21.485 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.513+0000 7fe8d77fe700 1 -- 172.21.15.60:0/2399649226 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe8d004d190 con 0x7fe8e0106450 2024-02-26T04:48:21.485 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.513+0000 7fe8e5d05700 1 --2- 172.21.15.60:0/2399649226 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe8cc038f30 0x7fe8cc03b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:21.485 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.513+0000 7fe8e5d05700 1 --2- 172.21.15.60:0/2399649226 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe8cc038f30 0x7fe8cc03b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc006fd0 tx=0x7fe8dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:21.485 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.518+0000 7fe8d77fe700 1 -- 172.21.15.60:0/2399649226 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe8d001a950 con 0x7fe8e0106450 2024-02-26T04:48:21.485 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.730+0000 7fe8e876a700 1 -- 172.21.15.60:0/2399649226 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8e010a850 con 0x7fe8cc038f30 2024-02-26T04:48:21.485 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.954+0000 7fe8d77fe700 1 -- 172.21.15.60:0/2399649226 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fe8e010a850 con 0x7fe8cc038f30 2024-02-26T04:48:21.486 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.957+0000 7fe8e876a700 1 -- 172.21.15.60:0/2399649226 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe8cc038f30 msgr2=0x7fe8cc03b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:21.486 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.957+0000 7fe8e876a700 1 --2- 172.21.15.60:0/2399649226 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe8cc038f30 0x7fe8cc03b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc006fd0 tx=0x7fe8dc006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:21.486 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.957+0000 7fe8e876a700 1 -- 172.21.15.60:0/2399649226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e0106450 msgr2=0x7fe8e01915c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:21.486 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.957+0000 7fe8e876a700 1 --2- 172.21.15.60:0/2399649226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e0106450 0x7fe8e01915c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0000c00 tx=0x7fe8d0011b00 comp rx=0 tx=0).stop 2024-02-26T04:48:21.486 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.957+0000 7fe8e876a700 1 -- 172.21.15.60:0/2399649226 shutdown_connections 2024-02-26T04:48:21.486 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.957+0000 7fe8e876a700 1 --2- 172.21.15.60:0/2399649226 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe8cc038f30 0x7fe8cc03b3f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:21.486 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.957+0000 7fe8e876a700 1 --2- 172.21.15.60:0/2399649226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8e0106450 0x7fe8e01915c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:21.486 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.957+0000 7fe8e876a700 1 -- 172.21.15.60:0/2399649226 >> 172.21.15.60:0/2399649226 conn(0x7fe8e0101a10 msgr2=0x7fe8e0102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:21.486 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.957+0000 7fe8e876a700 1 -- 172.21.15.60:0/2399649226 shutdown_connections 2024-02-26T04:48:21.486 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:20.957+0000 7fe8e876a700 1 -- 172.21.15.60:0/2399649226 wait complete. 2024-02-26T04:48:21.486 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial admin user... 2024-02-26T04:48:21.598 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:21 smithi060 ceph-mon[32382]: mgrmap e12: x(active, since 2s) 2024-02-26T04:48:21.598 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:21.598 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:21 smithi060 ceph-mon[32382]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:48:21.598 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:21.599 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:22.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$AYvGcNouOuVhADq9wOMMRewGCkhvLSFnypXJ3gdRbCh8ar3kNLI9y", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708922902, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-26T04:48:22.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.885+0000 7f856a7ab700 1 Processor -- start 2024-02-26T04:48:22.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.885+0000 7f856a7ab700 1 -- start start 2024-02-26T04:48:22.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.885+0000 7f856a7ab700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8564106450 0x7f8564106870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:22.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.885+0000 7f856a7ab700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8564106e40 con 0x7f8564106450 2024-02-26T04:48:22.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.886+0000 7f8563fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8564106450 0x7f8564106870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:22.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.886+0000 7f8563fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8564106450 0x7f8564106870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59562/0 (socket says 172.21.15.60:59562) 2024-02-26T04:48:22.912 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.886+0000 7f8563fff700 1 -- 172.21.15.60:0/2458927778 learned_addr learned my addr 172.21.15.60:0/2458927778 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:22.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.886+0000 7f8563fff700 1 -- 172.21.15.60:0/2458927778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8564107650 con 0x7f8564106450 2024-02-26T04:48:22.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.886+0000 7f8563fff700 1 --2- 172.21.15.60:0/2458927778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8564106450 0x7f8564106870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f855400bb40 tx=0x7f855400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c81ea20d85d417e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:22.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.886+0000 7f8562ffd700 1 -- 172.21.15.60:0/2458927778 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f855400dbe0 con 0x7f8564106450 2024-02-26T04:48:22.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.886+0000 7f8562ffd700 1 -- 172.21.15.60:0/2458927778 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8554004510 con 0x7f8564106450 2024-02-26T04:48:22.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.886+0000 7f8562ffd700 1 -- 172.21.15.60:0/2458927778 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8554010470 con 0x7f8564106450 2024-02-26T04:48:22.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.887+0000 7f856a7ab700 1 -- 172.21.15.60:0/2458927778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8564106450 msgr2=0x7f8564106870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:22.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.887+0000 7f856a7ab700 1 --2- 172.21.15.60:0/2458927778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8564106450 0x7f8564106870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f855400bb40 tx=0x7f855400be50 comp rx=0 tx=0).stop 2024-02-26T04:48:22.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.887+0000 7f856a7ab700 1 -- 172.21.15.60:0/2458927778 shutdown_connections 2024-02-26T04:48:22.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.887+0000 7f856a7ab700 1 --2- 172.21.15.60:0/2458927778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8564106450 0x7f8564106870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:22.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.887+0000 7f856a7ab700 1 -- 172.21.15.60:0/2458927778 >> 172.21.15.60:0/2458927778 conn(0x7f8564101a10 msgr2=0x7f8564103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:22.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.887+0000 7f856a7ab700 1 -- 172.21.15.60:0/2458927778 shutdown_connections 2024-02-26T04:48:22.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.887+0000 7f856a7ab700 1 -- 172.21.15.60:0/2458927778 wait complete. 2024-02-26T04:48:22.914 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.888+0000 7f856a7ab700 1 Processor -- start 2024-02-26T04:48:22.914 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.888+0000 7f856a7ab700 1 -- start start 2024-02-26T04:48:22.914 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.888+0000 7f856a7ab700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8564106450 0x7f8564199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:22.914 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.888+0000 7f856a7ab700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8564199760 con 0x7f8564106450 2024-02-26T04:48:22.914 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.888+0000 7f8563fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8564106450 0x7f8564199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:22.914 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.888+0000 7f8563fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8564106450 0x7f8564199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59568/0 (socket says 172.21.15.60:59568) 2024-02-26T04:48:22.914 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.888+0000 7f8563fff700 1 -- 172.21.15.60:0/2610415108 learned_addr learned my addr 172.21.15.60:0/2610415108 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:22.914 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.889+0000 7f8563fff700 1 -- 172.21.15.60:0/2610415108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855400b7f0 con 0x7f8564106450 2024-02-26T04:48:22.914 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.889+0000 7f8563fff700 1 --2- 172.21.15.60:0/2610415108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8564106450 0x7f8564199220 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f855400bfd0 tx=0x7f85540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:22.914 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.889+0000 7f85617fa700 1 -- 172.21.15.60:0/2610415108 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f855401e070 con 0x7f8564106450 2024-02-26T04:48:22.914 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.889+0000 7f856a7ab700 1 -- 172.21.15.60:0/2610415108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8564199960 con 0x7f8564106450 2024-02-26T04:48:22.914 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.889+0000 7f85617fa700 1 -- 172.21.15.60:0/2610415108 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8554004260 con 0x7f8564106450 2024-02-26T04:48:22.915 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.889+0000 7f85617fa700 1 -- 172.21.15.60:0/2610415108 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8554022a20 con 0x7f8564106450 2024-02-26T04:48:22.915 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.889+0000 7f856a7ab700 1 -- 172.21.15.60:0/2610415108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8564199e00 con 0x7f8564106450 2024-02-26T04:48:22.915 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.890+0000 7f85617fa700 1 -- 172.21.15.60:0/2610415108 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f855402c440 con 0x7f8564106450 2024-02-26T04:48:22.915 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.890+0000 7f85617fa700 1 --2- 172.21.15.60:0/2610415108 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f854c038ee0 0x7f854c03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:22.915 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.890+0000 7f85617fa700 1 -- 172.21.15.60:0/2610415108 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f855404fb50 con 0x7f8564106450 2024-02-26T04:48:22.915 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.890+0000 7f856a7ab700 1 -- 172.21.15.60:0/2610415108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8564068d30 con 0x7f8564106450 2024-02-26T04:48:22.915 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.890+0000 7f85637fe700 1 --2- 172.21.15.60:0/2610415108 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f854c038ee0 0x7f854c03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:22.915 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.891+0000 7f85637fe700 1 --2- 172.21.15.60:0/2610415108 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f854c038ee0 0x7f854c03b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8558006fd0 tx=0x7f8558006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:22.915 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:21.895+0000 7f85617fa700 1 -- 172.21.15.60:0/2610415108 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f855404fe60 con 0x7f8564106450 2024-02-26T04:48:22.915 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:22.128+0000 7f856a7ab700 1 -- 172.21.15.60:0/2610415108 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f856407a0b0 con 0x7f854c038ee0 2024-02-26T04:48:22.915 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:22.375+0000 7f85617fa700 1 -- 172.21.15.60:0/2610415108 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f856407a0b0 con 0x7f854c038ee0 2024-02-26T04:48:22.915 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:22.377+0000 7f856a7ab700 1 -- 172.21.15.60:0/2610415108 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f854c038ee0 msgr2=0x7f854c03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:22.916 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:22.377+0000 7f856a7ab700 1 --2- 172.21.15.60:0/2610415108 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f854c038ee0 0x7f854c03b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8558006fd0 tx=0x7f8558006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:22.916 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:22.377+0000 7f856a7ab700 1 -- 172.21.15.60:0/2610415108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8564106450 msgr2=0x7f8564199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:22.916 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:22.377+0000 7f856a7ab700 1 --2- 172.21.15.60:0/2610415108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8564106450 0x7f8564199220 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f855400bfd0 tx=0x7f85540047c0 comp rx=0 tx=0).stop 2024-02-26T04:48:22.916 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:22.378+0000 7f856a7ab700 1 -- 172.21.15.60:0/2610415108 shutdown_connections 2024-02-26T04:48:22.916 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:22.378+0000 7f856a7ab700 1 --2- 172.21.15.60:0/2610415108 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f854c038ee0 0x7f854c03b3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:22.916 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:22.378+0000 7f856a7ab700 1 --2- 172.21.15.60:0/2610415108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8564106450 0x7f8564199220 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:22.916 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:22.378+0000 7f856a7ab700 1 -- 172.21.15.60:0/2610415108 >> 172.21.15.60:0/2610415108 conn(0x7f8564101a10 msgr2=0x7f856418f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:22.916 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:22.378+0000 7f856a7ab700 1 -- 172.21.15.60:0/2610415108 shutdown_connections 2024-02-26T04:48:22.916 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:22.378+0000 7f856a7ab700 1 -- 172.21.15.60:0/2610415108 wait complete. 2024-02-26T04:48:22.916 INFO:teuthology.orchestra.run.smithi060.stdout:Fetching dashboard port number... 2024-02-26T04:48:23.190 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:22 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:23.190 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:22 smithi060 ceph-mon[32382]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:48:23.190 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:22 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:24.036 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 8443 2024-02-26T04:48:24.036 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.295+0000 7fbae4870700 1 Processor -- start 2024-02-26T04:48:24.036 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.295+0000 7fbae4870700 1 -- start start 2024-02-26T04:48:24.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.296+0000 7fbae4870700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbae01064a0 0x7fbae01068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:24.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.296+0000 7fbae4870700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae0106e90 con 0x7fbae01064a0 2024-02-26T04:48:24.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.296+0000 7fbade59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbae01064a0 0x7fbae01068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:24.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.296+0000 7fbade59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbae01064a0 0x7fbae01068c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59584/0 (socket says 172.21.15.60:59584) 2024-02-26T04:48:24.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.296+0000 7fbade59c700 1 -- 172.21.15.60:0/2643986350 learned_addr learned my addr 172.21.15.60:0/2643986350 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:24.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.296+0000 7fbade59c700 1 -- 172.21.15.60:0/2643986350 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae01076a0 con 0x7fbae01064a0 2024-02-26T04:48:24.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.297+0000 7fbade59c700 1 --2- 172.21.15.60:0/2643986350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbae01064a0 0x7fbae01068c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbad00098a0 tx=0x7fbad0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51a3bf60647a230 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:24.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.297+0000 7fbadd59a700 1 -- 172.21.15.60:0/2643986350 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad0004160 con 0x7fbae01064a0 2024-02-26T04:48:24.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.297+0000 7fbadd59a700 1 -- 172.21.15.60:0/2643986350 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbad00042c0 con 0x7fbae01064a0 2024-02-26T04:48:24.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.297+0000 7fbadd59a700 1 -- 172.21.15.60:0/2643986350 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad0011a80 con 0x7fbae01064a0 2024-02-26T04:48:24.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.297+0000 7fbae4870700 1 -- 172.21.15.60:0/2643986350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbae01064a0 msgr2=0x7fbae01068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:24.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.297+0000 7fbae4870700 1 --2- 172.21.15.60:0/2643986350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbae01064a0 0x7fbae01068c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbad00098a0 tx=0x7fbad0009bb0 comp rx=0 tx=0).stop 2024-02-26T04:48:24.037 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.297+0000 7fbae4870700 1 -- 172.21.15.60:0/2643986350 shutdown_connections 2024-02-26T04:48:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.297+0000 7fbae4870700 1 --2- 172.21.15.60:0/2643986350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbae01064a0 0x7fbae01068c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.297+0000 7fbae4870700 1 -- 172.21.15.60:0/2643986350 >> 172.21.15.60:0/2643986350 conn(0x7fbae0101a80 msgr2=0x7fbae0103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.298+0000 7fbae4870700 1 -- 172.21.15.60:0/2643986350 shutdown_connections 2024-02-26T04:48:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.298+0000 7fbae4870700 1 -- 172.21.15.60:0/2643986350 wait complete. 2024-02-26T04:48:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.298+0000 7fbae4870700 1 Processor -- start 2024-02-26T04:48:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.298+0000 7fbae4870700 1 -- start start 2024-02-26T04:48:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.299+0000 7fbae4870700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbae01064a0 0x7fbae019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.299+0000 7fbae4870700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae019b950 con 0x7fbae01064a0 2024-02-26T04:48:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.299+0000 7fbade59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbae01064a0 0x7fbae019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.299+0000 7fbade59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbae01064a0 0x7fbae019b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59598/0 (socket says 172.21.15.60:59598) 2024-02-26T04:48:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.299+0000 7fbade59c700 1 -- 172.21.15.60:0/724535131 learned_addr learned my addr 172.21.15.60:0/724535131 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.299+0000 7fbade59c700 1 -- 172.21.15.60:0/724535131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad0009550 con 0x7fbae01064a0 2024-02-26T04:48:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.299+0000 7fbade59c700 1 --2- 172.21.15.60:0/724535131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbae01064a0 0x7fbae019b410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbad0003f20 tx=0x7fbad0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.299+0000 7fbad77fe700 1 -- 172.21.15.60:0/724535131 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad00038b0 con 0x7fbae01064a0 2024-02-26T04:48:24.039 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.299+0000 7fbae4870700 1 -- 172.21.15.60:0/724535131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae019bb50 con 0x7fbae01064a0 2024-02-26T04:48:24.039 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.299+0000 7fbae4870700 1 -- 172.21.15.60:0/724535131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae019bff0 con 0x7fbae01064a0 2024-02-26T04:48:24.039 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.299+0000 7fbad77fe700 1 -- 172.21.15.60:0/724535131 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbad0003a10 con 0x7fbae01064a0 2024-02-26T04:48:24.039 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.299+0000 7fbad77fe700 1 -- 172.21.15.60:0/724535131 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad001a5c0 con 0x7fbae01064a0 2024-02-26T04:48:24.039 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.300+0000 7fbad77fe700 1 -- 172.21.15.60:0/724535131 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fbad001a860 con 0x7fbae01064a0 2024-02-26T04:48:24.039 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.300+0000 7fbae4870700 1 -- 172.21.15.60:0/724535131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac0005360 con 0x7fbae01064a0 2024-02-26T04:48:24.039 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.301+0000 7fbad77fe700 1 --2- 172.21.15.60:0/724535131 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbacc038f40 0x7fbacc03b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:24.039 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.301+0000 7fbad77fe700 1 -- 172.21.15.60:0/724535131 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbad004e150 con 0x7fbae01064a0 2024-02-26T04:48:24.039 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.301+0000 7fbaddd9b700 1 --2- 172.21.15.60:0/724535131 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbacc038f40 0x7fbacc03b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:24.039 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.301+0000 7fbaddd9b700 1 --2- 172.21.15.60:0/724535131 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbacc038f40 0x7fbacc03b400 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbac8006fd0 tx=0x7fbac8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:24.040 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.306+0000 7fbad77fe700 1 -- 172.21.15.60:0/724535131 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbad0018c20 con 0x7fbae01064a0 2024-02-26T04:48:24.040 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.514+0000 7fbae4870700 1 -- 172.21.15.60:0/724535131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fbac0005610 con 0x7fbae01064a0 2024-02-26T04:48:24.040 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.514+0000 7fbad77fe700 1 -- 172.21.15.60:0/724535131 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fbad0018350 con 0x7fbae01064a0 2024-02-26T04:48:24.040 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.516+0000 7fbae4870700 1 -- 172.21.15.60:0/724535131 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbacc038f40 msgr2=0x7fbacc03b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:24.040 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.516+0000 7fbae4870700 1 --2- 172.21.15.60:0/724535131 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbacc038f40 0x7fbacc03b400 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbac8006fd0 tx=0x7fbac8006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:24.040 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.516+0000 7fbae4870700 1 -- 172.21.15.60:0/724535131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbae01064a0 msgr2=0x7fbae019b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:24.040 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.516+0000 7fbae4870700 1 --2- 172.21.15.60:0/724535131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbae01064a0 0x7fbae019b410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbad0003f20 tx=0x7fbad0011b00 comp rx=0 tx=0).stop 2024-02-26T04:48:24.040 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.516+0000 7fbae4870700 1 -- 172.21.15.60:0/724535131 shutdown_connections 2024-02-26T04:48:24.040 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.516+0000 7fbae4870700 1 --2- 172.21.15.60:0/724535131 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbacc038f40 0x7fbacc03b400 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:24.040 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.516+0000 7fbae4870700 1 --2- 172.21.15.60:0/724535131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbae01064a0 0x7fbae019b410 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:24.040 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.516+0000 7fbae4870700 1 -- 172.21.15.60:0/724535131 >> 172.21.15.60:0/724535131 conn(0x7fbae0101a80 msgr2=0x7fbae01029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:24.040 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.516+0000 7fbae4870700 1 -- 172.21.15.60:0/724535131 shutdown_connections 2024-02-26T04:48:24.040 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:23.516+0000 7fbae4870700 1 -- 172.21.15.60:0/724535131 wait complete. 2024-02-26T04:48:24.042 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-26T04:48:24.042 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-02-26T04:48:24.049 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-26T04:48:24.050 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-02-26T04:48:24.050 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-02-26T04:48:24.050 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-26T04:48:24.051 INFO:teuthology.orchestra.run.smithi060.stdout:Ceph Dashboard is now available at: 2024-02-26T04:48:24.051 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:48:24.051 INFO:teuthology.orchestra.run.smithi060.stdout: URL: https://smithi060.front.sepia.ceph.com:8443/ 2024-02-26T04:48:24.051 INFO:teuthology.orchestra.run.smithi060.stdout: User: admin 2024-02-26T04:48:24.051 INFO:teuthology.orchestra.run.smithi060.stdout: Password: 2imr5f07d5 2024-02-26T04:48:24.051 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:48:24.051 INFO:teuthology.orchestra.run.smithi060.stdout:Saving cluster configuration to /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config directory 2024-02-26T04:48:24.052 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling autotune for osd_memory_target 2024-02-26T04:48:24.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:23 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/724535131' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-26T04:48:25.799 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.811+0000 7f5100c11700 1 Processor -- start 2024-02-26T04:48:25.800 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.811+0000 7f5100c11700 1 -- start start 2024-02-26T04:48:25.800 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.811+0000 7f5100c11700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50fc1075e0 0x7f50fc107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:25.800 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.811+0000 7f5100c11700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc107fd0 con 0x7f50fc1075e0 2024-02-26T04:48:25.800 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.812+0000 7f50fa59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50fc1075e0 0x7f50fc107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:25.800 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.812+0000 7f50fa59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50fc1075e0 0x7f50fc107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59612/0 (socket says 172.21.15.60:59612) 2024-02-26T04:48:25.801 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.812+0000 7f50fa59c700 1 -- 172.21.15.60:0/633246107 learned_addr learned my addr 172.21.15.60:0/633246107 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:25.801 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.812+0000 7f50fa59c700 1 -- 172.21.15.60:0/633246107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc108740 con 0x7f50fc1075e0 2024-02-26T04:48:25.801 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.812+0000 7f50fa59c700 1 --2- 172.21.15.60:0/633246107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50fc1075e0 0x7f50fc107a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009e80 tx=0x7f50ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160c1cb892330f19 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:25.801 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.812+0000 7f50f959a700 1 -- 172.21.15.60:0/633246107 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50ec00e070 con 0x7f50fc1075e0 2024-02-26T04:48:25.801 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.812+0000 7f50f959a700 1 -- 172.21.15.60:0/633246107 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50ec00b8c0 con 0x7f50fc1075e0 2024-02-26T04:48:25.801 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.812+0000 7f50f959a700 1 -- 172.21.15.60:0/633246107 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50ec0044a0 con 0x7f50fc1075e0 2024-02-26T04:48:25.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.813+0000 7f5100c11700 1 -- 172.21.15.60:0/633246107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50fc1075e0 msgr2=0x7f50fc107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:25.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.813+0000 7f5100c11700 1 --2- 172.21.15.60:0/633246107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50fc1075e0 0x7f50fc107a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009e80 tx=0x7f50ec00b330 comp rx=0 tx=0).stop 2024-02-26T04:48:25.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.813+0000 7f5100c11700 1 -- 172.21.15.60:0/633246107 shutdown_connections 2024-02-26T04:48:25.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.813+0000 7f5100c11700 1 --2- 172.21.15.60:0/633246107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50fc1075e0 0x7f50fc107a00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:25.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.813+0000 7f5100c11700 1 -- 172.21.15.60:0/633246107 >> 172.21.15.60:0/633246107 conn(0x7f50fc102c80 msgr2=0x7f50fc1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:25.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.813+0000 7f5100c11700 1 -- 172.21.15.60:0/633246107 shutdown_connections 2024-02-26T04:48:25.802 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.813+0000 7f5100c11700 1 -- 172.21.15.60:0/633246107 wait complete. 2024-02-26T04:48:25.803 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.814+0000 7f5100c11700 1 Processor -- start 2024-02-26T04:48:25.803 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.814+0000 7f5100c11700 1 -- start start 2024-02-26T04:48:25.803 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.814+0000 7f5100c11700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50fc1075e0 0x7f50fc19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:25.803 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.814+0000 7f5100c11700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc19b840 con 0x7f50fc1075e0 2024-02-26T04:48:25.803 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.814+0000 7f50fa59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50fc1075e0 0x7f50fc19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:25.803 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.814+0000 7f50fa59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50fc1075e0 0x7f50fc19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59620/0 (socket says 172.21.15.60:59620) 2024-02-26T04:48:25.804 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.814+0000 7f50fa59c700 1 -- 172.21.15.60:0/3370818308 learned_addr learned my addr 172.21.15.60:0/3370818308 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:25.804 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.814+0000 7f50fa59c700 1 -- 172.21.15.60:0/3370818308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec009b30 con 0x7f50fc1075e0 2024-02-26T04:48:25.804 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.814+0000 7f50fa59c700 1 --2- 172.21.15.60:0/3370818308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50fc1075e0 0x7f50fc19b300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009fd0 tx=0x7f50ec00bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:25.804 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.815+0000 7f50f37fe700 1 -- 172.21.15.60:0/3370818308 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50ec00e040 con 0x7f50fc1075e0 2024-02-26T04:48:25.804 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.815+0000 7f50f37fe700 1 -- 172.21.15.60:0/3370818308 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50ec003d60 con 0x7f50fc1075e0 2024-02-26T04:48:25.804 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.815+0000 7f5100c11700 1 -- 172.21.15.60:0/3370818308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50fc19ba40 con 0x7f50fc1075e0 2024-02-26T04:48:25.805 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.815+0000 7f50f37fe700 1 -- 172.21.15.60:0/3370818308 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50ec01c670 con 0x7f50fc1075e0 2024-02-26T04:48:25.805 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.815+0000 7f5100c11700 1 -- 172.21.15.60:0/3370818308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50fc19bf40 con 0x7f50fc1075e0 2024-02-26T04:48:25.805 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.816+0000 7f50f37fe700 1 -- 172.21.15.60:0/3370818308 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f50ec01c8e0 con 0x7f50fc1075e0 2024-02-26T04:48:25.805 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.816+0000 7f5100c11700 1 -- 172.21.15.60:0/3370818308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50fc10b970 con 0x7f50fc1075e0 2024-02-26T04:48:25.805 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.816+0000 7f50f37fe700 1 --2- 172.21.15.60:0/3370818308 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f50e8038f90 0x7f50e803b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:25.805 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.816+0000 7f50f37fe700 1 -- 172.21.15.60:0/3370818308 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f50ec04d070 con 0x7f50fc1075e0 2024-02-26T04:48:25.806 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.816+0000 7f50f9d9b700 1 --2- 172.21.15.60:0/3370818308 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f50e8038f90 0x7f50e803b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:25.806 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.816+0000 7f50f9d9b700 1 --2- 172.21.15.60:0/3370818308 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f50e8038f90 0x7f50e803b450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f50e4006fd0 tx=0x7f50e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:25.806 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.821+0000 7f50f37fe700 1 -- 172.21.15.60:0/3370818308 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f50ec011980 con 0x7f50fc1075e0 2024-02-26T04:48:25.806 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:24.930+0000 7f50f37fe700 1 -- 172.21.15.60:0/3370818308 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f50ec01cb90 con 0x7f50fc1075e0 2024-02-26T04:48:25.806 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:25.025+0000 7f5100c11700 1 -- 172.21.15.60:0/3370818308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f50fc10bbb0 con 0x7f50fc1075e0 2024-02-26T04:48:25.806 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:25.026+0000 7f50f37fe700 1 -- 172.21.15.60:0/3370818308 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f50ec017030 con 0x7f50fc1075e0 2024-02-26T04:48:25.807 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:25.028+0000 7f5100c11700 1 -- 172.21.15.60:0/3370818308 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f50e8038f90 msgr2=0x7f50e803b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:25.807 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:25.028+0000 7f5100c11700 1 --2- 172.21.15.60:0/3370818308 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f50e8038f90 0x7f50e803b450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f50e4006fd0 tx=0x7f50e4006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:25.807 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:25.028+0000 7f5100c11700 1 -- 172.21.15.60:0/3370818308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50fc1075e0 msgr2=0x7f50fc19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:25.807 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:25.028+0000 7f5100c11700 1 --2- 172.21.15.60:0/3370818308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50fc1075e0 0x7f50fc19b300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009fd0 tx=0x7f50ec00bd10 comp rx=0 tx=0).stop 2024-02-26T04:48:25.807 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:25.028+0000 7f5100c11700 1 -- 172.21.15.60:0/3370818308 shutdown_connections 2024-02-26T04:48:25.807 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:25.028+0000 7f5100c11700 1 --2- 172.21.15.60:0/3370818308 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f50e8038f90 0x7f50e803b450 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:25.807 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:25.028+0000 7f5100c11700 1 --2- 172.21.15.60:0/3370818308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50fc1075e0 0x7f50fc19b300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:25.808 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:25.028+0000 7f5100c11700 1 -- 172.21.15.60:0/3370818308 >> 172.21.15.60:0/3370818308 conn(0x7f50fc102c80 msgr2=0x7f50fc103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:25.808 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:25.028+0000 7f5100c11700 1 -- 172.21.15.60:0/3370818308 shutdown_connections 2024-02-26T04:48:25.808 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:25.028+0000 7f5100c11700 1 -- 172.21.15.60:0/3370818308 wait complete. 2024-02-26T04:48:26.209 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:25 smithi060 ceph-mon[32382]: mgrmap e13: x(active, since 6s) 2024-02-26T04:48:27.104 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.246+0000 7f080499f700 1 Processor -- start 2024-02-26T04:48:27.104 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.246+0000 7f080499f700 1 -- start start 2024-02-26T04:48:27.104 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.247+0000 7f080499f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0800107410 0x7f0800107830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:27.105 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.247+0000 7f080499f700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0800107e00 con 0x7f0800107410 2024-02-26T04:48:27.105 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.247+0000 7f07ff7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0800107410 0x7f0800107830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:27.105 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.247+0000 7f07ff7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0800107410 0x7f0800107830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59626/0 (socket says 172.21.15.60:59626) 2024-02-26T04:48:27.105 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.247+0000 7f07ff7fe700 1 -- 172.21.15.60:0/809076239 learned_addr learned my addr 172.21.15.60:0/809076239 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:27.105 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.247+0000 7f07ff7fe700 1 -- 172.21.15.60:0/809076239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800108570 con 0x7f0800107410 2024-02-26T04:48:27.105 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.247+0000 7f07ff7fe700 1 --2- 172.21.15.60:0/809076239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0800107410 0x7f0800107830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009e80 tx=0x7f07f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a314b132babbf31 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:27.105 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.247+0000 7f07fe7fc700 1 -- 172.21.15.60:0/809076239 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07f800e070 con 0x7f0800107410 2024-02-26T04:48:27.105 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.247+0000 7f07fe7fc700 1 -- 172.21.15.60:0/809076239 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07f800b8c0 con 0x7f0800107410 2024-02-26T04:48:27.105 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.247+0000 7f07fe7fc700 1 -- 172.21.15.60:0/809076239 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07f80044a0 con 0x7f0800107410 2024-02-26T04:48:27.105 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.248+0000 7f080499f700 1 -- 172.21.15.60:0/809076239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0800107410 msgr2=0x7f0800107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:27.105 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.248+0000 7f080499f700 1 --2- 172.21.15.60:0/809076239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0800107410 0x7f0800107830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009e80 tx=0x7f07f800b330 comp rx=0 tx=0).stop 2024-02-26T04:48:27.105 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.248+0000 7f080499f700 1 -- 172.21.15.60:0/809076239 shutdown_connections 2024-02-26T04:48:27.105 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.248+0000 7f080499f700 1 --2- 172.21.15.60:0/809076239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0800107410 0x7f0800107830 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:27.105 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.248+0000 7f080499f700 1 -- 172.21.15.60:0/809076239 >> 172.21.15.60:0/809076239 conn(0x7f08001029f0 msgr2=0x7f0800104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:27.106 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.248+0000 7f080499f700 1 -- 172.21.15.60:0/809076239 shutdown_connections 2024-02-26T04:48:27.106 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.248+0000 7f080499f700 1 -- 172.21.15.60:0/809076239 wait complete. 2024-02-26T04:48:27.106 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.249+0000 7f080499f700 1 Processor -- start 2024-02-26T04:48:27.106 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.249+0000 7f080499f700 1 -- start start 2024-02-26T04:48:27.106 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.249+0000 7f080499f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0800107410 0x7f080019b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:27.106 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.249+0000 7f080499f700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080019b690 con 0x7f0800107410 2024-02-26T04:48:27.106 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.249+0000 7f07ff7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0800107410 0x7f080019b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:27.106 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.249+0000 7f07ff7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0800107410 0x7f080019b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59628/0 (socket says 172.21.15.60:59628) 2024-02-26T04:48:27.106 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.249+0000 7f07ff7fe700 1 -- 172.21.15.60:0/534973929 learned_addr learned my addr 172.21.15.60:0/534973929 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:27.106 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.250+0000 7f07ff7fe700 1 -- 172.21.15.60:0/534973929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f8009b30 con 0x7f0800107410 2024-02-26T04:48:27.106 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.250+0000 7f07ff7fe700 1 --2- 172.21.15.60:0/534973929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0800107410 0x7f080019b150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009fd0 tx=0x7f07f800bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:27.106 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.250+0000 7f07fcff9700 1 -- 172.21.15.60:0/534973929 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07f800e040 con 0x7f0800107410 2024-02-26T04:48:27.106 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.250+0000 7f07fcff9700 1 -- 172.21.15.60:0/534973929 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07f8003d60 con 0x7f0800107410 2024-02-26T04:48:27.106 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.250+0000 7f080499f700 1 -- 172.21.15.60:0/534973929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080019b890 con 0x7f0800107410 2024-02-26T04:48:27.107 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.250+0000 7f07fcff9700 1 -- 172.21.15.60:0/534973929 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07f801c670 con 0x7f0800107410 2024-02-26T04:48:27.107 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.250+0000 7f080499f700 1 -- 172.21.15.60:0/534973929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080019bd90 con 0x7f0800107410 2024-02-26T04:48:27.107 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.251+0000 7f07fcff9700 1 -- 172.21.15.60:0/534973929 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f07f801c8e0 con 0x7f0800107410 2024-02-26T04:48:27.107 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.251+0000 7f07fcff9700 1 --2- 172.21.15.60:0/534973929 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f07ec039030 0x7f07ec03b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:27.107 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.251+0000 7f07fcff9700 1 -- 172.21.15.60:0/534973929 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f07f804e040 con 0x7f0800107410 2024-02-26T04:48:27.107 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.251+0000 7f07feffd700 1 --2- 172.21.15.60:0/534973929 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f07ec039030 0x7f07ec03b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:27.107 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.251+0000 7f080499f700 1 -- 172.21.15.60:0/534973929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080010b7a0 con 0x7f0800107410 2024-02-26T04:48:27.107 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.252+0000 7f07feffd700 1 --2- 172.21.15.60:0/534973929 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f07ec039030 0x7f07ec03b4f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f07e8006fd0 tx=0x7f07e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:27.107 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.256+0000 7f07fcff9700 1 -- 172.21.15.60:0/534973929 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f07f8017070 con 0x7f0800107410 2024-02-26T04:48:27.107 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.571+0000 7f080499f700 1 -- 172.21.15.60:0/534973929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f080019e860 con 0x7f0800107410 2024-02-26T04:48:27.107 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.573+0000 7f07fcff9700 1 -- 172.21.15.60:0/534973929 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f07f8020030 con 0x7f0800107410 2024-02-26T04:48:27.107 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.575+0000 7f080499f700 1 -- 172.21.15.60:0/534973929 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f07ec039030 msgr2=0x7f07ec03b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:27.107 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.575+0000 7f080499f700 1 --2- 172.21.15.60:0/534973929 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f07ec039030 0x7f07ec03b4f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f07e8006fd0 tx=0x7f07e8006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:27.108 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.575+0000 7f080499f700 1 -- 172.21.15.60:0/534973929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0800107410 msgr2=0x7f080019b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:27.108 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.576+0000 7f080499f700 1 --2- 172.21.15.60:0/534973929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0800107410 0x7f080019b150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009fd0 tx=0x7f07f800bd10 comp rx=0 tx=0).stop 2024-02-26T04:48:27.108 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.576+0000 7f080499f700 1 -- 172.21.15.60:0/534973929 shutdown_connections 2024-02-26T04:48:27.108 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.576+0000 7f080499f700 1 --2- 172.21.15.60:0/534973929 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f07ec039030 0x7f07ec03b4f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:27.108 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.576+0000 7f080499f700 1 --2- 172.21.15.60:0/534973929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0800107410 0x7f080019b150 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:27.108 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.576+0000 7f080499f700 1 -- 172.21.15.60:0/534973929 >> 172.21.15.60:0/534973929 conn(0x7f08001029f0 msgr2=0x7f0800103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:27.108 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.576+0000 7f080499f700 1 -- 172.21.15.60:0/534973929 shutdown_connections 2024-02-26T04:48:27.108 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr 2024-02-26T04:48:26.576+0000 7f080499f700 1 -- 172.21.15.60:0/534973929 wait complete. 2024-02-26T04:48:27.108 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-26T04:48:27.108 INFO:teuthology.orchestra.run.smithi060.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-26T04:48:27.108 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:48:27.108 INFO:teuthology.orchestra.run.smithi060.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-26T04:48:27.108 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:48:27.108 INFO:teuthology.orchestra.run.smithi060.stdout:Or, if you are only running a single cluster on this host: 2024-02-26T04:48:27.109 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:48:27.109 INFO:teuthology.orchestra.run.smithi060.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-26T04:48:27.109 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:48:27.109 INFO:teuthology.orchestra.run.smithi060.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-26T04:48:27.109 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:48:27.109 INFO:teuthology.orchestra.run.smithi060.stdout: ceph telemetry on 2024-02-26T04:48:27.109 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:48:27.109 INFO:teuthology.orchestra.run.smithi060.stdout:For more information see: 2024-02-26T04:48:27.109 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:48:27.109 INFO:teuthology.orchestra.run.smithi060.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-26T04:48:27.109 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:48:27.109 INFO:teuthology.orchestra.run.smithi060.stdout:Bootstrap complete. 2024-02-26T04:48:27.144 INFO:tasks.cephadm:Fetching config... 2024-02-26T04:48:27.145 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:48:27.145 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-26T04:48:27.164 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-26T04:48:27.164 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:48:27.164 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-26T04:48:27.218 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-26T04:48:27.219 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:48:27.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-26T04:48:27.286 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-26T04:48:27.286 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:48:27.286 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-26T04:48:27.341 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-26T04:48:27.341 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpAmQHZ/X4yJFbrTJkldnmCUiZH/yImKnS4aVLXUZyreKkQ8yQAyfymSz+EWpNexr/tsjyUvvxhEU36Ls8Azf7trww3lYn3y+GG1sMZWOsiWmHlJBQcHg5PtiEjxtVj8Mc+7eKnDC9yWVwgUXciJYslL4nFcoXHpLkiqeZfJC5u+em/JcQwDJUYgyAeVyX9C99BqZd2ghfd5woGcDOtnbdTqjBkZ5jwb3dvk6cGaFtbwizrldu94SSppVQINJZPzsu+4ZYVvKtncNcvkbW93URlCzYqG72vvhD8dY65TGV+/vNNYmVLjKAXl4Mv14rhdRE1oknDKabGpme5Ge7pgeP56w+BIshYWKGaO0+MOFuIjMGobVvZSqeKy6SxvGUtkEhjnTddIF8aVebY+kkoQyWZdbxmPxsQZ3o9A9GrT2f9DWZYtYo1Z1CbjowZKo6zus5TitVD2Ui2bpkhrL70Bx8m+Bb9Xi7yfSDm5PPwU99LZb/f6K5lGIdi3g1HJqOGUs= ceph-f12e2206-d461-11ee-95c3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-26T04:48:27.432 INFO:teuthology.orchestra.run.smithi060.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpAmQHZ/X4yJFbrTJkldnmCUiZH/yImKnS4aVLXUZyreKkQ8yQAyfymSz+EWpNexr/tsjyUvvxhEU36Ls8Azf7trww3lYn3y+GG1sMZWOsiWmHlJBQcHg5PtiEjxtVj8Mc+7eKnDC9yWVwgUXciJYslL4nFcoXHpLkiqeZfJC5u+em/JcQwDJUYgyAeVyX9C99BqZd2ghfd5woGcDOtnbdTqjBkZ5jwb3dvk6cGaFtbwizrldu94SSppVQINJZPzsu+4ZYVvKtncNcvkbW93URlCzYqG72vvhD8dY65TGV+/vNNYmVLjKAXl4Mv14rhdRE1oknDKabGpme5Ge7pgeP56w+BIshYWKGaO0+MOFuIjMGobVvZSqeKy6SxvGUtkEhjnTddIF8aVebY+kkoQyWZdbxmPxsQZ3o9A9GrT2f9DWZYtYo1Z1CbjowZKo6zus5TitVD2Ui2bpkhrL70Bx8m+Bb9Xi7yfSDm5PPwU99LZb/f6K5lGIdi3g1HJqOGUs= ceph-f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:48:27.455 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpAmQHZ/X4yJFbrTJkldnmCUiZH/yImKnS4aVLXUZyreKkQ8yQAyfymSz+EWpNexr/tsjyUvvxhEU36Ls8Azf7trww3lYn3y+GG1sMZWOsiWmHlJBQcHg5PtiEjxtVj8Mc+7eKnDC9yWVwgUXciJYslL4nFcoXHpLkiqeZfJC5u+em/JcQwDJUYgyAeVyX9C99BqZd2ghfd5woGcDOtnbdTqjBkZ5jwb3dvk6cGaFtbwizrldu94SSppVQINJZPzsu+4ZYVvKtncNcvkbW93URlCzYqG72vvhD8dY65TGV+/vNNYmVLjKAXl4Mv14rhdRE1oknDKabGpme5Ge7pgeP56w+BIshYWKGaO0+MOFuIjMGobVvZSqeKy6SxvGUtkEhjnTddIF8aVebY+kkoQyWZdbxmPxsQZ3o9A9GrT2f9DWZYtYo1Z1CbjowZKo6zus5TitVD2Ui2bpkhrL70Bx8m+Bb9Xi7yfSDm5PPwU99LZb/f6K5lGIdi3g1HJqOGUs= ceph-f12e2206-d461-11ee-95c3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-26T04:48:27.502 INFO:teuthology.orchestra.run.smithi156.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpAmQHZ/X4yJFbrTJkldnmCUiZH/yImKnS4aVLXUZyreKkQ8yQAyfymSz+EWpNexr/tsjyUvvxhEU36Ls8Azf7trww3lYn3y+GG1sMZWOsiWmHlJBQcHg5PtiEjxtVj8Mc+7eKnDC9yWVwgUXciJYslL4nFcoXHpLkiqeZfJC5u+em/JcQwDJUYgyAeVyX9C99BqZd2ghfd5woGcDOtnbdTqjBkZ5jwb3dvk6cGaFtbwizrldu94SSppVQINJZPzsu+4ZYVvKtncNcvkbW93URlCzYqG72vvhD8dY65TGV+/vNNYmVLjKAXl4Mv14rhdRE1oknDKabGpme5Ge7pgeP56w+BIshYWKGaO0+MOFuIjMGobVvZSqeKy6SxvGUtkEhjnTddIF8aVebY+kkoQyWZdbxmPxsQZ3o9A9GrT2f9DWZYtYo1Z1CbjowZKo6zus5TitVD2Ui2bpkhrL70Bx8m+Bb9Xi7yfSDm5PPwU99LZb/f6K5lGIdi3g1HJqOGUs= ceph-f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:48:27.518 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpAmQHZ/X4yJFbrTJkldnmCUiZH/yImKnS4aVLXUZyreKkQ8yQAyfymSz+EWpNexr/tsjyUvvxhEU36Ls8Azf7trww3lYn3y+GG1sMZWOsiWmHlJBQcHg5PtiEjxtVj8Mc+7eKnDC9yWVwgUXciJYslL4nFcoXHpLkiqeZfJC5u+em/JcQwDJUYgyAeVyX9C99BqZd2ghfd5woGcDOtnbdTqjBkZ5jwb3dvk6cGaFtbwizrldu94SSppVQINJZPzsu+4ZYVvKtncNcvkbW93URlCzYqG72vvhD8dY65TGV+/vNNYmVLjKAXl4Mv14rhdRE1oknDKabGpme5Ge7pgeP56w+BIshYWKGaO0+MOFuIjMGobVvZSqeKy6SxvGUtkEhjnTddIF8aVebY+kkoQyWZdbxmPxsQZ3o9A9GrT2f9DWZYtYo1Z1CbjowZKo6zus5TitVD2Ui2bpkhrL70Bx8m+Bb9Xi7yfSDm5PPwU99LZb/f6K5lGIdi3g1HJqOGUs= ceph-f12e2206-d461-11ee-95c3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-26T04:48:27.565 INFO:teuthology.orchestra.run.smithi191.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpAmQHZ/X4yJFbrTJkldnmCUiZH/yImKnS4aVLXUZyreKkQ8yQAyfymSz+EWpNexr/tsjyUvvxhEU36Ls8Azf7trww3lYn3y+GG1sMZWOsiWmHlJBQcHg5PtiEjxtVj8Mc+7eKnDC9yWVwgUXciJYslL4nFcoXHpLkiqeZfJC5u+em/JcQwDJUYgyAeVyX9C99BqZd2ghfd5woGcDOtnbdTqjBkZ5jwb3dvk6cGaFtbwizrldu94SSppVQINJZPzsu+4ZYVvKtncNcvkbW93URlCzYqG72vvhD8dY65TGV+/vNNYmVLjKAXl4Mv14rhdRE1oknDKabGpme5Ge7pgeP56w+BIshYWKGaO0+MOFuIjMGobVvZSqeKy6SxvGUtkEhjnTddIF8aVebY+kkoQyWZdbxmPxsQZ3o9A9GrT2f9DWZYtYo1Z1CbjowZKo6zus5TitVD2Ui2bpkhrL70Bx8m+Bb9Xi7yfSDm5PPwU99LZb/f6K5lGIdi3g1HJqOGUs= ceph-f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:48:27.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-26T04:48:27.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:27 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/534973929' entity='client.admin' 2024-02-26T04:48:27.820 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:48:30.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.095+0000 7f0f1bb17700 1 -- 172.21.15.60:0/390335221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f141016c0 msgr2=0x7f0f14101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:30.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.095+0000 7f0f1bb17700 1 --2- 172.21.15.60:0/390335221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f141016c0 0x7f0f14101ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009910 tx=0x7f0f10009c20 comp rx=0 tx=0).stop 2024-02-26T04:48:30.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.095+0000 7f0f1bb17700 1 -- 172.21.15.60:0/390335221 shutdown_connections 2024-02-26T04:48:30.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.095+0000 7f0f1bb17700 1 --2- 172.21.15.60:0/390335221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f141016c0 0x7f0f14101ae0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:30.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.095+0000 7f0f1bb17700 1 -- 172.21.15.60:0/390335221 >> 172.21.15.60:0/390335221 conn(0x7f0f140fcc60 msgr2=0x7f0f140ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:30.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.095+0000 7f0f1bb17700 1 -- 172.21.15.60:0/390335221 shutdown_connections 2024-02-26T04:48:30.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.095+0000 7f0f1bb17700 1 -- 172.21.15.60:0/390335221 wait complete. 2024-02-26T04:48:30.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.096+0000 7f0f1bb17700 1 Processor -- start 2024-02-26T04:48:30.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.096+0000 7f0f1bb17700 1 -- start start 2024-02-26T04:48:30.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.096+0000 7f0f1bb17700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f141016c0 0x7f0f14198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:30.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.096+0000 7f0f1bb17700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f14199310 con 0x7f0f141016c0 2024-02-26T04:48:30.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.096+0000 7f0f198b3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f141016c0 0x7f0f14198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:30.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.096+0000 7f0f198b3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f141016c0 0x7f0f14198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49046/0 (socket says 172.21.15.60:49046) 2024-02-26T04:48:30.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.096+0000 7f0f198b3700 1 -- 172.21.15.60:0/522998343 learned_addr learned my addr 172.21.15.60:0/522998343 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:30.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.096+0000 7f0f198b3700 1 -- 172.21.15.60:0/522998343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f100095f0 con 0x7f0f141016c0 2024-02-26T04:48:30.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.097+0000 7f0f198b3700 1 --2- 172.21.15.60:0/522998343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f141016c0 0x7f0f14198dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f10005850 tx=0x7f0f10004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:30.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.097+0000 7f0f0affd700 1 -- 172.21.15.60:0/522998343 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f1001d070 con 0x7f0f141016c0 2024-02-26T04:48:30.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.097+0000 7f0f0affd700 1 -- 172.21.15.60:0/522998343 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f1000be40 con 0x7f0f141016c0 2024-02-26T04:48:30.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.097+0000 7f0f1bb17700 1 -- 172.21.15.60:0/522998343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f14199510 con 0x7f0f141016c0 2024-02-26T04:48:30.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.097+0000 7f0f0affd700 1 -- 172.21.15.60:0/522998343 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f10022ea0 con 0x7f0f141016c0 2024-02-26T04:48:30.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.097+0000 7f0f1bb17700 1 -- 172.21.15.60:0/522998343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f141999b0 con 0x7f0f141016c0 2024-02-26T04:48:30.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.098+0000 7f0f0affd700 1 -- 172.21.15.60:0/522998343 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f0f10022670 con 0x7f0f141016c0 2024-02-26T04:48:30.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.098+0000 7f0f0affd700 1 --2- 172.21.15.60:0/522998343 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0f00038fe0 0x7f0f0003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:30.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.098+0000 7f0f1bb17700 1 -- 172.21.15.60:0/522998343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f14192920 con 0x7f0f141016c0 2024-02-26T04:48:30.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.098+0000 7f0f0affd700 1 -- 172.21.15.60:0/522998343 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0f1004d0a0 con 0x7f0f141016c0 2024-02-26T04:48:30.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.098+0000 7f0f190b2700 1 --2- 172.21.15.60:0/522998343 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0f00038fe0 0x7f0f0003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:30.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.098+0000 7f0f190b2700 1 --2- 172.21.15.60:0/522998343 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0f00038fe0 0x7f0f0003b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0f04006fd0 tx=0x7f0f04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:30.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.103+0000 7f0f0affd700 1 -- 172.21.15.60:0/522998343 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0f10027020 con 0x7f0f141016c0 2024-02-26T04:48:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.305+0000 7f0f1bb17700 1 -- 172.21.15.60:0/522998343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f0f14107790 con 0x7f0f141016c0 2024-02-26T04:48:30.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.307+0000 7f0f0affd700 1 -- 172.21.15.60:0/522998343 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0f1000f4b0 con 0x7f0f141016c0 2024-02-26T04:48:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.309+0000 7f0f1bb17700 1 -- 172.21.15.60:0/522998343 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0f00038fe0 msgr2=0x7f0f0003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.309+0000 7f0f1bb17700 1 --2- 172.21.15.60:0/522998343 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0f00038fe0 0x7f0f0003b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0f04006fd0 tx=0x7f0f04006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.309+0000 7f0f1bb17700 1 -- 172.21.15.60:0/522998343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f141016c0 msgr2=0x7f0f14198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.309+0000 7f0f1bb17700 1 --2- 172.21.15.60:0/522998343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f141016c0 0x7f0f14198dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f10005850 tx=0x7f0f10004ff0 comp rx=0 tx=0).stop 2024-02-26T04:48:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.309+0000 7f0f1bb17700 1 -- 172.21.15.60:0/522998343 shutdown_connections 2024-02-26T04:48:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.309+0000 7f0f1bb17700 1 --2- 172.21.15.60:0/522998343 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0f00038fe0 0x7f0f0003b4a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.309+0000 7f0f1bb17700 1 --2- 172.21.15.60:0/522998343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f141016c0 0x7f0f14198dd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.309+0000 7f0f1bb17700 1 -- 172.21.15.60:0/522998343 >> 172.21.15.60:0/522998343 conn(0x7f0f140fcc60 msgr2=0x7f0f140ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.309+0000 7f0f1bb17700 1 -- 172.21.15.60:0/522998343 shutdown_connections 2024-02-26T04:48:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:30.309+0000 7f0f1bb17700 1 -- 172.21.15.60:0/522998343 wait complete. 2024-02-26T04:48:30.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:30 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:30.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:30 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:30.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:30 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi060", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:48:30.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:30 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:30.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:30 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:48:30.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:30 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:30.863 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-26T04:48:30.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-26T04:48:31.100 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:48:31.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:31 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/522998343' entity='client.admin' 2024-02-26T04:48:31.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:48:31.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:32.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.630+0000 7fba3d7cf700 1 -- 172.21.15.60:0/1716912532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba381016c0 msgr2=0x7fba38101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:32.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.630+0000 7fba3d7cf700 1 --2- 172.21.15.60:0/1716912532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba381016c0 0x7fba38101ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fba28009910 tx=0x7fba28009c20 comp rx=0 tx=0).stop 2024-02-26T04:48:32.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.630+0000 7fba3d7cf700 1 -- 172.21.15.60:0/1716912532 shutdown_connections 2024-02-26T04:48:32.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.630+0000 7fba3d7cf700 1 --2- 172.21.15.60:0/1716912532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba381016c0 0x7fba38101ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:32.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.630+0000 7fba3d7cf700 1 -- 172.21.15.60:0/1716912532 >> 172.21.15.60:0/1716912532 conn(0x7fba380fcc60 msgr2=0x7fba380ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:32.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.630+0000 7fba3d7cf700 1 -- 172.21.15.60:0/1716912532 shutdown_connections 2024-02-26T04:48:32.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.630+0000 7fba3d7cf700 1 -- 172.21.15.60:0/1716912532 wait complete. 2024-02-26T04:48:32.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.630+0000 7fba3d7cf700 1 Processor -- start 2024-02-26T04:48:32.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.630+0000 7fba3d7cf700 1 -- start start 2024-02-26T04:48:32.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.631+0000 7fba3d7cf700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba381016c0 0x7fba38196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:32.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.631+0000 7fba3d7cf700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba381971a0 con 0x7fba381016c0 2024-02-26T04:48:32.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.631+0000 7fba36ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba381016c0 0x7fba38196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:32.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.631+0000 7fba36ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba381016c0 0x7fba38196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49052/0 (socket says 172.21.15.60:49052) 2024-02-26T04:48:32.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.631+0000 7fba36ffd700 1 -- 172.21.15.60:0/835195833 learned_addr learned my addr 172.21.15.60:0/835195833 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:32.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.631+0000 7fba36ffd700 1 -- 172.21.15.60:0/835195833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba280095f0 con 0x7fba381016c0 2024-02-26T04:48:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.631+0000 7fba36ffd700 1 --2- 172.21.15.60:0/835195833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba381016c0 0x7fba38196c60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fba28005850 tx=0x7fba28004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.631+0000 7fba1ffff700 1 -- 172.21.15.60:0/835195833 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba2801d070 con 0x7fba381016c0 2024-02-26T04:48:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.631+0000 7fba1ffff700 1 -- 172.21.15.60:0/835195833 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba2800be40 con 0x7fba381016c0 2024-02-26T04:48:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.631+0000 7fba3d7cf700 1 -- 172.21.15.60:0/835195833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba381973a0 con 0x7fba381016c0 2024-02-26T04:48:32.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.631+0000 7fba1ffff700 1 -- 172.21.15.60:0/835195833 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba28022ea0 con 0x7fba381016c0 2024-02-26T04:48:32.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.631+0000 7fba3d7cf700 1 -- 172.21.15.60:0/835195833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba38197840 con 0x7fba381016c0 2024-02-26T04:48:32.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.632+0000 7fba1ffff700 1 -- 172.21.15.60:0/835195833 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fba2800b960 con 0x7fba381016c0 2024-02-26T04:48:32.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.632+0000 7fba3d7cf700 1 -- 172.21.15.60:0/835195833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba38190520 con 0x7fba381016c0 2024-02-26T04:48:32.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.632+0000 7fba1ffff700 1 --2- 172.21.15.60:0/835195833 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fba20038f80 0x7fba2003b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:32.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.632+0000 7fba1ffff700 1 -- 172.21.15.60:0/835195833 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fba2804d110 con 0x7fba381016c0 2024-02-26T04:48:32.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.633+0000 7fba367fc700 1 --2- 172.21.15.60:0/835195833 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fba20038f80 0x7fba2003b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:32.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.633+0000 7fba367fc700 1 --2- 172.21.15.60:0/835195833 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fba20038f80 0x7fba2003b440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fba2c006fd0 tx=0x7fba2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:32.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.637+0000 7fba1ffff700 1 -- 172.21.15.60:0/835195833 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fba28022670 con 0x7fba381016c0 2024-02-26T04:48:32.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.843+0000 7fba3d7cf700 1 -- 172.21.15.60:0/835195833 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fba380620b0 con 0x7fba20038f80 2024-02-26T04:48:32.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.846+0000 7fba1ffff700 1 -- 172.21.15.60:0/835195833 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fba380620b0 con 0x7fba20038f80 2024-02-26T04:48:32.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.848+0000 7fba3d7cf700 1 -- 172.21.15.60:0/835195833 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fba20038f80 msgr2=0x7fba2003b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.848+0000 7fba3d7cf700 1 --2- 172.21.15.60:0/835195833 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fba20038f80 0x7fba2003b440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fba2c006fd0 tx=0x7fba2c006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.848+0000 7fba3d7cf700 1 -- 172.21.15.60:0/835195833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba381016c0 msgr2=0x7fba38196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.848+0000 7fba3d7cf700 1 --2- 172.21.15.60:0/835195833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba381016c0 0x7fba38196c60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fba28005850 tx=0x7fba28004ff0 comp rx=0 tx=0).stop 2024-02-26T04:48:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.848+0000 7fba3d7cf700 1 -- 172.21.15.60:0/835195833 shutdown_connections 2024-02-26T04:48:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.848+0000 7fba3d7cf700 1 --2- 172.21.15.60:0/835195833 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fba20038f80 0x7fba2003b440 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.848+0000 7fba3d7cf700 1 --2- 172.21.15.60:0/835195833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba381016c0 0x7fba38196c60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.848+0000 7fba3d7cf700 1 -- 172.21.15.60:0/835195833 >> 172.21.15.60:0/835195833 conn(0x7fba380fcc60 msgr2=0x7fba380fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.848+0000 7fba3d7cf700 1 -- 172.21.15.60:0/835195833 shutdown_connections 2024-02-26T04:48:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:32.848+0000 7fba3d7cf700 1 -- 172.21.15.60:0/835195833 wait complete. 2024-02-26T04:48:33.554 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi156 2024-02-26T04:48:33.555 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-26T04:48:33.555 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.conf 2024-02-26T04:48:33.573 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-26T04:48:33.574 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T04:48:33.632 INFO:tasks.cephadm:Adding host smithi156 to orchestrator... 2024-02-26T04:48:33.632 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch host add smithi156 2024-02-26T04:48:33.873 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:48:34.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:33 smithi060 ceph-mon[32382]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:48:34.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:34.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:48:34.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:48:34.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:48:34.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:33 smithi060 ceph-mon[32382]: Updating smithi060:/etc/ceph/ceph.conf 2024-02-26T04:48:34.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:33 smithi060 ceph-mon[32382]: Updating smithi060:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:48:34.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:34.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:34.104 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:34.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:34 smithi060 ceph-mon[32382]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2024-02-26T04:48:34.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:34 smithi060 ceph-mon[32382]: Updating smithi060:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T04:48:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.357+0000 7fecc33a3700 1 -- 172.21.15.60:0/2773291286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecbc106600 msgr2=0x7fecbc1069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.357+0000 7fecc33a3700 1 --2- 172.21.15.60:0/2773291286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecbc106600 0x7fecbc1069e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009910 tx=0x7fecb8009c20 comp rx=0 tx=0).stop 2024-02-26T04:48:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.357+0000 7fecc33a3700 1 -- 172.21.15.60:0/2773291286 shutdown_connections 2024-02-26T04:48:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.357+0000 7fecc33a3700 1 --2- 172.21.15.60:0/2773291286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecbc106600 0x7fecbc1069e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.357+0000 7fecc33a3700 1 -- 172.21.15.60:0/2773291286 >> 172.21.15.60:0/2773291286 conn(0x7fecbc0fc020 msgr2=0x7fecbc0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.357+0000 7fecc33a3700 1 -- 172.21.15.60:0/2773291286 shutdown_connections 2024-02-26T04:48:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.357+0000 7fecc33a3700 1 -- 172.21.15.60:0/2773291286 wait complete. 2024-02-26T04:48:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.358+0000 7fecc33a3700 1 Processor -- start 2024-02-26T04:48:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.358+0000 7fecc33a3700 1 -- start start 2024-02-26T04:48:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.358+0000 7fecc33a3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecbc106600 0x7fecbc0fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.358+0000 7fecc33a3700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc0fe3c0 con 0x7fecbc106600 2024-02-26T04:48:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.358+0000 7fecc113f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecbc106600 0x7fecbc0fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.358+0000 7fecc113f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecbc106600 0x7fecbc0fc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49076/0 (socket says 172.21.15.60:49076) 2024-02-26T04:48:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.358+0000 7fecc113f700 1 -- 172.21.15.60:0/831182473 learned_addr learned my addr 172.21.15.60:0/831182473 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.358+0000 7fecc113f700 1 -- 172.21.15.60:0/831182473 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb80095f0 con 0x7fecbc106600 2024-02-26T04:48:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.358+0000 7fecc113f700 1 --2- 172.21.15.60:0/831182473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecbc106600 0x7fecbc0fc4d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fecb8006010 tx=0x7fecb8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.359+0000 7fecb27fc700 1 -- 172.21.15.60:0/831182473 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecb801d070 con 0x7fecbc106600 2024-02-26T04:48:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.359+0000 7fecc33a3700 1 -- 172.21.15.60:0/831182473 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecbc0fca10 con 0x7fecbc106600 2024-02-26T04:48:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.359+0000 7fecb27fc700 1 -- 172.21.15.60:0/831182473 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecb800bc30 con 0x7fecbc106600 2024-02-26T04:48:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.359+0000 7fecb27fc700 1 -- 172.21.15.60:0/831182473 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecb8003930 con 0x7fecbc106600 2024-02-26T04:48:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.359+0000 7fecc33a3700 1 -- 172.21.15.60:0/831182473 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc0fce90 con 0x7fecbc106600 2024-02-26T04:48:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.359+0000 7fecb27fc700 1 -- 172.21.15.60:0/831182473 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fecb8003b50 con 0x7fecbc106600 2024-02-26T04:48:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.359+0000 7fecc33a3700 1 -- 172.21.15.60:0/831182473 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecbc101c70 con 0x7fecbc106600 2024-02-26T04:48:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.360+0000 7fecb27fc700 1 --2- 172.21.15.60:0/831182473 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7feca8038f80 0x7feca803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.360+0000 7fecb27fc700 1 -- 172.21.15.60:0/831182473 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fecb804cda0 con 0x7fecbc106600 2024-02-26T04:48:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.360+0000 7fecc093e700 1 --2- 172.21.15.60:0/831182473 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7feca8038f80 0x7feca803b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.360+0000 7fecc093e700 1 --2- 172.21.15.60:0/831182473 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7feca8038f80 0x7feca803b440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fecac006fd0 tx=0x7fecac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.365+0000 7fecb27fc700 1 -- 172.21.15.60:0/831182473 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fecb8029540 con 0x7fecbc106600 2024-02-26T04:48:35.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:35.571+0000 7fecc33a3700 1 -- 172.21.15.60:0/831182473 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}) v1 -- 0x7fecbc0008d0 con 0x7feca8038f80 2024-02-26T04:48:36.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:36 smithi060 ceph-mon[32382]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:48:37.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:37 smithi060 ceph-mon[32382]: Deploying cephadm binary to smithi156 2024-02-26T04:48:39.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:39 smithi060 ceph-mon[32382]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:48:39.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:39.894+0000 7fecb27fc700 1 -- 172.21.15.60:0/831182473 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fecbc0008d0 con 0x7feca8038f80 2024-02-26T04:48:39.897 INFO:teuthology.orchestra.run.smithi060.stdout:Added host 'smithi156' with addr '172.21.15.156' 2024-02-26T04:48:39.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:39.897+0000 7fecc33a3700 1 -- 172.21.15.60:0/831182473 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7feca8038f80 msgr2=0x7feca803b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:39.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:39.897+0000 7fecc33a3700 1 --2- 172.21.15.60:0/831182473 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7feca8038f80 0x7feca803b440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fecac006fd0 tx=0x7fecac006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:39.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:39.897+0000 7fecc33a3700 1 -- 172.21.15.60:0/831182473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecbc106600 msgr2=0x7fecbc0fc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:39.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:39.897+0000 7fecc33a3700 1 --2- 172.21.15.60:0/831182473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecbc106600 0x7fecbc0fc4d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fecb8006010 tx=0x7fecb8004ff0 comp rx=0 tx=0).stop 2024-02-26T04:48:39.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:39.897+0000 7fecc33a3700 1 -- 172.21.15.60:0/831182473 shutdown_connections 2024-02-26T04:48:39.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:39.897+0000 7fecc33a3700 1 --2- 172.21.15.60:0/831182473 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7feca8038f80 0x7feca803b440 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:39.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:39.897+0000 7fecc33a3700 1 --2- 172.21.15.60:0/831182473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecbc106600 0x7fecbc0fc4d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:39.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:39.897+0000 7fecc33a3700 1 -- 172.21.15.60:0/831182473 >> 172.21.15.60:0/831182473 conn(0x7fecbc0fc020 msgr2=0x7fecbc0698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:39.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:39.897+0000 7fecc33a3700 1 -- 172.21.15.60:0/831182473 shutdown_connections 2024-02-26T04:48:39.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:39.897+0000 7fecc33a3700 1 -- 172.21.15.60:0/831182473 wait complete. 2024-02-26T04:48:40.650 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch host ls --format=json 2024-02-26T04:48:40.898 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:48:40.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:40 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:40.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:40 smithi060 ceph-mon[32382]: Added host smithi156 2024-02-26T04:48:40.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:40 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:48:40.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:40 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:42.188 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:41 smithi060 ceph-mon[32382]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:48:42.188 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:41 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:42.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.369+0000 7f16d5472700 1 -- 172.21.15.60:0/3729584875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d01076d0 msgr2=0x7f16d0107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:42.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.369+0000 7f16d5472700 1 --2- 172.21.15.60:0/3729584875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d01076d0 0x7f16d0107ab0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009910 tx=0x7f16c0009c20 comp rx=0 tx=0).stop 2024-02-26T04:48:42.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.369+0000 7f16d5472700 1 -- 172.21.15.60:0/3729584875 shutdown_connections 2024-02-26T04:48:42.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.369+0000 7f16d5472700 1 --2- 172.21.15.60:0/3729584875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d01076d0 0x7f16d0107ab0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:42.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.369+0000 7f16d5472700 1 -- 172.21.15.60:0/3729584875 >> 172.21.15.60:0/3729584875 conn(0x7f16d00fb7c0 msgr2=0x7f16d00fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:42.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.369+0000 7f16d5472700 1 -- 172.21.15.60:0/3729584875 shutdown_connections 2024-02-26T04:48:42.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.369+0000 7f16d5472700 1 -- 172.21.15.60:0/3729584875 wait complete. 2024-02-26T04:48:42.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.370+0000 7f16d5472700 1 Processor -- start 2024-02-26T04:48:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.370+0000 7f16d5472700 1 -- start start 2024-02-26T04:48:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.370+0000 7f16d5472700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d01076d0 0x7f16d0196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.370+0000 7f16d5472700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d0197430 con 0x7f16d01076d0 2024-02-26T04:48:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.370+0000 7f16ceffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d01076d0 0x7f16d0196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.370+0000 7f16ceffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d01076d0 0x7f16d0196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47080/0 (socket says 172.21.15.60:47080) 2024-02-26T04:48:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.370+0000 7f16ceffd700 1 -- 172.21.15.60:0/1753289798 learned_addr learned my addr 172.21.15.60:0/1753289798 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.371+0000 7f16ceffd700 1 -- 172.21.15.60:0/1753289798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c00095f0 con 0x7f16d01076d0 2024-02-26T04:48:42.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.371+0000 7f16ceffd700 1 --2- 172.21.15.60:0/1753289798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d01076d0 0x7f16d0196ef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16c0004f40 tx=0x7f16c0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:42.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.371+0000 7f16b7fff700 1 -- 172.21.15.60:0/1753289798 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16c001d070 con 0x7f16d01076d0 2024-02-26T04:48:42.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.371+0000 7f16b7fff700 1 -- 172.21.15.60:0/1753289798 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16c000bbd0 con 0x7f16d01076d0 2024-02-26T04:48:42.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.371+0000 7f16d5472700 1 -- 172.21.15.60:0/1753289798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d0197690 con 0x7f16d01076d0 2024-02-26T04:48:42.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.371+0000 7f16b7fff700 1 -- 172.21.15.60:0/1753289798 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16c0003900 con 0x7f16d01076d0 2024-02-26T04:48:42.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.371+0000 7f16d5472700 1 -- 172.21.15.60:0/1753289798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d0197b90 con 0x7f16d01076d0 2024-02-26T04:48:42.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.372+0000 7f16b7fff700 1 -- 172.21.15.60:0/1753289798 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f16c0003bc0 con 0x7f16d01076d0 2024-02-26T04:48:42.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.372+0000 7f16b7fff700 1 --2- 172.21.15.60:0/1753289798 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f16b8038f80 0x7f16b803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:42.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.372+0000 7f16b7fff700 1 -- 172.21.15.60:0/1753289798 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f16c004cc30 con 0x7f16d01076d0 2024-02-26T04:48:42.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.372+0000 7f16ce7fc700 1 --2- 172.21.15.60:0/1753289798 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f16b8038f80 0x7f16b803b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:42.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.372+0000 7f16d5472700 1 -- 172.21.15.60:0/1753289798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d0102dc0 con 0x7f16d01076d0 2024-02-26T04:48:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.372+0000 7f16ce7fc700 1 --2- 172.21.15.60:0/1753289798 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f16b8038f80 0x7f16b803b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f16c4006fd0 tx=0x7f16c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:42.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.377+0000 7f16b7fff700 1 -- 172.21.15.60:0/1753289798 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f16c0026080 con 0x7f16d01076d0 2024-02-26T04:48:42.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.597+0000 7f16d5472700 1 -- 172.21.15.60:0/1753289798 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f16d010ba90 con 0x7f16b8038f80 2024-02-26T04:48:42.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.599+0000 7f16b7fff700 1 -- 172.21.15.60:0/1753289798 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f16d010ba90 con 0x7f16b8038f80 2024-02-26T04:48:42.601 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:48:42.601 INFO:teuthology.orchestra.run.smithi060.stdout:[{"addr": "172.21.15.60", "hostname": "smithi060", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}] 2024-02-26T04:48:42.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.601+0000 7f16d5472700 1 -- 172.21.15.60:0/1753289798 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f16b8038f80 msgr2=0x7f16b803b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:42.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.601+0000 7f16d5472700 1 --2- 172.21.15.60:0/1753289798 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f16b8038f80 0x7f16b803b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f16c4006fd0 tx=0x7f16c4006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:42.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.601+0000 7f16d5472700 1 -- 172.21.15.60:0/1753289798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d01076d0 msgr2=0x7f16d0196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:42.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.601+0000 7f16d5472700 1 --2- 172.21.15.60:0/1753289798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d01076d0 0x7f16d0196ef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16c0004f40 tx=0x7f16c0005020 comp rx=0 tx=0).stop 2024-02-26T04:48:42.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.601+0000 7f16d5472700 1 -- 172.21.15.60:0/1753289798 shutdown_connections 2024-02-26T04:48:42.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.601+0000 7f16d5472700 1 --2- 172.21.15.60:0/1753289798 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f16b8038f80 0x7f16b803b440 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:42.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.601+0000 7f16d5472700 1 --2- 172.21.15.60:0/1753289798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16d01076d0 0x7f16d0196ef0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:42.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.601+0000 7f16d5472700 1 -- 172.21.15.60:0/1753289798 >> 172.21.15.60:0/1753289798 conn(0x7f16d00fb7c0 msgr2=0x7f16d00fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:42.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.601+0000 7f16d5472700 1 -- 172.21.15.60:0/1753289798 shutdown_connections 2024-02-26T04:48:42.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:42.601+0000 7f16d5472700 1 -- 172.21.15.60:0/1753289798 wait complete. 2024-02-26T04:48:43.212 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi191 2024-02-26T04:48:43.212 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-26T04:48:43.212 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/etc/ceph/ceph.conf 2024-02-26T04:48:43.230 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-26T04:48:43.230 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T04:48:43.287 INFO:tasks.cephadm:Adding host smithi191 to orchestrator... 2024-02-26T04:48:43.287 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch host add smithi191 2024-02-26T04:48:43.534 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:48:43.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:43 smithi060 ceph-mon[32382]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:48:43.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:43 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:43.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:43 smithi060 ceph-mon[32382]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:48:44.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.888+0000 7f8a26d42700 1 -- 172.21.15.60:0/3601794379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a20102870 msgr2=0x7f8a20102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:44.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.888+0000 7f8a26d42700 1 --2- 172.21.15.60:0/3601794379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a20102870 0x7f8a20102c50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009910 tx=0x7f8a14009c20 comp rx=0 tx=0).stop 2024-02-26T04:48:44.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.888+0000 7f8a26d42700 1 -- 172.21.15.60:0/3601794379 shutdown_connections 2024-02-26T04:48:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.888+0000 7f8a26d42700 1 --2- 172.21.15.60:0/3601794379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a20102870 0x7f8a20102c50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.888+0000 7f8a26d42700 1 -- 172.21.15.60:0/3601794379 >> 172.21.15.60:0/3601794379 conn(0x7f8a200fc040 msgr2=0x7f8a200fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.888+0000 7f8a26d42700 1 -- 172.21.15.60:0/3601794379 shutdown_connections 2024-02-26T04:48:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.889+0000 7f8a26d42700 1 -- 172.21.15.60:0/3601794379 wait complete. 2024-02-26T04:48:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.889+0000 7f8a26d42700 1 Processor -- start 2024-02-26T04:48:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.889+0000 7f8a26d42700 1 -- start start 2024-02-26T04:48:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.889+0000 7f8a26d42700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a20102870 0x7f8a201004d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.889+0000 7f8a26d42700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a201023c0 con 0x7f8a20102870 2024-02-26T04:48:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.889+0000 7f8a24ade700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a20102870 0x7f8a201004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.889+0000 7f8a24ade700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a20102870 0x7f8a201004d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47102/0 (socket says 172.21.15.60:47102) 2024-02-26T04:48:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.889+0000 7f8a24ade700 1 -- 172.21.15.60:0/4059763704 learned_addr learned my addr 172.21.15.60:0/4059763704 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.890+0000 7f8a24ade700 1 -- 172.21.15.60:0/4059763704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a140095f0 con 0x7f8a20102870 2024-02-26T04:48:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.890+0000 7f8a24ade700 1 --2- 172.21.15.60:0/4059763704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a20102870 0x7f8a201004d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005850 tx=0x7f8a14004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.890+0000 7f8a1dffb700 1 -- 172.21.15.60:0/4059763704 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a1401d070 con 0x7f8a20102870 2024-02-26T04:48:44.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.890+0000 7f8a1dffb700 1 -- 172.21.15.60:0/4059763704 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a1400bc30 con 0x7f8a20102870 2024-02-26T04:48:44.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.890+0000 7f8a26d42700 1 -- 172.21.15.60:0/4059763704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a20100a10 con 0x7f8a20102870 2024-02-26T04:48:44.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.890+0000 7f8a1dffb700 1 -- 172.21.15.60:0/4059763704 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a14003a20 con 0x7f8a20102870 2024-02-26T04:48:44.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.890+0000 7f8a26d42700 1 -- 172.21.15.60:0/4059763704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a20100e30 con 0x7f8a20102870 2024-02-26T04:48:44.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.891+0000 7f8a1dffb700 1 -- 172.21.15.60:0/4059763704 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f8a14003c90 con 0x7f8a20102870 2024-02-26T04:48:44.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.891+0000 7f8a26d42700 1 -- 172.21.15.60:0/4059763704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a200632a0 con 0x7f8a20102870 2024-02-26T04:48:44.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.891+0000 7f8a1dffb700 1 --2- 172.21.15.60:0/4059763704 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a10038f80 0x7f8a1003b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:44.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.891+0000 7f8a1dffb700 1 -- 172.21.15.60:0/4059763704 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8a1404cf10 con 0x7f8a20102870 2024-02-26T04:48:44.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.891+0000 7f8a1ffff700 1 --2- 172.21.15.60:0/4059763704 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a10038f80 0x7f8a1003b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:44.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.892+0000 7f8a1ffff700 1 --2- 172.21.15.60:0/4059763704 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a10038f80 0x7f8a1003b440 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c006fd0 tx=0x7f8a0c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:44.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:44.896+0000 7f8a1dffb700 1 -- 172.21.15.60:0/4059763704 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8a1400fd00 con 0x7f8a20102870 2024-02-26T04:48:45.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:45.103+0000 7f8a26d42700 1 -- 172.21.15.60:0/4059763704 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a20101150 con 0x7f8a10038f80 2024-02-26T04:48:45.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:45 smithi060 ceph-mon[32382]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:48:45.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:45 smithi060 ceph-mon[32382]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:48:45.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:45 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:45.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:45 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:45.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:45 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:45.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:45 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:45.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:45 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:48:45.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:45 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:48:45.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:45 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:48:46.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:46 smithi060 ceph-mon[32382]: Updating smithi156:/etc/ceph/ceph.conf 2024-02-26T04:48:46.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:46 smithi060 ceph-mon[32382]: Updating smithi156:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:48:46.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:46 smithi060 ceph-mon[32382]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-02-26T04:48:46.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:46 smithi060 ceph-mon[32382]: Deploying cephadm binary to smithi191 2024-02-26T04:48:46.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:46 smithi060 ceph-mon[32382]: Updating smithi156:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T04:48:46.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:46.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:46.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:47.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:47 smithi060 ceph-mon[32382]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:48:49.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:49.564+0000 7f8a1dffb700 1 -- 172.21.15.60:0/4059763704 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8a20101150 con 0x7f8a10038f80 2024-02-26T04:48:49.568 INFO:teuthology.orchestra.run.smithi060.stdout:Added host 'smithi191' with addr '172.21.15.191' 2024-02-26T04:48:49.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:49.567+0000 7f8a26d42700 1 -- 172.21.15.60:0/4059763704 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a10038f80 msgr2=0x7f8a1003b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:49.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:49.567+0000 7f8a26d42700 1 --2- 172.21.15.60:0/4059763704 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a10038f80 0x7f8a1003b440 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c006fd0 tx=0x7f8a0c006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:49.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:49.567+0000 7f8a26d42700 1 -- 172.21.15.60:0/4059763704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a20102870 msgr2=0x7f8a201004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:49.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:49.567+0000 7f8a26d42700 1 --2- 172.21.15.60:0/4059763704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a20102870 0x7f8a201004d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005850 tx=0x7f8a14004ff0 comp rx=0 tx=0).stop 2024-02-26T04:48:49.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:49.567+0000 7f8a26d42700 1 -- 172.21.15.60:0/4059763704 shutdown_connections 2024-02-26T04:48:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:49.567+0000 7f8a26d42700 1 --2- 172.21.15.60:0/4059763704 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a10038f80 0x7f8a1003b440 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:49.567+0000 7f8a26d42700 1 --2- 172.21.15.60:0/4059763704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a20102870 0x7f8a201004d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:49.567+0000 7f8a26d42700 1 -- 172.21.15.60:0/4059763704 >> 172.21.15.60:0/4059763704 conn(0x7f8a200fc040 msgr2=0x7f8a20103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:49.567+0000 7f8a26d42700 1 -- 172.21.15.60:0/4059763704 shutdown_connections 2024-02-26T04:48:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:49.567+0000 7f8a26d42700 1 -- 172.21.15.60:0/4059763704 wait complete. 2024-02-26T04:48:49.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:49 smithi060 ceph-mon[32382]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:48:50.133 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch host ls --format=json 2024-02-26T04:48:50.375 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:48:50.818 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:50.819 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:50 smithi060 ceph-mon[32382]: Added host smithi191 2024-02-26T04:48:50.819 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:48:50.819 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:51.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:51 smithi060 ceph-mon[32382]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:48:51.884 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:51 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:51.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.905+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/3754434784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc106610 msgr2=0x7ff6dc1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:51.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.905+0000 7ff6e3f7c700 1 --2- 172.21.15.60:0/3754434784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc106610 0x7ff6dc1069f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400b0a0 tx=0x7ff6d400b3b0 comp rx=0 tx=0).stop 2024-02-26T04:48:51.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.905+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/3754434784 shutdown_connections 2024-02-26T04:48:51.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.905+0000 7ff6e3f7c700 1 --2- 172.21.15.60:0/3754434784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc106610 0x7ff6dc1069f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:51.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.905+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/3754434784 >> 172.21.15.60:0/3754434784 conn(0x7ff6dc0fc030 msgr2=0x7ff6dc0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:51.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.905+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/3754434784 shutdown_connections 2024-02-26T04:48:51.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.905+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/3754434784 wait complete. 2024-02-26T04:48:51.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.906+0000 7ff6e3f7c700 1 Processor -- start 2024-02-26T04:48:51.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.906+0000 7ff6e3f7c700 1 -- start start 2024-02-26T04:48:51.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.906+0000 7ff6e3f7c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc106610 0x7ff6dc197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:51.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.906+0000 7ff6e3f7c700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc06aa20 con 0x7ff6dc106610 2024-02-26T04:48:51.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.906+0000 7ff6e1d18700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc106610 0x7ff6dc197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:51.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.906+0000 7ff6e1d18700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc106610 0x7ff6dc197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60398/0 (socket says 172.21.15.60:60398) 2024-02-26T04:48:51.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.906+0000 7ff6e1d18700 1 -- 172.21.15.60:0/147117153 learned_addr learned my addr 172.21.15.60:0/147117153 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:51.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.906+0000 7ff6e1d18700 1 -- 172.21.15.60:0/147117153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d4009d00 con 0x7ff6dc106610 2024-02-26T04:48:51.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.906+0000 7ff6e1d18700 1 --2- 172.21.15.60:0/147117153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc106610 0x7ff6dc197c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4015040 tx=0x7ff6d400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:51.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.907+0000 7ff6daffd700 1 -- 172.21.15.60:0/147117153 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6d400e040 con 0x7ff6dc106610 2024-02-26T04:48:51.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.907+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/147117153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6dc198270 con 0x7ff6dc106610 2024-02-26T04:48:51.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.907+0000 7ff6daffd700 1 -- 172.21.15.60:0/147117153 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6d4003c10 con 0x7ff6dc106610 2024-02-26T04:48:51.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.907+0000 7ff6daffd700 1 -- 172.21.15.60:0/147117153 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6d401d5e0 con 0x7ff6dc106610 2024-02-26T04:48:51.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.907+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/147117153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6dc1986f0 con 0x7ff6dc106610 2024-02-26T04:48:51.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.907+0000 7ff6daffd700 1 -- 172.21.15.60:0/147117153 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7ff6d4019070 con 0x7ff6dc106610 2024-02-26T04:48:51.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.907+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/147117153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6dc0632a0 con 0x7ff6dc106610 2024-02-26T04:48:51.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.908+0000 7ff6daffd700 1 --2- 172.21.15.60:0/147117153 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6c4038f80 0x7ff6c403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:51.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.908+0000 7ff6daffd700 1 -- 172.21.15.60:0/147117153 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff6d404d100 con 0x7ff6dc106610 2024-02-26T04:48:51.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.908+0000 7ff6e1517700 1 --2- 172.21.15.60:0/147117153 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6c4038f80 0x7ff6c403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:51.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.908+0000 7ff6e1517700 1 --2- 172.21.15.60:0/147117153 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6c4038f80 0x7ff6c403b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0006fd0 tx=0x7ff6d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:51.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:51.913+0000 7ff6daffd700 1 -- 172.21.15.60:0/147117153 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff6d401c3d0 con 0x7ff6dc106610 2024-02-26T04:48:52.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:52.123+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/147117153 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff6dc198a90 con 0x7ff6c4038f80 2024-02-26T04:48:52.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:52.124+0000 7ff6daffd700 1 -- 172.21.15.60:0/147117153 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7ff6dc198a90 con 0x7ff6c4038f80 2024-02-26T04:48:52.127 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:48:52.128 INFO:teuthology.orchestra.run.smithi060.stdout:[{"addr": "172.21.15.60", "hostname": "smithi060", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}, {"addr": "172.21.15.191", "hostname": "smithi191", "labels": [], "status": ""}] 2024-02-26T04:48:52.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:52.126+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/147117153 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6c4038f80 msgr2=0x7ff6c403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:52.126+0000 7ff6e3f7c700 1 --2- 172.21.15.60:0/147117153 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6c4038f80 0x7ff6c403b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0006fd0 tx=0x7ff6d0006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:52.126+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/147117153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc106610 msgr2=0x7ff6dc197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:52.126+0000 7ff6e3f7c700 1 --2- 172.21.15.60:0/147117153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc106610 0x7ff6dc197c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4015040 tx=0x7ff6d400bd40 comp rx=0 tx=0).stop 2024-02-26T04:48:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:52.126+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/147117153 shutdown_connections 2024-02-26T04:48:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:52.126+0000 7ff6e3f7c700 1 --2- 172.21.15.60:0/147117153 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6c4038f80 0x7ff6c403b440 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:52.126+0000 7ff6e3f7c700 1 --2- 172.21.15.60:0/147117153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc106610 0x7ff6dc197c70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:52.126+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/147117153 >> 172.21.15.60:0/147117153 conn(0x7ff6dc0fc030 msgr2=0x7ff6dc0fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:52.126+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/147117153 shutdown_connections 2024-02-26T04:48:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:52.126+0000 7ff6e3f7c700 1 -- 172.21.15.60:0/147117153 wait complete. 2024-02-26T04:48:52.742 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-26T04:48:52.742 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd crush tunables default 2024-02-26T04:48:52.981 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:48:53.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:53 smithi060 ceph-mon[32382]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:48:53.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:53 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.314+0000 7f145e4c8700 1 -- 172.21.15.60:0/2214788565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1458107650 msgr2=0x7f1458107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.314+0000 7f145e4c8700 1 --2- 172.21.15.60:0/2214788565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1458107650 0x7f1458107a30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1440009910 tx=0x7f1440009c20 comp rx=0 tx=0).stop 2024-02-26T04:48:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.314+0000 7f145e4c8700 1 -- 172.21.15.60:0/2214788565 shutdown_connections 2024-02-26T04:48:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.314+0000 7f145e4c8700 1 --2- 172.21.15.60:0/2214788565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1458107650 0x7f1458107a30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.314+0000 7f145e4c8700 1 -- 172.21.15.60:0/2214788565 >> 172.21.15.60:0/2214788565 conn(0x7f14580fb7c0 msgr2=0x7f14580fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.314+0000 7f145e4c8700 1 -- 172.21.15.60:0/2214788565 shutdown_connections 2024-02-26T04:48:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.314+0000 7f145e4c8700 1 -- 172.21.15.60:0/2214788565 wait complete. 2024-02-26T04:48:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.314+0000 7f145e4c8700 1 Processor -- start 2024-02-26T04:48:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.315+0000 7f145e4c8700 1 -- start start 2024-02-26T04:48:54.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.315+0000 7f145e4c8700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1458107650 0x7f1458196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:54.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.315+0000 7f145e4c8700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14581974f0 con 0x7f1458107650 2024-02-26T04:48:54.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.315+0000 7f1457fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1458107650 0x7f1458196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:54.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.315+0000 7f1457fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1458107650 0x7f1458196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60412/0 (socket says 172.21.15.60:60412) 2024-02-26T04:48:54.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.315+0000 7f1457fff700 1 -- 172.21.15.60:0/787424222 learned_addr learned my addr 172.21.15.60:0/787424222 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:48:54.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.315+0000 7f1457fff700 1 -- 172.21.15.60:0/787424222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14400095f0 con 0x7f1458107650 2024-02-26T04:48:54.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.315+0000 7f1457fff700 1 --2- 172.21.15.60:0/787424222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1458107650 0x7f1458196fb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1440004f40 tx=0x7f1440005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:54.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.316+0000 7f14557fa700 1 -- 172.21.15.60:0/787424222 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f144001d070 con 0x7f1458107650 2024-02-26T04:48:54.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.316+0000 7f14557fa700 1 -- 172.21.15.60:0/787424222 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f144000bbd0 con 0x7f1458107650 2024-02-26T04:48:54.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.316+0000 7f14557fa700 1 -- 172.21.15.60:0/787424222 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1440003900 con 0x7f1458107650 2024-02-26T04:48:54.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.316+0000 7f145e4c8700 1 -- 172.21.15.60:0/787424222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1458197750 con 0x7f1458107650 2024-02-26T04:48:54.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.316+0000 7f145e4c8700 1 -- 172.21.15.60:0/787424222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1458197bd0 con 0x7f1458107650 2024-02-26T04:48:54.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.316+0000 7f14557fa700 1 -- 172.21.15.60:0/787424222 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f144000bd40 con 0x7f1458107650 2024-02-26T04:48:54.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.317+0000 7f145e4c8700 1 -- 172.21.15.60:0/787424222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14580632a0 con 0x7f1458107650 2024-02-26T04:48:54.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.317+0000 7f14557fa700 1 --2- 172.21.15.60:0/787424222 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1444038f80 0x7f144403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:54.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.317+0000 7f14557fa700 1 -- 172.21.15.60:0/787424222 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f144004cd90 con 0x7f1458107650 2024-02-26T04:48:54.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.317+0000 7f14577fe700 1 --2- 172.21.15.60:0/787424222 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1444038f80 0x7f144403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:54.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.317+0000 7f14577fe700 1 --2- 172.21.15.60:0/787424222 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1444038f80 0x7f144403b440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1448006fd0 tx=0x7f1448006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.322+0000 7f14557fa700 1 -- 172.21.15.60:0/787424222 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f144002ae20 con 0x7f1458107650 2024-02-26T04:48:54.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:54 smithi060 ceph-mon[32382]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:48:54.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:54.530+0000 7f145e4c8700 1 -- 172.21.15.60:0/787424222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f145810bbd0 con 0x7f1458107650 2024-02-26T04:48:55.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:55.127+0000 7f14557fa700 1 -- 172.21.15.60:0/787424222 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f144000e600 con 0x7f1458107650 2024-02-26T04:48:55.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:55.131+0000 7f145e4c8700 1 -- 172.21.15.60:0/787424222 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1444038f80 msgr2=0x7f144403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:55.131+0000 7f145e4c8700 1 --2- 172.21.15.60:0/787424222 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1444038f80 0x7f144403b440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1448006fd0 tx=0x7f1448006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:55.131+0000 7f145e4c8700 1 -- 172.21.15.60:0/787424222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1458107650 msgr2=0x7f1458196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:55.131+0000 7f145e4c8700 1 --2- 172.21.15.60:0/787424222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1458107650 0x7f1458196fb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1440004f40 tx=0x7f1440005020 comp rx=0 tx=0).stop 2024-02-26T04:48:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:55.131+0000 7f145e4c8700 1 -- 172.21.15.60:0/787424222 shutdown_connections 2024-02-26T04:48:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:55.131+0000 7f145e4c8700 1 --2- 172.21.15.60:0/787424222 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1444038f80 0x7f144403b440 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:55.131+0000 7f145e4c8700 1 --2- 172.21.15.60:0/787424222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1458107650 0x7f1458196fb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:55.131+0000 7f145e4c8700 1 -- 172.21.15.60:0/787424222 >> 172.21.15.60:0/787424222 conn(0x7f14580fb7c0 msgr2=0x7f14580fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:55.131+0000 7f145e4c8700 1 -- 172.21.15.60:0/787424222 shutdown_connections 2024-02-26T04:48:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:48:55.131+0000 7f145e4c8700 1 -- 172.21.15.60:0/787424222 wait complete. 2024-02-26T04:48:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:adjusted tunables profile to default 2024-02-26T04:48:55.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:55 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/787424222' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-26T04:48:55.773 INFO:tasks.cephadm:Adding mon.a on smithi060 2024-02-26T04:48:55.773 INFO:tasks.cephadm:Adding mon.b on smithi156 2024-02-26T04:48:55.773 INFO:tasks.cephadm:Adding mon.c on smithi191 2024-02-26T04:48:55.773 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch apply mon '3;smithi060:172.21.15.60=a;smithi156:172.21.15.156=b;smithi191:172.21.15.191=c' 2024-02-26T04:48:56.009 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:48:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:56 smithi060 ceph-mon[32382]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:48:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:56 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/787424222' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-26T04:48:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:56 smithi060 ceph-mon[32382]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T04:48:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:48:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:48:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:48:57.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:57 smithi060 ceph-mon[32382]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-26T04:48:57.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:57 smithi060 ceph-mon[32382]: Updating smithi191:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:48:57.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:57 smithi060 ceph-mon[32382]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-02-26T04:48:57.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:57 smithi060 ceph-mon[32382]: Updating smithi191:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T04:48:57.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:57 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:57.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:57 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:57.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:57 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:57.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.591+0000 7f8e67a43700 1 -- 172.21.15.191:0/731839977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e600fff30 msgr2=0x7f8e60100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:57.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.591+0000 7f8e67a43700 1 --2- 172.21.15.191:0/731839977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e600fff30 0x7f8e60100310 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00aa20 tx=0x7f8e5c00ad30 comp rx=0 tx=0).stop 2024-02-26T04:48:57.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.592+0000 7f8e67a43700 1 -- 172.21.15.191:0/731839977 shutdown_connections 2024-02-26T04:48:57.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.592+0000 7f8e67a43700 1 --2- 172.21.15.191:0/731839977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e600fff30 0x7f8e60100310 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:57.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.592+0000 7f8e67a43700 1 -- 172.21.15.191:0/731839977 >> 172.21.15.191:0/731839977 conn(0x7f8e600fb7c0 msgr2=0x7f8e600fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:57.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.592+0000 7f8e67a43700 1 -- 172.21.15.191:0/731839977 shutdown_connections 2024-02-26T04:48:57.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.592+0000 7f8e67a43700 1 -- 172.21.15.191:0/731839977 wait complete. 2024-02-26T04:48:57.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.592+0000 7f8e67a43700 1 Processor -- start 2024-02-26T04:48:57.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.592+0000 7f8e67a43700 1 -- start start 2024-02-26T04:48:57.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.592+0000 7f8e67a43700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e600fff30 0x7f8e601019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:57.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.592+0000 7f8e67a43700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e601038c0 con 0x7f8e600fff30 2024-02-26T04:48:57.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.593+0000 7f8e657df700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e600fff30 0x7f8e601019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:57.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.593+0000 7f8e657df700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e600fff30 0x7f8e601019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.191:46612/0 (socket says 172.21.15.191:46612) 2024-02-26T04:48:57.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.593+0000 7f8e657df700 1 -- 172.21.15.191:0/3947642087 learned_addr learned my addr 172.21.15.191:0/3947642087 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-26T04:48:57.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.593+0000 7f8e657df700 1 -- 172.21.15.191:0/3947642087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c00a700 con 0x7f8e600fff30 2024-02-26T04:48:57.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.593+0000 7f8e657df700 1 --2- 172.21.15.191:0/3947642087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e600fff30 0x7f8e601019d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c005b40 tx=0x7f8e5c004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:57.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.593+0000 7f8e567fc700 1 -- 172.21.15.191:0/3947642087 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e5c01d070 con 0x7f8e600fff30 2024-02-26T04:48:57.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.593+0000 7f8e67a43700 1 -- 172.21.15.191:0/3947642087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e60101f10 con 0x7f8e600fff30 2024-02-26T04:48:57.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.593+0000 7f8e567fc700 1 -- 172.21.15.191:0/3947642087 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e5c016470 con 0x7f8e600fff30 2024-02-26T04:48:57.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.593+0000 7f8e567fc700 1 -- 172.21.15.191:0/3947642087 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e5c00e5e0 con 0x7f8e600fff30 2024-02-26T04:48:57.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.593+0000 7f8e67a43700 1 -- 172.21.15.191:0/3947642087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e60102330 con 0x7f8e600fff30 2024-02-26T04:48:57.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.594+0000 7f8e67a43700 1 -- 172.21.15.191:0/3947642087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e600632a0 con 0x7f8e600fff30 2024-02-26T04:48:57.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.594+0000 7f8e567fc700 1 -- 172.21.15.191:0/3947642087 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f8e5c005340 con 0x7f8e600fff30 2024-02-26T04:48:57.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.594+0000 7f8e567fc700 1 --2- 172.21.15.191:0/3947642087 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8e4c038f80 0x7f8e4c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:48:57.598 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.595+0000 7f8e567fc700 1 -- 172.21.15.191:0/3947642087 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8e5c012070 con 0x7f8e600fff30 2024-02-26T04:48:57.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.595+0000 7f8e64fde700 1 --2- 172.21.15.191:0/3947642087 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8e4c038f80 0x7f8e4c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:48:57.599 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.595+0000 7f8e64fde700 1 --2- 172.21.15.191:0/3947642087 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8e4c038f80 0x7f8e4c03b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e50006fd0 tx=0x7f8e50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:48:57.602 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.600+0000 7f8e567fc700 1 -- 172.21.15.191:0/3947642087 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8e5c014550 con 0x7f8e600fff30 2024-02-26T04:48:57.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.804+0000 7f8e67a43700 1 -- 172.21.15.191:0/3947642087 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi156:172.21.15.156=b;smithi191:172.21.15.191=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e60102650 con 0x7f8e4c038f80 2024-02-26T04:48:57.813 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.811+0000 7f8e567fc700 1 -- 172.21.15.191:0/3947642087 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8e60102650 con 0x7f8e4c038f80 2024-02-26T04:48:57.814 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mon update... 2024-02-26T04:48:57.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.813+0000 7f8e67a43700 1 -- 172.21.15.191:0/3947642087 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8e4c038f80 msgr2=0x7f8e4c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:57.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.813+0000 7f8e67a43700 1 --2- 172.21.15.191:0/3947642087 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8e4c038f80 0x7f8e4c03b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e50006fd0 tx=0x7f8e50006e40 comp rx=0 tx=0).stop 2024-02-26T04:48:57.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.813+0000 7f8e67a43700 1 -- 172.21.15.191:0/3947642087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e600fff30 msgr2=0x7f8e601019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:48:57.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.813+0000 7f8e67a43700 1 --2- 172.21.15.191:0/3947642087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e600fff30 0x7f8e601019d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c005b40 tx=0x7f8e5c004da0 comp rx=0 tx=0).stop 2024-02-26T04:48:57.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.813+0000 7f8e67a43700 1 -- 172.21.15.191:0/3947642087 shutdown_connections 2024-02-26T04:48:57.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.813+0000 7f8e67a43700 1 --2- 172.21.15.191:0/3947642087 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8e4c038f80 0x7f8e4c03b440 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:57.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.813+0000 7f8e67a43700 1 --2- 172.21.15.191:0/3947642087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e600fff30 0x7f8e601019d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:48:57.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.813+0000 7f8e67a43700 1 -- 172.21.15.191:0/3947642087 >> 172.21.15.191:0/3947642087 conn(0x7f8e600fb7c0 msgr2=0x7f8e60107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:48:57.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.813+0000 7f8e67a43700 1 -- 172.21.15.191:0/3947642087 shutdown_connections 2024-02-26T04:48:57.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:48:57.813+0000 7f8e67a43700 1 -- 172.21.15.191:0/3947642087 wait complete. 2024-02-26T04:48:58.411 DEBUG:teuthology.orchestra.run.smithi156:mon.b> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.b.service 2024-02-26T04:48:58.415 DEBUG:teuthology.orchestra.run.smithi191:mon.c> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.c.service 2024-02-26T04:48:58.418 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-26T04:48:58.418 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-26T04:48:58.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:58 smithi060 ceph-mon[32382]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:48:58.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:58.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:48:58.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:48:58.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:48:58.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:48:58.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T04:48:58.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:48:59.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:59 smithi060 ceph-mon[32382]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi156:172.21.15.156=b;smithi191:172.21.15.191=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:48:59.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:59 smithi060 ceph-mon[32382]: Saving service mon spec with placement smithi060:172.21.15.60=a;smithi156:172.21.15.156=b;smithi191:172.21.15.191=c;count:3 2024-02-26T04:48:59.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:48:59 smithi060 ceph-mon[32382]: Deploying daemon mon.c on smithi191 2024-02-26T04:49:00.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:00 smithi060 ceph-mon[32382]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:00.507 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T04:49:02.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:02 smithi060 ceph-mon[32382]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:02.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.508+0000 7f877a11f700 1 -- 172.21.15.191:0/2371461186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8774102930 msgr2=0x7f8774102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:02.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.508+0000 7f877a11f700 1 --2- 172.21.15.191:0/2371461186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8774102930 0x7f8774102d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8764009910 tx=0x7f8764009c20 comp rx=0 tx=0).stop 2024-02-26T04:49:02.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.508+0000 7f877a11f700 1 -- 172.21.15.191:0/2371461186 shutdown_connections 2024-02-26T04:49:02.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.508+0000 7f877a11f700 1 --2- 172.21.15.191:0/2371461186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8774102930 0x7f8774102d10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:02.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.508+0000 7f877a11f700 1 -- 172.21.15.191:0/2371461186 >> 172.21.15.191:0/2371461186 conn(0x7f87740fc180 msgr2=0x7f87740fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:49:02.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.508+0000 7f877a11f700 1 -- 172.21.15.191:0/2371461186 shutdown_connections 2024-02-26T04:49:02.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.508+0000 7f877a11f700 1 -- 172.21.15.191:0/2371461186 wait complete. 2024-02-26T04:49:02.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.509+0000 7f877a11f700 1 Processor -- start 2024-02-26T04:49:02.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.509+0000 7f877a11f700 1 -- start start 2024-02-26T04:49:02.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.509+0000 7f877a11f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8774102930 0x7f8774197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:02.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.509+0000 7f877a11f700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87741976b0 con 0x7f8774102930 2024-02-26T04:49:02.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.509+0000 7f87737fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8774102930 0x7f8774197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:02.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.509+0000 7f87737fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8774102930 0x7f8774197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.191:46626/0 (socket says 172.21.15.191:46626) 2024-02-26T04:49:02.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.509+0000 7f87737fe700 1 -- 172.21.15.191:0/400432231 learned_addr learned my addr 172.21.15.191:0/400432231 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-26T04:49:02.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.510+0000 7f87737fe700 1 -- 172.21.15.191:0/400432231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87640095f0 con 0x7f8774102930 2024-02-26T04:49:02.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.510+0000 7f87737fe700 1 --2- 172.21.15.191:0/400432231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8774102930 0x7f8774197170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8764005850 tx=0x7f8764004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:49:02.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.510+0000 7f8770ff9700 1 -- 172.21.15.191:0/400432231 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f876401d070 con 0x7f8774102930 2024-02-26T04:49:02.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.510+0000 7f877a11f700 1 -- 172.21.15.191:0/400432231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87741978b0 con 0x7f8774102930 2024-02-26T04:49:02.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.510+0000 7f8770ff9700 1 -- 172.21.15.191:0/400432231 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f876400bc30 con 0x7f8774102930 2024-02-26T04:49:02.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.510+0000 7f8770ff9700 1 -- 172.21.15.191:0/400432231 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8764003a20 con 0x7f8774102930 2024-02-26T04:49:02.514 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.510+0000 7f877a11f700 1 -- 172.21.15.191:0/400432231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8774197cd0 con 0x7f8774102930 2024-02-26T04:49:02.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.511+0000 7f877a11f700 1 -- 172.21.15.191:0/400432231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87740632a0 con 0x7f8774102930 2024-02-26T04:49:02.515 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.511+0000 7f8770ff9700 1 -- 172.21.15.191:0/400432231 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f8764003c90 con 0x7f8774102930 2024-02-26T04:49:02.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.511+0000 7f8770ff9700 1 --2- 172.21.15.191:0/400432231 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f875c038f80 0x7f875c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:02.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.511+0000 7f8770ff9700 1 -- 172.21.15.191:0/400432231 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f876404cdf0 con 0x7f8774102930 2024-02-26T04:49:02.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.512+0000 7f8772ffd700 1 --2- 172.21.15.191:0/400432231 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f875c038f80 0x7f875c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:02.516 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.512+0000 7f8772ffd700 1 --2- 172.21.15.191:0/400432231 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f875c038f80 0x7f875c03b440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8768006fd0 tx=0x7f8768006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:49:02.519 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.517+0000 7f8770ff9700 1 -- 172.21.15.191:0/400432231 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f876400b6f0 con 0x7f8774102930 2024-02-26T04:49:02.747 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: mon.c@-1(???) e0 preinit fsid f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:49:02.748 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: mon.c@-1(synchronizing).mds e1 new map 2024-02-26T04:49:02.748 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-26T04:49:02.748 INFO:journalctl@ceph.mon.c.smithi191.stdout: e1 2024-02-26T04:49:02.748 INFO:journalctl@ceph.mon.c.smithi191.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-26T04:49:02.748 INFO:journalctl@ceph.mon.c.smithi191.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T04:49:02.748 INFO:journalctl@ceph.mon.c.smithi191.stdout: legacy client fscid: -1 2024-02-26T04:49:02.748 INFO:journalctl@ceph.mon.c.smithi191.stdout: 2024-02-26T04:49:02.748 INFO:journalctl@ceph.mon.c.smithi191.stdout: No filesystems configured 2024-02-26T04:49:02.748 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-26T04:49:02.748 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-26T04:49:02.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.751+0000 7f8770ff9700 1 -- 172.21.15.191:0/400432231 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f876400b6f0 con 0x7f8774102930 2024-02-26T04:49:02.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:02.818+0000 7f877a11f700 1 -- 172.21.15.191:0/400432231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f877410bc40 con 0x7f8774102930 2024-02-26T04:49:03.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-26T04:49:03.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-26T04:49:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-26T04:49:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-26T04:49:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-26T04:49:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T04:49:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T04:49:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-26T04:49:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:49:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:49:03.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:49:03.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:03.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:03.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: Updating smithi156:/etc/ceph/ceph.conf 2024-02-26T04:49:03.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: Updating smithi156:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:49:03.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-02-26T04:49:03.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: Deploying cephadm binary to smithi191 2024-02-26T04:49:03.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: Updating smithi156:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T04:49:03.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:03.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:03.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: Added host smithi191 2024-02-26T04:49:03.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:03.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:03.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:49:03.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.013 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:03.013 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/787424222' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-26T04:49:03.013 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:03.013 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/787424222' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-26T04:49:03.013 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T04:49:03.013 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.013 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.013 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.013 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.013 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:49:03.013 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:03.013 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:03.013 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-26T04:49:03.013 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: Updating smithi191:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:49:03.014 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-02-26T04:49:03.014 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: Updating smithi191:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.client.admin.keyring 2024-02-26T04:49:03.014 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.014 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.014 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.014 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:03.014 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.014 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:03.014 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:03.014 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:03.014 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:03.014 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T04:49:03.014 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:03.014 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi156:172.21.15.156=b;smithi191:172.21.15.191=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:49:03.015 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: Saving service mon spec with placement smithi060:172.21.15.60=a;smithi156:172.21.15.156=b;smithi191:172.21.15.191=c;count:3 2024-02-26T04:49:03.015 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: Deploying daemon mon.c on smithi191 2024-02-26T04:49:03.015 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:03.015 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:03.015 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 ceph-mon[36352]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-26T04:49:03.015 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 bash[36304]: 462f964553c96d22fff62bdc55db613ec249122b2b18ba64d9e5142730c8c3e2 2024-02-26T04:49:03.015 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:02 smithi191 systemd[1]: Started Ceph mon.c for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:49:07.800 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:07.797+0000 7f8770ff9700 1 -- 172.21.15.191:0/400432231 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f8764026070 con 0x7f8774102930 2024-02-26T04:49:07.800 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:49:07.800 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":2,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","modified":"2024-02-26T04:49:02.751658Z","created":"2024-02-26T04:47:09.985932Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-26T04:49:07.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:07.800+0000 7f877a11f700 1 -- 172.21.15.191:0/400432231 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f875c038f80 msgr2=0x7f875c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:07.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:07.800+0000 7f877a11f700 1 --2- 172.21.15.191:0/400432231 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f875c038f80 0x7f875c03b440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8768006fd0 tx=0x7f8768006e40 comp rx=0 tx=0).stop 2024-02-26T04:49:07.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:07.800+0000 7f877a11f700 1 -- 172.21.15.191:0/400432231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8774102930 msgr2=0x7f8774197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:07.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:07.800+0000 7f877a11f700 1 --2- 172.21.15.191:0/400432231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8774102930 0x7f8774197170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8764005850 tx=0x7f8764004ff0 comp rx=0 tx=0).stop 2024-02-26T04:49:07.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:07.800+0000 7f877a11f700 1 -- 172.21.15.191:0/400432231 shutdown_connections 2024-02-26T04:49:07.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:07.800+0000 7f877a11f700 1 --2- 172.21.15.191:0/400432231 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f875c038f80 0x7f875c03b440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:07.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:07.800+0000 7f877a11f700 1 --2- 172.21.15.191:0/400432231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8774102930 0x7f8774197170 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:07.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:07.800+0000 7f877a11f700 1 -- 172.21.15.191:0/400432231 >> 172.21.15.191:0/400432231 conn(0x7f87740fc180 msgr2=0x7f87741056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:49:07.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:07.800+0000 7f877a11f700 1 -- 172.21.15.191:0/400432231 shutdown_connections 2024-02-26T04:49:07.804 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:07.800+0000 7f877a11f700 1 -- 172.21.15.191:0/400432231 wait complete. 2024-02-26T04:49:07.804 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 2 2024-02-26T04:49:08.111 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:08.111 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:49:08.111 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: mon.a calling monitor election 2024-02-26T04:49:08.112 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:08.112 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: from='client.? 172.21.15.191:0/400432231' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-26T04:49:08.112 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:08.112 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:08.112 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:08.112 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: mon.c calling monitor election 2024-02-26T04:49:08.112 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:08.112 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:08.112 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:08.112 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:08.112 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T04:49:08.112 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: monmap e2: 2 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T04:49:08.113 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: fsmap 2024-02-26T04:49:08.113 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T04:49:08.113 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: mgrmap e13: x(active, since 49s) 2024-02-26T04:49:08.113 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: overall HEALTH_OK 2024-02-26T04:49:08.113 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:08.113 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:08.168 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:08.169 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:49:08.169 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: mon.a calling monitor election 2024-02-26T04:49:08.169 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:08.169 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: from='client.? 172.21.15.191:0/400432231' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-26T04:49:08.169 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:08.169 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:08.169 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:08.169 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: mon.c calling monitor election 2024-02-26T04:49:08.169 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:08.169 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:08.170 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:08.170 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:08.170 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T04:49:08.170 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: monmap e2: 2 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T04:49:08.170 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: fsmap 2024-02-26T04:49:08.170 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T04:49:08.170 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: mgrmap e13: x(active, since 49s) 2024-02-26T04:49:08.170 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: overall HEALTH_OK 2024-02-26T04:49:08.170 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:08.170 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:09.138 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:09.138 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T04:49:09.138 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:09.138 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:08 smithi060 ceph-mon[32382]: Deploying daemon mon.b on smithi156 2024-02-26T04:49:09.139 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:09.151 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:09.151 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T04:49:09.151 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:09.151 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:08 smithi191 ceph-mon[36352]: Deploying daemon mon.b on smithi156 2024-02-26T04:49:09.151 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:09.407 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-26T04:49:09.408 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-26T04:49:09.652 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T04:49:09.999 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:09 smithi191 ceph-mon[36352]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:10.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:09 smithi060 ceph-mon[32382]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:11.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.096+0000 7fadb2cc9700 1 -- 172.21.15.191:0/37526509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadac106670 msgr2=0x7fadac106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:11.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.096+0000 7fadb2cc9700 1 --2- 172.21.15.191:0/37526509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadac106670 0x7fadac106a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fada0009910 tx=0x7fada0009c20 comp rx=0 tx=0).stop 2024-02-26T04:49:11.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.096+0000 7fadb2cc9700 1 -- 172.21.15.191:0/37526509 shutdown_connections 2024-02-26T04:49:11.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.096+0000 7fadb2cc9700 1 --2- 172.21.15.191:0/37526509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadac106670 0x7fadac106a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:11.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.096+0000 7fadb2cc9700 1 -- 172.21.15.191:0/37526509 >> 172.21.15.191:0/37526509 conn(0x7fadac0fc180 msgr2=0x7fadac0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:49:11.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.096+0000 7fadb2cc9700 1 -- 172.21.15.191:0/37526509 shutdown_connections 2024-02-26T04:49:11.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.096+0000 7fadb2cc9700 1 -- 172.21.15.191:0/37526509 wait complete. 2024-02-26T04:49:11.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.096+0000 7fadb2cc9700 1 Processor -- start 2024-02-26T04:49:11.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.096+0000 7fadb2cc9700 1 -- start start 2024-02-26T04:49:11.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadb2cc9700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadac106670 0x7fadac196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:11.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadb2cc9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac196660 0x7fadac19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:11.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadb2cc9700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac196bf0 con 0x7fadac106670 2024-02-26T04:49:11.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadb2cc9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac196d60 con 0x7fadac196660 2024-02-26T04:49:11.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadabfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac196660 0x7fadac19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:11.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadabfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac196660 0x7fadac19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:38242/0 (socket says 172.21.15.191:38242) 2024-02-26T04:49:11.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadb0a65700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadac106670 0x7fadac196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:11.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadabfff700 1 -- 172.21.15.191:0/2889064099 learned_addr learned my addr 172.21.15.191:0/2889064099 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-26T04:49:11.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadabfff700 1 -- 172.21.15.191:0/2889064099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac196660 msgr2=0x7fadac19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-26T04:49:11.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadabfff700 1 -- 172.21.15.191:0/2889064099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac196660 msgr2=0x7fadac19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-26T04:49:11.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadabfff700 1 --2- 172.21.15.191:0/2889064099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac196660 0x7fadac19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-26T04:49:11.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadabfff700 1 --2- 172.21.15.191:0/2889064099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac196660 0x7fadac19a9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T04:49:11.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadb0a65700 1 -- 172.21.15.191:0/2889064099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac196660 msgr2=0x7fadac19a9f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T04:49:11.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadb0a65700 1 --2- 172.21.15.191:0/2889064099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac196660 0x7fadac19a9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:11.102 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadb0a65700 1 -- 172.21.15.191:0/2889064099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada00095f0 con 0x7fadac106670 2024-02-26T04:49:11.103 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.097+0000 7fadb0a65700 1 --2- 172.21.15.191:0/2889064099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadac106670 0x7fadac196120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fada0014000 tx=0x7fada0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:49:11.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.098+0000 7fada9ffb700 1 -- 172.21.15.191:0/2889064099 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fada001d070 con 0x7fadac106670 2024-02-26T04:49:11.110 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.098+0000 7fadb2cc9700 1 -- 172.21.15.191:0/2889064099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadac19af90 con 0x7fadac106670 2024-02-26T04:49:11.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.098+0000 7fada9ffb700 1 -- 172.21.15.191:0/2889064099 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fada0005dc0 con 0x7fadac106670 2024-02-26T04:49:11.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.098+0000 7fada9ffb700 1 -- 172.21.15.191:0/2889064099 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fada0003880 con 0x7fadac106670 2024-02-26T04:49:11.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.098+0000 7fadb2cc9700 1 -- 172.21.15.191:0/2889064099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadac19b4e0 con 0x7fadac106670 2024-02-26T04:49:11.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.099+0000 7fadb2cc9700 1 -- 172.21.15.191:0/2889064099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadac101db0 con 0x7fadac106670 2024-02-26T04:49:11.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.099+0000 7fada9ffb700 1 -- 172.21.15.191:0/2889064099 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fada000b6f0 con 0x7fadac106670 2024-02-26T04:49:11.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.099+0000 7fada9ffb700 1 --2- 172.21.15.191:0/2889064099 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fad9c038d20 0x7fad9c03b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:11.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.099+0000 7fada9ffb700 1 -- 172.21.15.191:0/2889064099 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fada004cd90 con 0x7fadac106670 2024-02-26T04:49:11.111 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.099+0000 7fadabfff700 1 --2- 172.21.15.191:0/2889064099 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fad9c038d20 0x7fad9c03b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:11.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.100+0000 7fadabfff700 1 --2- 172.21.15.191:0/2889064099 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fad9c038d20 0x7fad9c03b1e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fad98009180 tx=0x7fad98008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:49:11.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.105+0000 7fada9ffb700 1 -- 172.21.15.191:0/2889064099 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fada0029360 con 0x7fadac106670 2024-02-26T04:49:11.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.377+0000 7fadb2cc9700 1 -- 172.21.15.191:0/2889064099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fadac0632a0 con 0x7fadac106670 2024-02-26T04:49:11.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.378+0000 7fada9ffb700 1 -- 172.21.15.191:0/2889064099 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fada002ac90 con 0x7fadac106670 2024-02-26T04:49:11.381 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:49:11.381 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":2,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","modified":"2024-02-26T04:49:02.751658Z","created":"2024-02-26T04:47:09.985932Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-26T04:49:11.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.380+0000 7fadb2cc9700 1 -- 172.21.15.191:0/2889064099 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fad9c038d20 msgr2=0x7fad9c03b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:11.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.380+0000 7fadb2cc9700 1 --2- 172.21.15.191:0/2889064099 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fad9c038d20 0x7fad9c03b1e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fad98009180 tx=0x7fad98008040 comp rx=0 tx=0).stop 2024-02-26T04:49:11.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.380+0000 7fadb2cc9700 1 -- 172.21.15.191:0/2889064099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadac106670 msgr2=0x7fadac196120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:11.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.380+0000 7fadb2cc9700 1 --2- 172.21.15.191:0/2889064099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadac106670 0x7fadac196120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fada0014000 tx=0x7fada0004ff0 comp rx=0 tx=0).stop 2024-02-26T04:49:11.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.380+0000 7fadb2cc9700 1 -- 172.21.15.191:0/2889064099 shutdown_connections 2024-02-26T04:49:11.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.380+0000 7fadb2cc9700 1 --2- 172.21.15.191:0/2889064099 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fad9c038d20 0x7fad9c03b1e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:11.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.380+0000 7fadb2cc9700 1 --2- 172.21.15.191:0/2889064099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadac106670 0x7fadac196120 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:11.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.380+0000 7fadb2cc9700 1 --2- 172.21.15.191:0/2889064099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac196660 0x7fadac19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:11.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.380+0000 7fadb2cc9700 1 -- 172.21.15.191:0/2889064099 >> 172.21.15.191:0/2889064099 conn(0x7fadac0fc180 msgr2=0x7fadac102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:49:11.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.380+0000 7fadb2cc9700 1 -- 172.21.15.191:0/2889064099 shutdown_connections 2024-02-26T04:49:11.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:11.380+0000 7fadb2cc9700 1 -- 172.21.15.191:0/2889064099 wait complete. 2024-02-26T04:49:11.384 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 2 2024-02-26T04:49:11.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:11 smithi156 systemd[1]: Started Ceph mon.b for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:49:12.959 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-26T04:49:12.959 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph mon dump -f json 2024-02-26T04:49:13.200 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T04:49:17.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:49:17.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: mon.c calling monitor election 2024-02-26T04:49:17.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: mon.a calling monitor election 2024-02-26T04:49:17.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:17.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:17.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:17.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:17.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:17.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:17.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:17.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:17.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:17.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T04:49:17.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: monmap e3: 3 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T04:49:17.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: fsmap 2024-02-26T04:49:17.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T04:49:17.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: mgrmap e13: x(active, since 58s) 2024-02-26T04:49:17.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-26T04:49:17.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-26T04:49:17.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-26T04:49:17.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: mon.b (rank 2) addr [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] is down (out of quorum) 2024-02-26T04:49:17.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:17.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:17.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:17.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:17.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:49:17.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: mon.c calling monitor election 2024-02-26T04:49:17.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: mon.a calling monitor election 2024-02-26T04:49:17.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:17.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:17.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:17.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:17.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:17.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:17.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:17.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:17.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:17.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T04:49:17.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: monmap e3: 3 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T04:49:17.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: fsmap 2024-02-26T04:49:17.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T04:49:17.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: mgrmap e13: x(active, since 58s) 2024-02-26T04:49:17.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-26T04:49:17.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-26T04:49:17.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-26T04:49:17.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: mon.b (rank 2) addr [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] is down (out of quorum) 2024-02-26T04:49:17.261 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:17.261 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:17.261 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:17.261 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:17.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.638+0000 7fdc34cdd700 1 -- 172.21.15.191:0/1786976676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc180057d0 msgr2=0x7fdc18007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:17.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.638+0000 7fdc34cdd700 1 --2- 172.21.15.191:0/1786976676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc180057d0 0x7fdc18007c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdc301005a0 tx=0x7fdc28009e80 comp rx=0 tx=0).stop 2024-02-26T04:49:17.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.638+0000 7fdc34cdd700 1 -- 172.21.15.191:0/1786976676 shutdown_connections 2024-02-26T04:49:17.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.638+0000 7fdc34cdd700 1 --2- 172.21.15.191:0/1786976676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc180057d0 0x7fdc18007c60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:17.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.638+0000 7fdc34cdd700 1 -- 172.21.15.191:0/1786976676 >> 172.21.15.191:0/1786976676 conn(0x7fdc300fc180 msgr2=0x7fdc300fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:49:17.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.638+0000 7fdc34cdd700 1 -- 172.21.15.191:0/1786976676 shutdown_connections 2024-02-26T04:49:17.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.638+0000 7fdc34cdd700 1 -- 172.21.15.191:0/1786976676 wait complete. 2024-02-26T04:49:17.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc34cdd700 1 Processor -- start 2024-02-26T04:49:17.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc34cdd700 1 -- start start 2024-02-26T04:49:17.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc34cdd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc180057d0 0x7fdc30195020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:17.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc34cdd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc30195560 0x7fdc301998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:17.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc34cdd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc30199e30 0x7fdc3019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:17.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc34cdd700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc30195c50 con 0x7fdc180057d0 2024-02-26T04:49:17.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc34cdd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc301a1ba0 con 0x7fdc30195560 2024-02-26T04:49:17.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc34cdd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc301a1ce0 con 0x7fdc30199e30 2024-02-26T04:49:17.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc2ed9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc30199e30 0x7fdc3019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:17.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc2ed9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc30199e30 0x7fdc3019a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:42968/0 (socket says 172.21.15.191:42968) 2024-02-26T04:49:17.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc2ed9d700 1 -- 172.21.15.191:0/3200518514 learned_addr learned my addr 172.21.15.191:0/3200518514 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-26T04:49:17.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc2e59c700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc180057d0 0x7fdc30195020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:17.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc2dd9b700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc30195560 0x7fdc301998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:17.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc2ed9d700 1 -- 172.21.15.191:0/3200518514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc30199e30 msgr2=0x7fdc3019a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-26T04:49:17.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc2ed9d700 1 -- 172.21.15.191:0/3200518514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc30199e30 msgr2=0x7fdc3019a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-26T04:49:17.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.639+0000 7fdc2ed9d700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc30199e30 0x7fdc3019a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-26T04:49:17.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc2ed9d700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc30199e30 0x7fdc3019a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T04:49:17.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc2dd9b700 1 -- 172.21.15.191:0/3200518514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc30195560 msgr2=0x7fdc301998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-02-26T04:49:17.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc2dd9b700 1 -- 172.21.15.191:0/3200518514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc30195560 msgr2=0x7fdc301998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-26T04:49:17.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc2dd9b700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc30195560 0x7fdc301998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-26T04:49:17.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc2dd9b700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc30195560 0x7fdc301998f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T04:49:17.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc2e59c700 1 -- 172.21.15.191:0/3200518514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc30199e30 msgr2=0x7fdc3019a2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T04:49:17.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc2e59c700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc30199e30 0x7fdc3019a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:17.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc2e59c700 1 -- 172.21.15.191:0/3200518514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc30195560 msgr2=0x7fdc301998f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-26T04:49:17.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc2e59c700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc30195560 0x7fdc301998f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:17.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc2e59c700 1 -- 172.21.15.191:0/3200518514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280098d0 con 0x7fdc180057d0 2024-02-26T04:49:17.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc2e59c700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc180057d0 0x7fdc30195020 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdc28002230 tx=0x7fdc280177d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:49:17.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc1f7fe700 1 -- 172.21.15.191:0/3200518514 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28004d80 con 0x7fdc180057d0 2024-02-26T04:49:17.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc34cdd700 1 -- 172.21.15.191:0/3200518514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc301a1f60 con 0x7fdc180057d0 2024-02-26T04:49:17.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc34cdd700 1 -- 172.21.15.191:0/3200518514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc301a23d0 con 0x7fdc180057d0 2024-02-26T04:49:17.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc1f7fe700 1 -- 172.21.15.191:0/3200518514 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc28021b40 con 0x7fdc180057d0 2024-02-26T04:49:17.646 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.640+0000 7fdc1f7fe700 1 -- 172.21.15.191:0/3200518514 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28005820 con 0x7fdc180057d0 2024-02-26T04:49:17.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.641+0000 7fdc34cdd700 1 -- 172.21.15.191:0/3200518514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc300632a0 con 0x7fdc180057d0 2024-02-26T04:49:17.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.641+0000 7fdc1f7fe700 1 -- 172.21.15.191:0/3200518514 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fdc280203e0 con 0x7fdc180057d0 2024-02-26T04:49:17.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.641+0000 7fdc1f7fe700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdc10039410 0x7fdc1003b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:17.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.642+0000 7fdc1f7fe700 1 -- 172.21.15.191:0/3200518514 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdc2804dda0 con 0x7fdc180057d0 2024-02-26T04:49:17.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.642+0000 7fdc2dd9b700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdc10039410 0x7fdc1003b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:17.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.642+0000 7fdc2dd9b700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdc10039410 0x7fdc1003b8d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009850 tx=0x7fdc24008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:49:17.650 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.647+0000 7fdc1f7fe700 1 -- 172.21.15.191:0/3200518514 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdc280293b0 con 0x7fdc180057d0 2024-02-26T04:49:17.927 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:17 smithi191 ceph-mon[36352]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:17.928 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:17.928 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:17.933 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.931+0000 7fdc34cdd700 1 -- 172.21.15.191:0/3200518514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdc30107510 con 0x7fdc180057d0 2024-02-26T04:49:17.934 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.931+0000 7fdc1f7fe700 1 -- 172.21.15.191:0/3200518514 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1283 (secure 0 0 0) 0x7fdc28025520 con 0x7fdc180057d0 2024-02-26T04:49:17.934 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-26T04:49:17.934 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":3,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","modified":"2024-02-26T04:49:11.699680Z","created":"2024-02-26T04:47:09.985932Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-26T04:49:17.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.933+0000 7fdc34cdd700 1 -- 172.21.15.191:0/3200518514 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdc10039410 msgr2=0x7fdc1003b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:17.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.933+0000 7fdc34cdd700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdc10039410 0x7fdc1003b8d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009850 tx=0x7fdc24008040 comp rx=0 tx=0).stop 2024-02-26T04:49:17.936 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.934+0000 7fdc34cdd700 1 -- 172.21.15.191:0/3200518514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc180057d0 msgr2=0x7fdc30195020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:17.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.934+0000 7fdc34cdd700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc180057d0 0x7fdc30195020 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdc28002230 tx=0x7fdc280177d0 comp rx=0 tx=0).stop 2024-02-26T04:49:17.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.934+0000 7fdc34cdd700 1 -- 172.21.15.191:0/3200518514 shutdown_connections 2024-02-26T04:49:17.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.934+0000 7fdc34cdd700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc30199e30 0x7fdc3019a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:17.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.934+0000 7fdc34cdd700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc180057d0 0x7fdc30195020 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:17.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.934+0000 7fdc34cdd700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdc10039410 0x7fdc1003b8d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:17.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.934+0000 7fdc34cdd700 1 --2- 172.21.15.191:0/3200518514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc30195560 0x7fdc301998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:17.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.934+0000 7fdc34cdd700 1 -- 172.21.15.191:0/3200518514 >> 172.21.15.191:0/3200518514 conn(0x7fdc300fc180 msgr2=0x7fdc300fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:49:17.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.934+0000 7fdc34cdd700 1 -- 172.21.15.191:0/3200518514 shutdown_connections 2024-02-26T04:49:17.937 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:17.934+0000 7fdc34cdd700 1 -- 172.21.15.191:0/3200518514 wait complete. 2024-02-26T04:49:17.937 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 3 2024-02-26T04:49:18.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:17 smithi060 ceph-mon[32382]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:18.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:18.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:18.820 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-26T04:49:18.820 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph config generate-minimal-conf 2024-02-26T04:49:19.070 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:49:19.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:49:19.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: mon.c calling monitor election 2024-02-26T04:49:19.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: mon.a calling monitor election 2024-02-26T04:49:19.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:19.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:49:19.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:19.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:19.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:19.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:19.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:19.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:19.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:19.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-26T04:49:19.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: monmap e3: 3 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T04:49:19.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: fsmap 2024-02-26T04:49:19.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T04:49:19.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: mgrmap e13: x(active, since 58s) 2024-02-26T04:49:19.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-26T04:49:19.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-26T04:49:19.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-26T04:49:19.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: mon.b (rank 2) addr [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] is down (out of quorum) 2024-02-26T04:49:19.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:19.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:19.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:19.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:19.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:19.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:19.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:19.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: mon.b calling monitor election 2024-02-26T04:49:19.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: Updating smithi060:/etc/ceph/ceph.conf 2024-02-26T04:49:19.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: Updating smithi156:/etc/ceph/ceph.conf 2024-02-26T04:49:19.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-26T04:49:19.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:19.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: Updating smithi060:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:49:19.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: Updating smithi156:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:49:19.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: Updating smithi191:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:49:19.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: mon.b calling monitor election 2024-02-26T04:49:19.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: mon.c calling monitor election 2024-02-26T04:49:19.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: mon.a calling monitor election 2024-02-26T04:49:19.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-26T04:49:19.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: monmap e3: 3 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T04:49:19.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: fsmap 2024-02-26T04:49:19.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T04:49:19.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: mgrmap e13: x(active, since 60s) 2024-02-26T04:49:19.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-26T04:49:19.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: Cluster is now healthy 2024-02-26T04:49:19.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: overall HEALTH_OK 2024-02-26T04:49:19.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:19.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:19.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:19.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:19.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:19.982 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:19.982 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:19.982 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:19.982 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:19.982 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: Reconfiguring mon.a (unknown last config time)... 2024-02-26T04:49:19.982 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T04:49:19.982 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T04:49:19.982 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:19.982 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: Reconfiguring daemon mon.a on smithi060 2024-02-26T04:49:19.982 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:20.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: mon.b calling monitor election 2024-02-26T04:49:20.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: Updating smithi060:/etc/ceph/ceph.conf 2024-02-26T04:49:20.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: Updating smithi156:/etc/ceph/ceph.conf 2024-02-26T04:49:20.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-26T04:49:20.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:20.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: Updating smithi060:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:49:20.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: Updating smithi156:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:49:20.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: Updating smithi191:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:49:20.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: mon.b calling monitor election 2024-02-26T04:49:20.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: mon.c calling monitor election 2024-02-26T04:49:20.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: mon.a calling monitor election 2024-02-26T04:49:20.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-26T04:49:20.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: monmap e3: 3 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T04:49:20.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: fsmap 2024-02-26T04:49:20.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T04:49:20.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: mgrmap e13: x(active, since 60s) 2024-02-26T04:49:20.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-26T04:49:20.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: Cluster is now healthy 2024-02-26T04:49:20.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: overall HEALTH_OK 2024-02-26T04:49:20.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: Reconfiguring mon.a (unknown last config time)... 2024-02-26T04:49:20.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T04:49:20.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T04:49:20.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:20.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: Reconfiguring daemon mon.a on smithi060 2024-02-26T04:49:20.012 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:20.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: mon.b calling monitor election 2024-02-26T04:49:20.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: Updating smithi060:/etc/ceph/ceph.conf 2024-02-26T04:49:20.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: Updating smithi156:/etc/ceph/ceph.conf 2024-02-26T04:49:20.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-26T04:49:20.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:20.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: Updating smithi060:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:49:20.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: Updating smithi156:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:49:20.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: Updating smithi191:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/config/ceph.conf 2024-02-26T04:49:20.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: mon.b calling monitor election 2024-02-26T04:49:20.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: mon.c calling monitor election 2024-02-26T04:49:20.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: mon.a calling monitor election 2024-02-26T04:49:20.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-26T04:49:20.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: monmap e3: 3 mons at {a=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0],b=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-26T04:49:20.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: fsmap 2024-02-26T04:49:20.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: osdmap e4: 0 total, 0 up, 0 in 2024-02-26T04:49:20.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: mgrmap e13: x(active, since 60s) 2024-02-26T04:49:20.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-26T04:49:20.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: Cluster is now healthy 2024-02-26T04:49:20.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: overall HEALTH_OK 2024-02-26T04:49:20.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.180 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.180 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:20.180 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: Reconfiguring mon.a (unknown last config time)... 2024-02-26T04:49:20.180 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T04:49:20.180 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T04:49:20.180 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:20.180 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: Reconfiguring daemon mon.a on smithi060 2024-02-26T04:49:20.180 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:49:21.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.210+0000 7ff77f822700 1 -- 172.21.15.60:0/2101071484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7781027f0 msgr2=0x7ff778102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:21.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.210+0000 7ff77f822700 1 --2- 172.21.15.60:0/2101071484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7781027f0 0x7ff778102bd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff774009910 tx=0x7ff774009c20 comp rx=0 tx=0).stop 2024-02-26T04:49:21.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.210+0000 7ff77f822700 1 -- 172.21.15.60:0/2101071484 shutdown_connections 2024-02-26T04:49:21.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.210+0000 7ff77f822700 1 --2- 172.21.15.60:0/2101071484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7781027f0 0x7ff778102bd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:21.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.210+0000 7ff77f822700 1 -- 172.21.15.60:0/2101071484 >> 172.21.15.60:0/2101071484 conn(0x7ff7780fc020 msgr2=0x7ff7780fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:49:21.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.210+0000 7ff77f822700 1 -- 172.21.15.60:0/2101071484 shutdown_connections 2024-02-26T04:49:21.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.210+0000 7ff77f822700 1 -- 172.21.15.60:0/2101071484 wait complete. 2024-02-26T04:49:21.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.211+0000 7ff77f822700 1 Processor -- start 2024-02-26T04:49:21.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.211+0000 7ff77f822700 1 -- start start 2024-02-26T04:49:21.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.211+0000 7ff77f822700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7781027f0 0x7ff778197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:21.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.211+0000 7ff77f822700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff778197780 0x7ff77819bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:21.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.211+0000 7ff77f822700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff77819c050 0x7ff77819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:21.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.211+0000 7ff77f822700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff778197e70 con 0x7ff77819c050 2024-02-26T04:49:21.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.211+0000 7ff77f822700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7781a3dc0 con 0x7ff778197780 2024-02-26T04:49:21.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.211+0000 7ff77f822700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7781a3f00 con 0x7ff7781027f0 2024-02-26T04:49:21.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.211+0000 7ff77ddbf700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff77819c050 0x7ff77819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:21.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.211+0000 7ff77ddbf700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff77819c050 0x7ff77819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49114/0 (socket says 172.21.15.60:49114) 2024-02-26T04:49:21.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.211+0000 7ff77ddbf700 1 -- 172.21.15.60:0/191093118 learned_addr learned my addr 172.21.15.60:0/191093118 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:49:21.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.211+0000 7ff77d5be700 1 --2- 172.21.15.60:0/191093118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7781027f0 0x7ff778197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:21.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.211+0000 7ff77ddbf700 1 -- 172.21.15.60:0/191093118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7781027f0 msgr2=0x7ff778197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:21.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.211+0000 7ff77ddbf700 1 --2- 172.21.15.60:0/191093118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7781027f0 0x7ff778197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:21.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.212+0000 7ff77ddbf700 1 -- 172.21.15.60:0/191093118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff778197780 msgr2=0x7ff77819bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:21.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.212+0000 7ff77cdbd700 1 --2- 172.21.15.60:0/191093118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff778197780 0x7ff77819bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:21.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.212+0000 7ff77ddbf700 1 --2- 172.21.15.60:0/191093118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff778197780 0x7ff77819bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:21.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.212+0000 7ff77ddbf700 1 -- 172.21.15.60:0/191093118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7740095f0 con 0x7ff77819c050 2024-02-26T04:49:21.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.212+0000 7ff77ddbf700 1 --2- 172.21.15.60:0/191093118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff77819c050 0x7ff77819c500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff77000ba50 tx=0x7ff77000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:49:21.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.212+0000 7ff76e7fc700 1 -- 172.21.15.60:0/191093118 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77000cab0 con 0x7ff77819c050 2024-02-26T04:49:21.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.212+0000 7ff76e7fc700 1 -- 172.21.15.60:0/191093118 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff77000cc10 con 0x7ff77819c050 2024-02-26T04:49:21.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.212+0000 7ff76e7fc700 1 -- 172.21.15.60:0/191093118 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7700075d0 con 0x7ff77819c050 2024-02-26T04:49:21.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.212+0000 7ff77f822700 1 -- 172.21.15.60:0/191093118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7781a41e0 con 0x7ff77819c050 2024-02-26T04:49:21.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.212+0000 7ff77f822700 1 -- 172.21.15.60:0/191093118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7781a46e0 con 0x7ff77819c050 2024-02-26T04:49:21.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.213+0000 7ff76e7fc700 1 -- 172.21.15.60:0/191093118 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7ff770014440 con 0x7ff77819c050 2024-02-26T04:49:21.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.213+0000 7ff77f822700 1 -- 172.21.15.60:0/191093118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7780632a0 con 0x7ff77819c050 2024-02-26T04:49:21.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.213+0000 7ff76e7fc700 1 --2- 172.21.15.60:0/191093118 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff764039410 0x7ff76403b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:21.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.213+0000 7ff76e7fc700 1 -- 172.21.15.60:0/191093118 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff77004b430 con 0x7ff77819c050 2024-02-26T04:49:21.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.213+0000 7ff77d5be700 1 --2- 172.21.15.60:0/191093118 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff764039410 0x7ff76403b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:21.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.214+0000 7ff77d5be700 1 --2- 172.21.15.60:0/191093118 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff764039410 0x7ff76403b8d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff77400b5c0 tx=0x7ff774005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:49:21.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.218+0000 7ff76e7fc700 1 -- 172.21.15.60:0/191093118 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7700146f0 con 0x7ff77819c050 2024-02-26T04:49:21.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.421+0000 7ff77f822700 1 -- 172.21.15.60:0/191093118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff77810ba80 con 0x7ff77819c050 2024-02-26T04:49:21.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.422+0000 7ff76e7fc700 1 -- 172.21.15.60:0/191093118 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7ff770012340 con 0x7ff77819c050 2024-02-26T04:49:21.424 INFO:teuthology.orchestra.run.smithi060.stdout:# minimal ceph.conf for f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:49:21.424 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-02-26T04:49:21.425 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:49:21.425 INFO:teuthology.orchestra.run.smithi060.stdout: mon_host = [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] 2024-02-26T04:49:21.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.424+0000 7ff77f822700 1 -- 172.21.15.60:0/191093118 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff764039410 msgr2=0x7ff76403b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:21.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.424+0000 7ff77f822700 1 --2- 172.21.15.60:0/191093118 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff764039410 0x7ff76403b8d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff77400b5c0 tx=0x7ff774005a90 comp rx=0 tx=0).stop 2024-02-26T04:49:21.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.424+0000 7ff77f822700 1 -- 172.21.15.60:0/191093118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff77819c050 msgr2=0x7ff77819c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:21.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.424+0000 7ff77f822700 1 --2- 172.21.15.60:0/191093118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff77819c050 0x7ff77819c500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff77000ba50 tx=0x7ff77000be10 comp rx=0 tx=0).stop 2024-02-26T04:49:21.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.424+0000 7ff77f822700 1 -- 172.21.15.60:0/191093118 shutdown_connections 2024-02-26T04:49:21.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.424+0000 7ff77f822700 1 --2- 172.21.15.60:0/191093118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff77819c050 0x7ff77819c500 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:21.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.424+0000 7ff77f822700 1 --2- 172.21.15.60:0/191093118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7781027f0 0x7ff778197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.424+0000 7ff77f822700 1 --2- 172.21.15.60:0/191093118 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff764039410 0x7ff76403b8d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.424+0000 7ff77f822700 1 --2- 172.21.15.60:0/191093118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff778197780 0x7ff77819bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.424+0000 7ff77f822700 1 -- 172.21.15.60:0/191093118 >> 172.21.15.60:0/191093118 conn(0x7ff7780fc020 msgr2=0x7ff7781055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:49:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.424+0000 7ff77f822700 1 -- 172.21.15.60:0/191093118 shutdown_connections 2024-02-26T04:49:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:21.424+0000 7ff77f822700 1 -- 172.21.15.60:0/191093118 wait complete. 2024-02-26T04:49:21.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:21 smithi060 ceph-mon[32382]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:21.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:21 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/191093118' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:21.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:21.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:21.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T04:49:21.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T04:49:21.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:21 smithi191 ceph-mon[36352]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:21 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/191093118' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T04:49:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T04:49:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:22.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:21 smithi156 ceph-mon[35805]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:22.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:21 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/191093118' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:22.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:22.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T04:49:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T04:49:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:22.222 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-26T04:49:22.222 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:49:22.222 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.conf 2024-02-26T04:49:22.254 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:49:22.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T04:49:22.322 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-26T04:49:22.322 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.conf 2024-02-26T04:49:22.351 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-26T04:49:22.352 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T04:49:22.420 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-26T04:49:22.420 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.conf 2024-02-26T04:49:22.454 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-26T04:49:22.454 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-26T04:49:22.525 INFO:tasks.cephadm:Adding mgr.x on smithi060 2024-02-26T04:49:22.526 INFO:tasks.cephadm:Adding mgr.y on smithi156 2024-02-26T04:49:22.526 INFO:tasks.cephadm:Adding mgr.z on smithi191 2024-02-26T04:49:22.526 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch apply mgr '3;smithi060=x;smithi156=y;smithi191=z' 2024-02-26T04:49:22.797 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T04:49:22.845 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:22 smithi156 ceph-mon[35805]: Reconfiguring mon.b (monmap changed)... 2024-02-26T04:49:22.846 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:22 smithi156 ceph-mon[35805]: Reconfiguring daemon mon.b on smithi156 2024-02-26T04:49:22.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:22 smithi060 ceph-mon[32382]: Reconfiguring mon.b (monmap changed)... 2024-02-26T04:49:22.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:22 smithi060 ceph-mon[32382]: Reconfiguring daemon mon.b on smithi156 2024-02-26T04:49:23.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:22 smithi191 ceph-mon[36352]: Reconfiguring mon.b (monmap changed)... 2024-02-26T04:49:23.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:22 smithi191 ceph-mon[36352]: Reconfiguring daemon mon.b on smithi156 2024-02-26T04:49:24.427 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:24 smithi191 ceph-mon[36352]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:24.427 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:24.427 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:24.427 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:24 smithi191 ceph-mon[36352]: Reconfiguring mon.c (monmap changed)... 2024-02-26T04:49:24.427 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T04:49:24.427 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T04:49:24.427 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:24.427 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:24 smithi191 ceph-mon[36352]: Reconfiguring daemon mon.c on smithi191 2024-02-26T04:49:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:24 smithi156 ceph-mon[35805]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:24 smithi156 ceph-mon[35805]: Reconfiguring mon.c (monmap changed)... 2024-02-26T04:49:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T04:49:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T04:49:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:24 smithi156 ceph-mon[35805]: Reconfiguring daemon mon.c on smithi191 2024-02-26T04:49:24.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:24 smithi060 ceph-mon[32382]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:24.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:24.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:24.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:24 smithi060 ceph-mon[32382]: Reconfiguring mon.c (monmap changed)... 2024-02-26T04:49:24.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-26T04:49:24.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-26T04:49:24.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:24.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:24 smithi060 ceph-mon[32382]: Reconfiguring daemon mon.c on smithi191 2024-02-26T04:49:24.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.937+0000 7f53a47ee700 1 -- 172.21.15.191:0/2855206579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539c102ec0 msgr2=0x7f539c1032e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:24.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.937+0000 7f53a47ee700 1 --2- 172.21.15.191:0/2855206579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539c102ec0 0x7f539c1032e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5398009910 tx=0x7f5398009c20 comp rx=0 tx=0).stop 2024-02-26T04:49:24.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.937+0000 7f53a47ee700 1 -- 172.21.15.191:0/2855206579 shutdown_connections 2024-02-26T04:49:24.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.937+0000 7f53a47ee700 1 --2- 172.21.15.191:0/2855206579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539c102ec0 0x7f539c1032e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:24.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.937+0000 7f53a47ee700 1 -- 172.21.15.191:0/2855206579 >> 172.21.15.191:0/2855206579 conn(0x7f539c0fcd60 msgr2=0x7f539c0ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:49:24.941 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.937+0000 7f53a47ee700 1 -- 172.21.15.191:0/2855206579 shutdown_connections 2024-02-26T04:49:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.937+0000 7f53a47ee700 1 -- 172.21.15.191:0/2855206579 wait complete. 2024-02-26T04:49:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.937+0000 7f53a47ee700 1 Processor -- start 2024-02-26T04:49:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.937+0000 7f53a47ee700 1 -- start start 2024-02-26T04:49:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a47ee700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f539c102ec0 0x7f539c196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a47ee700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539c197410 0x7f539c19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a47ee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f539c197890 0x7f539c19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a47ee700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c19f1b0 con 0x7f539c197410 2024-02-26T04:49:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a47ee700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c19f320 con 0x7f539c197890 2024-02-26T04:49:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a47ee700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c19f4e0 con 0x7f539c102ec0 2024-02-26T04:49:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a258a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f539c102ec0 0x7f539c196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a258a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f539c102ec0 0x7f539c196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:42970/0 (socket says 172.21.15.191:42970) 2024-02-26T04:49:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a258a700 1 -- 172.21.15.191:0/737385616 learned_addr learned my addr 172.21.15.191:0/737385616 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-26T04:49:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a2d8b700 1 --2- 172.21.15.191:0/737385616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f539c197890 0x7f539c19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a1d89700 1 --2- 172.21.15.191:0/737385616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539c197410 0x7f539c19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:24.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a258a700 1 -- 172.21.15.191:0/737385616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f539c197890 msgr2=0x7f539c19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:24.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a258a700 1 --2- 172.21.15.191:0/737385616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f539c197890 0x7f539c19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:24.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a258a700 1 -- 172.21.15.191:0/737385616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539c197410 msgr2=0x7f539c19c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:24.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a258a700 1 --2- 172.21.15.191:0/737385616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539c197410 0x7f539c19c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:24.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a258a700 1 -- 172.21.15.191:0/737385616 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53980095f0 con 0x7f539c102ec0 2024-02-26T04:49:24.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a258a700 1 --2- 172.21.15.191:0/737385616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f539c102ec0 0x7f539c196ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f539800bcf0 tx=0x7f539800bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:49:24.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53937fe700 1 -- 172.21.15.191:0/737385616 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539801d070 con 0x7f539c102ec0 2024-02-26T04:49:24.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53a47ee700 1 -- 172.21.15.191:0/737385616 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539c19f760 con 0x7f539c102ec0 2024-02-26T04:49:24.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53937fe700 1 -- 172.21.15.191:0/737385616 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5398004f40 con 0x7f539c102ec0 2024-02-26T04:49:24.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.938+0000 7f53937fe700 1 -- 172.21.15.191:0/737385616 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539800f460 con 0x7f539c102ec0 2024-02-26T04:49:24.943 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.939+0000 7f53a47ee700 1 -- 172.21.15.191:0/737385616 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539c19fc80 con 0x7f539c102ec0 2024-02-26T04:49:24.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.939+0000 7f53a47ee700 1 -- 172.21.15.191:0/737385616 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539c1906f0 con 0x7f539c102ec0 2024-02-26T04:49:24.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.940+0000 7f53937fe700 1 -- 172.21.15.191:0/737385616 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f5398004a60 con 0x7f539c102ec0 2024-02-26T04:49:24.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.940+0000 7f53937fe700 1 --2- 172.21.15.191:0/737385616 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5388039410 0x7f538803b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:24.944 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.940+0000 7f53937fe700 1 -- 172.21.15.191:0/737385616 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f539804d1e0 con 0x7f539c102ec0 2024-02-26T04:49:24.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.940+0000 7f53a1d89700 1 --2- 172.21.15.191:0/737385616 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5388039410 0x7f538803b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:24.945 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.940+0000 7f53a1d89700 1 --2- 172.21.15.191:0/737385616 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5388039410 0x7f538803b8d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f538c009850 tx=0x7f538c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:49:24.947 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:24.945+0000 7f53937fe700 1 -- 172.21.15.191:0/737385616 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f539800eeb0 con 0x7f539c102ec0 2024-02-26T04:49:25.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:25.154+0000 7f53a47ee700 1 -- 172.21.15.191:0/737385616 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi060=x;smithi156=y;smithi191=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f539c0620b0 con 0x7f5388039410 2024-02-26T04:49:25.163 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:25.161+0000 7f53937fe700 1 -- 172.21.15.191:0/737385616 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f539c0620b0 con 0x7f5388039410 2024-02-26T04:49:25.164 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mgr update... 2024-02-26T04:49:25.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:25.163+0000 7f53a47ee700 1 -- 172.21.15.191:0/737385616 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5388039410 msgr2=0x7f538803b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:25.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:25.163+0000 7f53a47ee700 1 --2- 172.21.15.191:0/737385616 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5388039410 0x7f538803b8d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f538c009850 tx=0x7f538c008040 comp rx=0 tx=0).stop 2024-02-26T04:49:25.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:25.163+0000 7f53a47ee700 1 -- 172.21.15.191:0/737385616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f539c102ec0 msgr2=0x7f539c196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:25.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:25.163+0000 7f53a47ee700 1 --2- 172.21.15.191:0/737385616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f539c102ec0 0x7f539c196ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f539800bcf0 tx=0x7f539800bdd0 comp rx=0 tx=0).stop 2024-02-26T04:49:25.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:25.163+0000 7f53a47ee700 1 -- 172.21.15.191:0/737385616 shutdown_connections 2024-02-26T04:49:25.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:25.163+0000 7f53a47ee700 1 --2- 172.21.15.191:0/737385616 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5388039410 0x7f538803b8d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:25.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:25.163+0000 7f53a47ee700 1 --2- 172.21.15.191:0/737385616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f539c197890 0x7f539c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:25.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:25.163+0000 7f53a47ee700 1 --2- 172.21.15.191:0/737385616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f539c102ec0 0x7f539c196ed0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:25.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:25.163+0000 7f53a47ee700 1 --2- 172.21.15.191:0/737385616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539c197410 0x7f539c19c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:25.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:25.163+0000 7f53a47ee700 1 -- 172.21.15.191:0/737385616 >> 172.21.15.191:0/737385616 conn(0x7f539c0fcd60 msgr2=0x7f539c0fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:49:25.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:25.163+0000 7f53a47ee700 1 -- 172.21.15.191:0/737385616 shutdown_connections 2024-02-26T04:49:25.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:49:25.163+0000 7f53a47ee700 1 -- 172.21.15.191:0/737385616 wait complete. 2024-02-26T04:49:26.002 DEBUG:teuthology.orchestra.run.smithi156:mgr.y> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mgr.y.service 2024-02-26T04:49:26.005 DEBUG:teuthology.orchestra.run.smithi191:mgr.z> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mgr.z.service 2024-02-26T04:49:26.008 INFO:tasks.cephadm:Deploying OSDs... 2024-02-26T04:49:26.008 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:49:26.008 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2024-02-26T04:49:26.023 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-26T04:49:26.024 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2024-02-26T04:49:26.080 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-26T04:49:26.080 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T04:49:26.080 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-26T04:49:26.080 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T04:49:26.080 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T04:49:26.080 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-02-26 04:48:28.143219289 +0000 2024-02-26T04:49:26.080 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-02-26 04:42:35.176556937 +0000 2024-02-26T04:49:26.080 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-02-26 04:42:35.176556937 +0000 2024-02-26T04:49:26.080 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-02-26 04:42:35.176556937 +0000 2024-02-26T04:49:26.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-26T04:49:26.145 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-02-26T04:49:26.146 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-02-26T04:49:26.146 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000105426 s, 4.9 MB/s 2024-02-26T04:49:26.147 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-26T04:49:26.203 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2024-02-26T04:49:26.257 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-26T04:49:26.258 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T04:49:26.258 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-02-26T04:49:26.258 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T04:49:26.258 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T04:49:26.258 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-02-26 04:48:28.181218896 +0000 2024-02-26T04:49:26.258 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-02-26 04:42:35.509553400 +0000 2024-02-26T04:49:26.258 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-02-26 04:42:35.509553400 +0000 2024-02-26T04:49:26.258 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-02-26 04:42:35.509553400 +0000 2024-02-26T04:49:26.258 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-26T04:49:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:26 smithi191 ceph-mon[36352]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:26 smithi191 ceph-mon[36352]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi060=x;smithi156=y;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:49:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:26 smithi191 ceph-mon[36352]: Saving service mgr spec with placement smithi060=x;smithi156=y;smithi191=z;count:3 2024-02-26T04:49:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:26.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:26.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T04:49:26.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T04:49:26.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T04:49:26.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-02-26T04:49:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-02-26T04:49:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 9.3451e-05 s, 5.5 MB/s 2024-02-26T04:49:26.324 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-26T04:49:26.379 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2024-02-26T04:49:26.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:26 smithi156 ceph-mon[35805]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:26 smithi156 ceph-mon[35805]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi060=x;smithi156=y;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:49:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:26 smithi156 ceph-mon[35805]: Saving service mgr spec with placement smithi060=x;smithi156=y;smithi191=z;count:3 2024-02-26T04:49:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T04:49:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T04:49:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T04:49:26.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:26.435 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-26T04:49:26.435 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T04:49:26.435 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-26T04:49:26.436 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T04:49:26.436 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T04:49:26.436 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-02-26 04:48:28.224218451 +0000 2024-02-26T04:49:26.436 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-02-26 04:42:35.841549874 +0000 2024-02-26T04:49:26.436 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-02-26 04:42:35.841549874 +0000 2024-02-26T04:49:26.437 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-02-26 04:42:35.841549874 +0000 2024-02-26T04:49:26.437 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-26T04:49:26.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:26 smithi060 ceph-mon[32382]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:26 smithi060 ceph-mon[32382]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi060=x;smithi156=y;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:49:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:26 smithi060 ceph-mon[32382]: Saving service mgr spec with placement smithi060=x;smithi156=y;smithi191=z;count:3 2024-02-26T04:49:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:26.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:26.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:26.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:26.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:26.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T04:49:26.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T04:49:26.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T04:49:26.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-02-26T04:49:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-02-26T04:49:26.501 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.00010558 s, 4.8 MB/s 2024-02-26T04:49:26.502 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-26T04:49:26.558 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2024-02-26T04:49:26.615 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-26T04:49:26.615 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T04:49:26.615 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-26T04:49:26.615 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T04:49:26.615 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T04:49:26.615 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-02-26 04:48:28.267218006 +0000 2024-02-26T04:49:26.615 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-02-26 04:42:36.179546284 +0000 2024-02-26T04:49:26.616 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-02-26 04:42:36.179546284 +0000 2024-02-26T04:49:26.616 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-02-26 04:42:36.179546284 +0000 2024-02-26T04:49:26.616 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-26T04:49:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-02-26T04:49:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-02-26T04:49:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000103287 s, 5.0 MB/s 2024-02-26T04:49:26.681 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-26T04:49:26.737 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-02-26T04:49:26.738 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2024-02-26T04:49:26.756 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-26T04:49:26.757 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2024-02-26T04:49:26.816 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-26T04:49:26.816 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T04:49:26.816 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-02-26T04:49:26.816 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T04:49:26.817 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T04:49:26.817 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-02-26 04:49:11.190492753 +0000 2024-02-26T04:49:26.817 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-02-26 04:42:51.151166744 +0000 2024-02-26T04:49:26.817 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-02-26 04:42:51.151166744 +0000 2024-02-26T04:49:26.817 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-02-26 04:42:51.151166744 +0000 2024-02-26T04:49:26.818 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-26T04:49:26.886 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-02-26T04:49:26.886 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-02-26T04:49:26.886 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000136225 s, 3.8 MB/s 2024-02-26T04:49:26.887 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-26T04:49:26.944 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2024-02-26T04:49:27.001 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-26T04:49:27.001 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T04:49:27.002 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-02-26T04:49:27.002 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T04:49:27.002 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T04:49:27.002 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-02-26 04:49:11.190492753 +0000 2024-02-26T04:49:27.002 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-02-26 04:42:51.482161652 +0000 2024-02-26T04:49:27.002 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-02-26 04:42:51.482161652 +0000 2024-02-26T04:49:27.002 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-02-26 04:42:51.482161652 +0000 2024-02-26T04:49:27.002 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-26T04:49:27.070 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-02-26T04:49:27.070 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-02-26T04:49:27.070 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000127378 s, 4.0 MB/s 2024-02-26T04:49:27.071 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-26T04:49:27.128 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2024-02-26T04:49:27.184 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-26T04:49:27.184 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T04:49:27.184 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-02-26T04:49:27.184 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T04:49:27.184 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T04:49:27.184 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-02-26 04:49:11.190492753 +0000 2024-02-26T04:49:27.184 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-02-26 04:42:51.831156283 +0000 2024-02-26T04:49:27.184 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-02-26 04:42:51.831156283 +0000 2024-02-26T04:49:27.184 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-02-26 04:42:51.831156283 +0000 2024-02-26T04:49:27.185 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-26T04:49:27.250 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-02-26T04:49:27.250 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-02-26T04:49:27.250 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 9.7406e-05 s, 5.3 MB/s 2024-02-26T04:49:27.251 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-26T04:49:27.258 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:27 smithi191 ceph-mon[36352]: Deploying daemon mgr.z on smithi191 2024-02-26T04:49:27.306 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2024-02-26T04:49:27.362 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-26T04:49:27.362 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T04:49:27.362 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-02-26T04:49:27.362 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T04:49:27.362 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T04:49:27.362 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-02-26 04:49:11.191492738 +0000 2024-02-26T04:49:27.363 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-02-26 04:42:52.156151283 +0000 2024-02-26T04:49:27.363 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-02-26 04:42:52.156151283 +0000 2024-02-26T04:49:27.363 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-02-26 04:42:52.156151283 +0000 2024-02-26T04:49:27.363 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-26T04:49:27.426 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:27 smithi156 ceph-mon[35805]: Deploying daemon mgr.z on smithi191 2024-02-26T04:49:27.427 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-02-26T04:49:27.428 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-02-26T04:49:27.428 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000116456 s, 4.4 MB/s 2024-02-26T04:49:27.429 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-26T04:49:27.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:27 smithi060 ceph-mon[32382]: Deploying daemon mgr.z on smithi191 2024-02-26T04:49:27.484 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-26T04:49:27.484 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2024-02-26T04:49:27.501 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-26T04:49:27.501 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2024-02-26T04:49:27.556 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-26T04:49:27.557 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T04:49:27.557 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-02-26T04:49:27.557 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T04:49:27.557 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T04:49:27.557 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-26 04:49:01.664600079 +0000 2024-02-26T04:49:27.557 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-26 04:43:04.309381085 +0000 2024-02-26T04:49:27.557 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-26 04:43:04.309381085 +0000 2024-02-26T04:49:27.557 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-26 04:43:04.309381085 +0000 2024-02-26T04:49:27.557 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-26T04:49:27.755 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-26T04:49:27.755 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-26T04:49:27.755 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000270855 s, 1.9 MB/s 2024-02-26T04:49:27.757 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-26T04:49:27.772 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2024-02-26T04:49:27.832 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-26T04:49:27.833 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T04:49:27.833 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 679 Links: 1 2024-02-26T04:49:27.833 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T04:49:27.833 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T04:49:27.833 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-26 04:49:27.693147850 +0000 2024-02-26T04:49:27.833 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-26 04:43:04.648375069 +0000 2024-02-26T04:49:27.833 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-26 04:43:04.648375069 +0000 2024-02-26T04:49:27.833 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-26 04:43:04.648375069 +0000 2024-02-26T04:49:27.833 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-26T04:49:27.944 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-26T04:49:27.945 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-26T04:49:27.945 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000257764 s, 2.0 MB/s 2024-02-26T04:49:27.946 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-26T04:49:27.961 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2024-02-26T04:49:28.017 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-26T04:49:28.017 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T04:49:28.017 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-02-26T04:49:28.017 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T04:49:28.017 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T04:49:28.017 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-26 04:49:27.890144427 +0000 2024-02-26T04:49:28.017 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-26 04:43:05.024368398 +0000 2024-02-26T04:49:28.017 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-26 04:43:05.024368398 +0000 2024-02-26T04:49:28.018 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-26 04:43:05.024368398 +0000 2024-02-26T04:49:28.018 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-26T04:49:28.088 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-26T04:49:28.088 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-26T04:49:28.089 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000220529 s, 2.3 MB/s 2024-02-26T04:49:28.089 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-26T04:49:28.146 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2024-02-26T04:49:28.201 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-26T04:49:28.201 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T04:49:28.201 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-02-26T04:49:28.201 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T04:49:28.201 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T04:49:28.202 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-26 04:49:27.890144427 +0000 2024-02-26T04:49:28.202 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-26 04:43:05.365362347 +0000 2024-02-26T04:49:28.202 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-26 04:43:05.365362347 +0000 2024-02-26T04:49:28.202 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-26 04:43:05.365362347 +0000 2024-02-26T04:49:28.202 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-26T04:49:28.268 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-26T04:49:28.268 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-26T04:49:28.268 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000193044 s, 2.7 MB/s 2024-02-26T04:49:28.269 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-26T04:49:28.324 INFO:tasks.cephadm:Deploying osd.0 on smithi060 with /dev/vg_nvme/lv_4... 2024-02-26T04:49:28.325 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-26T04:49:28.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:28 smithi156 ceph-mon[35805]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:28.429 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:28 smithi191 ceph-mon[36352]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:28.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:28 smithi060 ceph-mon[32382]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:28.574 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:49:28.759 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:28 smithi191 systemd[1]: Started Ceph mgr.z for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:49:29.259 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:28 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:28.991+0000 7f7770ddc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T04:49:29.259 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:29 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:29.087+0000 7f7770ddc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T04:49:29.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:29 smithi060 ceph-mon[32382]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:29.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:29 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:29.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:29 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:29.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:29 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:29.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:29 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T04:49:29.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:29 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T04:49:29.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:29 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T04:49:29.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:29 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:29.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:29 smithi060 ceph-mon[32382]: Deploying daemon mgr.y on smithi156 2024-02-26T04:49:29.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:29 smithi156 ceph-mon[35805]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:29.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:29 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:29.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:29 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:29.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:29 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:29.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:29 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T04:49:29.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:29 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T04:49:29.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:29 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T04:49:29.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:29 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:29.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:29 smithi156 ceph-mon[35805]: Deploying daemon mgr.y on smithi156 2024-02-26T04:49:29.970 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:29 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:29.733+0000 7f7770ddc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T04:49:29.970 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:29 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:29.826+0000 7f7770ddc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T04:49:29.970 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:29 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:29.967+0000 7f7770ddc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T04:49:29.970 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:29 smithi191 ceph-mon[36352]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:29.971 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:29 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:29.971 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:29 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:29.971 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:29 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:29.971 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:29 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T04:49:29.971 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:29 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-26T04:49:29.971 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:29 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T04:49:29.971 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:29 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:29.971 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:29 smithi191 ceph-mon[36352]: Deploying daemon mgr.y on smithi156 2024-02-26T04:49:30.259 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:30.062+0000 7f7770ddc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T04:49:30.259 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:30.143+0000 7f7770ddc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T04:49:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-26T04:49:30.436 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-26T04:49:30.436 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-02-26T04:49:30.436 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-02-26T04:49:30.436 INFO:teuthology.orchestra.run.smithi060.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00893957 s, 1.2 GB/s 2024-02-26T04:49:30.436 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-02-26T04:49:30.707 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:30.418+0000 7f7770ddc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T04:49:30.707 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:30.525+0000 7f7770ddc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T04:49:30.707 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:30.608+0000 7f7770ddc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T04:49:30.707 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:30.701+0000 7f7770ddc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T04:49:30.919 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:30 smithi156 systemd[1]: Starting Ceph mgr.y for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T04:49:31.009 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:30.878+0000 7f7770ddc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T04:49:31.045 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi060:vg_nvme/lv_4 2024-02-26T04:49:31.177 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:30 smithi156 podman[37172]: 2024-02-26 04:49:30.820475327 +0000 UTC m=+0.017411809 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T04:49:31.178 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:30 smithi156 podman[37172]: 2024-02-26 04:49:30.92683579 +0000 UTC m=+0.123772256 container create 5d25ce3df1b78bc61026a60744a8f11b97c53856bf115f0584e41eed0b821259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD) 2024-02-26T04:49:31.178 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:31 smithi156 podman[37172]: 2024-02-26 04:49:31.104026873 +0000 UTC m=+0.300963366 container init 5d25ce3df1b78bc61026a60744a8f11b97c53856bf115f0584e41eed0b821259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T04:49:31.178 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:31 smithi156 podman[37172]: 2024-02-26 04:49:31.106568611 +0000 UTC m=+0.303505079 container start 5d25ce3df1b78bc61026a60744a8f11b97c53856bf115f0584e41eed0b821259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-26T04:49:31.178 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:31 smithi156 ceph-mgr[37191]: -- 172.21.15.156:0/3193990255 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55691aea81a0 con 0x55691ae7ec00 2024-02-26T04:49:31.178 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:31 smithi156 bash[37172]: 5d25ce3df1b78bc61026a60744a8f11b97c53856bf115f0584e41eed0b821259 2024-02-26T04:49:31.178 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:31 smithi156 systemd[1]: Started Ceph mgr.y for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:49:31.288 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:49:31.726 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:31 smithi060 ceph-mon[32382]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:31.726 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:31.726 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:31.727 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:31.727 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:31.727 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:31.872 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:31 smithi191 ceph-mon[36352]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:31.872 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:31 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:31.872 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:31 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:31.872 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:31 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:31.872 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:31 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:31.872 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:31 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:31.927 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:31 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:31.663+0000 7fd5ff67c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T04:49:31.927 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:31 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:31.762+0000 7fd5ff67c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T04:49:31.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:31 smithi156 ceph-mon[35805]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:31.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:31 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:31.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:31 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:31.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:31 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:31.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:31 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:31.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:31 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:32.252 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:31 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:31.869+0000 7f7770ddc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T04:49:32.253 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:32 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:32.013+0000 7f7770ddc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T04:49:32.509 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:32 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:32.365+0000 7f7770ddc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T04:49:32.509 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:32 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:32.456+0000 7f7770ddc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T04:49:32.543 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:32 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:32.447+0000 7fd5ff67c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T04:49:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.721+0000 7f14ffcd7700 1 -- 172.21.15.60:0/4181771451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8103840 msgr2=0x7f14f810a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.721+0000 7f14ffcd7700 1 --2- 172.21.15.60:0/4181771451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8103840 0x7f14f810a0f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f14f00099c0 tx=0x7f14f0009cd0 comp rx=0 tx=0).stop 2024-02-26T04:49:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.721+0000 7f14ffcd7700 1 -- 172.21.15.60:0/4181771451 shutdown_connections 2024-02-26T04:49:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.721+0000 7f14ffcd7700 1 --2- 172.21.15.60:0/4181771451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8103840 0x7f14f810a0f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.721+0000 7f14ffcd7700 1 --2- 172.21.15.60:0/4181771451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14f8101ca0 0x7f14f81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.721+0000 7f14ffcd7700 1 --2- 172.21.15.60:0/4181771451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14f8102ea0 0x7f14f8103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.721+0000 7f14ffcd7700 1 -- 172.21.15.60:0/4181771451 >> 172.21.15.60:0/4181771451 conn(0x7f14f80fd280 msgr2=0x7f14f80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:49:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.721+0000 7f14ffcd7700 1 -- 172.21.15.60:0/4181771451 shutdown_connections 2024-02-26T04:49:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.721+0000 7f14ffcd7700 1 -- 172.21.15.60:0/4181771451 wait complete. 2024-02-26T04:49:32.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.722+0000 7f14ffcd7700 1 Processor -- start 2024-02-26T04:49:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.722+0000 7f14ffcd7700 1 -- start start 2024-02-26T04:49:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.722+0000 7f14ffcd7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14f8101ca0 0x7f14f8075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.722+0000 7f14ffcd7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14f8102ea0 0x7f14f8074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.722+0000 7f14ffcd7700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8103840 0x7f14f80745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.722+0000 7f14ffcd7700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f8074d20 con 0x7f14f8103840 2024-02-26T04:49:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.722+0000 7f14ffcd7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f8074e90 con 0x7f14f8101ca0 2024-02-26T04:49:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.722+0000 7f14ffcd7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f81a1c70 con 0x7f14f8102ea0 2024-02-26T04:49:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.722+0000 7f14fe274700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8103840 0x7f14f80745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.722+0000 7f14fe274700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8103840 0x7f14f80745a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52718/0 (socket says 172.21.15.60:52718) 2024-02-26T04:49:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.722+0000 7f14fda73700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14f8101ca0 0x7f14f8075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.722+0000 7f14fe274700 1 -- 172.21.15.60:0/3294248563 learned_addr learned my addr 172.21.15.60:0/3294248563 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:49:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.722+0000 7f14fd272700 1 --2- 172.21.15.60:0/3294248563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14f8102ea0 0x7f14f8074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:32.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.723+0000 7f14fe274700 1 -- 172.21.15.60:0/3294248563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14f8102ea0 msgr2=0x7f14f8074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.723+0000 7f14fe274700 1 --2- 172.21.15.60:0/3294248563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14f8102ea0 0x7f14f8074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.723+0000 7f14fe274700 1 -- 172.21.15.60:0/3294248563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14f8101ca0 msgr2=0x7f14f8075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.723+0000 7f14fe274700 1 --2- 172.21.15.60:0/3294248563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14f8101ca0 0x7f14f8075a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.723+0000 7f14fe274700 1 -- 172.21.15.60:0/3294248563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f00095f0 con 0x7f14f8103840 2024-02-26T04:49:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.723+0000 7f14fd272700 1 --2- 172.21.15.60:0/3294248563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14f8102ea0 0x7f14f8074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:49:32.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.723+0000 7f14fe274700 1 --2- 172.21.15.60:0/3294248563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8103840 0x7f14f80745a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f14f00098e0 tx=0x7f14f0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:49:32.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.723+0000 7f14f6ffd700 1 -- 172.21.15.60:0/3294248563 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f001d070 con 0x7f14f8103840 2024-02-26T04:49:32.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.723+0000 7f14f6ffd700 1 -- 172.21.15.60:0/3294248563 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14f000fe40 con 0x7f14f8103840 2024-02-26T04:49:32.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.723+0000 7f14f6ffd700 1 -- 172.21.15.60:0/3294248563 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f00173f0 con 0x7f14f8103840 2024-02-26T04:49:32.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.723+0000 7f14ffcd7700 1 -- 172.21.15.60:0/3294248563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f81a1ef0 con 0x7f14f8103840 2024-02-26T04:49:32.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.723+0000 7f14ffcd7700 1 -- 172.21.15.60:0/3294248563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f81a2250 con 0x7f14f8103840 2024-02-26T04:49:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.724+0000 7f14f6ffd700 1 -- 172.21.15.60:0/3294248563 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f14f000f960 con 0x7f14f8103840 2024-02-26T04:49:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.724+0000 7f14ffcd7700 1 -- 172.21.15.60:0/3294248563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f8109000 con 0x7f14f8103840 2024-02-26T04:49:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.724+0000 7f14f6ffd700 1 --2- 172.21.15.60:0/3294248563 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f14e00391c0 0x7f14e003b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.724+0000 7f14f6ffd700 1 -- 172.21.15.60:0/3294248563 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f14f004ce50 con 0x7f14f8103840 2024-02-26T04:49:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.724+0000 7f14fda73700 1 --2- 172.21.15.60:0/3294248563 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f14e00391c0 0x7f14e003b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:32.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.725+0000 7f14fda73700 1 --2- 172.21.15.60:0/3294248563 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f14e00391c0 0x7f14e003b680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f14f8102d00 tx=0x7f14e800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:49:32.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.729+0000 7f14f6ffd700 1 -- 172.21.15.60:0/3294248563 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f14f0016a50 con 0x7f14f8103840 2024-02-26T04:49:32.878 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:32 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:32.540+0000 7fd5ff67c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T04:49:32.879 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:32 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:32.688+0000 7fd5ff67c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T04:49:32.879 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:32 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:32.790+0000 7fd5ff67c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T04:49:32.879 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:32 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:32.876+0000 7fd5ff67c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T04:49:32.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:32.951+0000 7f14ffcd7700 1 -- 172.21.15.60:0/3294248563 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f14f802d8e0 con 0x7f14e00391c0 2024-02-26T04:49:33.009 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:32 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:32.649+0000 7f7770ddc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T04:49:33.009 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:32 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:32.840+0000 7f7770ddc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T04:49:33.384 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:33.122+0000 7f7770ddc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T04:49:33.384 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:33.382+0000 7f7770ddc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T04:49:33.427 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:33.139+0000 7fd5ff67c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T04:49:33.427 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:33.248+0000 7fd5ff67c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T04:49:33.427 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:33.333+0000 7fd5ff67c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T04:49:33.745 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:33.427+0000 7fd5ff67c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T04:49:33.746 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:33.604+0000 7fd5ff67c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T04:49:33.746 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:33.612+0000 7f7770ddc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T04:49:34.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:34.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:34.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:34.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: Reconfiguring mgr.x (unknown last config time)... 2024-02-26T04:49:34.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T04:49:34.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T04:49:34.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:34.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: Reconfiguring daemon mgr.x on smithi060 2024-02-26T04:49:34.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:49:34.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:49:34.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:49:34.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:34.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:34.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:34.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:34.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: Reconfiguring mgr.x (unknown last config time)... 2024-02-26T04:49:34.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T04:49:34.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T04:49:34.105 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:34.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: Reconfiguring daemon mgr.x on smithi060 2024-02-26T04:49:34.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:49:34.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:49:34.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:49:34.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:34.106 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:34.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:34.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:34.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: Reconfiguring mgr.x (unknown last config time)... 2024-02-26T04:49:34.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-26T04:49:34.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-26T04:49:34.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:34.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: Reconfiguring daemon mgr.x on smithi060 2024-02-26T04:49:34.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:49:34.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:49:34.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:49:34.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:34.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:34.759 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:34 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:34.404+0000 7f7770ddc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T04:49:34.927 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:34 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:34.586+0000 7fd5ff67c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T04:49:34.927 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:34 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:34.716+0000 7fd5ff67c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T04:49:35.259 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:34 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:34.976+0000 7f7770ddc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T04:49:35.259 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:35.072+0000 7f7770ddc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T04:49:35.333 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:35.049+0000 7fd5ff67c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T04:49:35.333 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:35.138+0000 7fd5ff67c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T04:49:35.333 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:35.330+0000 7fd5ff67c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T04:49:35.677 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:35.515+0000 7fd5ff67c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T04:49:35.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-mon[35805]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:35.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:35.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:35.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:35.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:35.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:35.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:35.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-mon[35805]: Standby manager daemon z started 2024-02-26T04:49:35.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-mon[35805]: from='mgr.? 172.21.15.191:0/4013018886' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T04:49:35.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-mon[35805]: from='mgr.? 172.21.15.191:0/4013018886' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T04:49:35.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-mon[35805]: from='mgr.? 172.21.15.191:0/4013018886' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T04:49:35.679 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-mon[35805]: from='mgr.? 172.21.15.191:0/4013018886' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T04:49:35.690 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:35 smithi060 ceph-mon[32382]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:35.691 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:35 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:35.691 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:35 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:35.691 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:35 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:35.691 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:35 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:35.691 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:35 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:35.691 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:35 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:35.691 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:35 smithi060 ceph-mon[32382]: Standby manager daemon z started 2024-02-26T04:49:35.691 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:35 smithi060 ceph-mon[32382]: from='mgr.? 172.21.15.191:0/4013018886' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T04:49:35.691 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:35 smithi060 ceph-mon[32382]: from='mgr.? 172.21.15.191:0/4013018886' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T04:49:35.691 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:35 smithi060 ceph-mon[32382]: from='mgr.? 172.21.15.191:0/4013018886' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T04:49:35.691 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:35 smithi060 ceph-mon[32382]: from='mgr.? 172.21.15.191:0/4013018886' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T04:49:35.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:35.750+0000 7f14f6ffd700 1 -- 172.21.15.60:0/3294248563 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 14) v1 ==== 91350+0+0 (secure 0 0 0) 0x7f14f0017550 con 0x7f14f8103840 2024-02-26T04:49:35.759 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 04:49:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z[38210]: 2024-02-26T04:49:35.283+0000 7f7770ddc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T04:49:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:35 smithi191 ceph-mon[36352]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:35 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:35 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:35 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:35 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:35 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:35 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:35 smithi191 ceph-mon[36352]: Standby manager daemon z started 2024-02-26T04:49:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:35 smithi191 ceph-mon[36352]: from='mgr.? 172.21.15.191:0/4013018886' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-26T04:49:35.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:35 smithi191 ceph-mon[36352]: from='mgr.? 172.21.15.191:0/4013018886' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T04:49:35.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:35 smithi191 ceph-mon[36352]: from='mgr.? 172.21.15.191:0/4013018886' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-26T04:49:35.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:35 smithi191 ceph-mon[36352]: from='mgr.? 172.21.15.191:0/4013018886' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T04:49:36.060 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:35 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:35.797+0000 7fd5ff67c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T04:49:36.427 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:36 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:36.058+0000 7fd5ff67c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T04:49:36.427 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:36 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:36.292+0000 7fd5ff67c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T04:49:37.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:36 smithi191 ceph-mon[36352]: mgrmap e14: x(active, since 77s), standbys: z 2024-02-26T04:49:37.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:36 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T04:49:37.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:36 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4018630040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd04e2cc-5293-493c-a8e2-bef04f89c6ce"}]: dispatch 2024-02-26T04:49:37.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:36 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4018630040' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd04e2cc-5293-493c-a8e2-bef04f89c6ce"}]': finished 2024-02-26T04:49:37.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:36 smithi191 ceph-mon[36352]: osdmap e5: 1 total, 0 up, 1 in 2024-02-26T04:49:37.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:36 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:37.094 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:36 smithi156 ceph-mon[35805]: mgrmap e14: x(active, since 77s), standbys: z 2024-02-26T04:49:37.094 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:36 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T04:49:37.094 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:36 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4018630040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd04e2cc-5293-493c-a8e2-bef04f89c6ce"}]: dispatch 2024-02-26T04:49:37.094 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:36 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4018630040' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd04e2cc-5293-493c-a8e2-bef04f89c6ce"}]': finished 2024-02-26T04:49:37.094 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:36 smithi156 ceph-mon[35805]: osdmap e5: 1 total, 0 up, 1 in 2024-02-26T04:49:37.094 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:36 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:37.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:36 smithi060 ceph-mon[32382]: mgrmap e14: x(active, since 77s), standbys: z 2024-02-26T04:49:37.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:36 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-26T04:49:37.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:36 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4018630040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd04e2cc-5293-493c-a8e2-bef04f89c6ce"}]: dispatch 2024-02-26T04:49:37.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:36 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4018630040' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd04e2cc-5293-493c-a8e2-bef04f89c6ce"}]': finished 2024-02-26T04:49:37.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:36 smithi060 ceph-mon[32382]: osdmap e5: 1 total, 0 up, 1 in 2024-02-26T04:49:37.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:36 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:37.427 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:37 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:37.092+0000 7fd5ff67c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T04:49:37.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:37 smithi156 ceph-mon[35805]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:37.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:37 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1515850289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:49:37.928 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:37 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:37.663+0000 7fd5ff67c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T04:49:37.928 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:37 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:37.763+0000 7fd5ff67c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T04:49:38.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:37 smithi191 ceph-mon[36352]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:38.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:37 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1515850289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:49:38.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:37 smithi060 ceph-mon[32382]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:38.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:37 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1515850289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:49:38.427 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 04:49:37 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y[37187]: 2024-02-26T04:49:37.976+0000 7fd5ff67c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T04:49:38.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:38.765+0000 7f14f6ffd700 1 -- 172.21.15.60:0/3294248563 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7f14f002a360 con 0x7f14f8103840 2024-02-26T04:49:39.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:38 smithi191 ceph-mon[36352]: Standby manager daemon y started 2024-02-26T04:49:39.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:38 smithi191 ceph-mon[36352]: from='mgr.? 172.21.15.156:0/2541416773' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T04:49:39.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:38 smithi191 ceph-mon[36352]: from='mgr.? 172.21.15.156:0/2541416773' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T04:49:39.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:38 smithi191 ceph-mon[36352]: from='mgr.? 172.21.15.156:0/2541416773' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T04:49:39.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:38 smithi191 ceph-mon[36352]: from='mgr.? 172.21.15.156:0/2541416773' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T04:49:39.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:38 smithi156 ceph-mon[35805]: Standby manager daemon y started 2024-02-26T04:49:39.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:38 smithi156 ceph-mon[35805]: from='mgr.? 172.21.15.156:0/2541416773' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T04:49:39.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:38 smithi156 ceph-mon[35805]: from='mgr.? 172.21.15.156:0/2541416773' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T04:49:39.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:38 smithi156 ceph-mon[35805]: from='mgr.? 172.21.15.156:0/2541416773' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T04:49:39.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:38 smithi156 ceph-mon[35805]: from='mgr.? 172.21.15.156:0/2541416773' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T04:49:39.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:38 smithi060 ceph-mon[32382]: Standby manager daemon y started 2024-02-26T04:49:39.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:38 smithi060 ceph-mon[32382]: from='mgr.? 172.21.15.156:0/2541416773' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-26T04:49:39.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:38 smithi060 ceph-mon[32382]: from='mgr.? 172.21.15.156:0/2541416773' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-26T04:49:39.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:38 smithi060 ceph-mon[32382]: from='mgr.? 172.21.15.156:0/2541416773' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-26T04:49:39.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:38 smithi060 ceph-mon[32382]: from='mgr.? 172.21.15.156:0/2541416773' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-26T04:49:39.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:39 smithi156 ceph-mon[35805]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:39.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:39 smithi156 ceph-mon[35805]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-02-26T04:49:39.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:39 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T04:49:40.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:39 smithi191 ceph-mon[36352]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:40.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:39 smithi191 ceph-mon[36352]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-02-26T04:49:40.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:39 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T04:49:40.202 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:39 smithi060 ceph-mon[32382]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:40.202 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:39 smithi060 ceph-mon[32382]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-02-26T04:49:40.202 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:39 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-26T04:49:42.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:41 smithi156 ceph-mon[35805]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:42.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:41 smithi060 ceph-mon[32382]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:42.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:41 smithi191 ceph-mon[36352]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:44.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:43 smithi156 ceph-mon[35805]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:44.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:43 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-26T04:49:44.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:43 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:44.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:43 smithi060 ceph-mon[32382]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:44.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:43 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-26T04:49:44.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:43 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:44.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:43 smithi191 ceph-mon[36352]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:44.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:43 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-26T04:49:44.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:43 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:44.956 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:44 smithi060 ceph-mon[32382]: Deploying daemon osd.0 on smithi060 2024-02-26T04:49:45.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:44 smithi156 ceph-mon[35805]: Deploying daemon osd.0 on smithi060 2024-02-26T04:49:45.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:44 smithi191 ceph-mon[36352]: Deploying daemon osd.0 on smithi060 2024-02-26T04:49:45.911 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:45 smithi060 ceph-mon[32382]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:46.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:45 smithi156 ceph-mon[35805]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:46.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:45 smithi191 ceph-mon[36352]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:48.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:47 smithi156 ceph-mon[35805]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:48.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:47 smithi060 ceph-mon[32382]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:48.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:47 smithi191 ceph-mon[36352]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:50.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:50 smithi156 ceph-mon[35805]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:50.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:50 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:50.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:50 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:50.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:50 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:50.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:50 smithi060 ceph-mon[32382]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:50.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:50.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:50.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:50.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:50 smithi191 ceph-mon[36352]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:50.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:50 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:50.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:50 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:50.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:50 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:52.261 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:52 smithi060 ceph-mon[32382]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:52.261 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:52 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:52.261 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:52 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:52.261 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:52 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:52.261 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:52 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:52.261 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:52 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:52.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:52 smithi156 ceph-mon[35805]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:52.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:52 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:52.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:52 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:52.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:52 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:52.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:52 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:52.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:52 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:52.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:52 smithi191 ceph-mon[36352]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:52.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:52 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:52.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:52 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:52.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:52 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:52.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:52 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:52.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:52 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:52.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:52.813+0000 7f14f6ffd700 1 -- 172.21.15.60:0/3294248563 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f14f802d8e0 con 0x7f14e00391c0 2024-02-26T04:49:52.816 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 0 on host 'smithi060' 2024-02-26T04:49:52.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:52.816+0000 7f14ffcd7700 1 -- 172.21.15.60:0/3294248563 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f14e00391c0 msgr2=0x7f14e003b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:52.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:52.816+0000 7f14ffcd7700 1 --2- 172.21.15.60:0/3294248563 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f14e00391c0 0x7f14e003b680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f14f8102d00 tx=0x7f14e800b040 comp rx=0 tx=0).stop 2024-02-26T04:49:52.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:52.816+0000 7f14ffcd7700 1 -- 172.21.15.60:0/3294248563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8103840 msgr2=0x7f14f80745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:52.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:52.816+0000 7f14ffcd7700 1 --2- 172.21.15.60:0/3294248563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8103840 0x7f14f80745a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f14f00098e0 tx=0x7f14f0009580 comp rx=0 tx=0).stop 2024-02-26T04:49:52.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:52.816+0000 7f14ffcd7700 1 -- 172.21.15.60:0/3294248563 shutdown_connections 2024-02-26T04:49:52.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:52.816+0000 7f14ffcd7700 1 --2- 172.21.15.60:0/3294248563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8103840 0x7f14f80745a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:52.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:52.816+0000 7f14ffcd7700 1 --2- 172.21.15.60:0/3294248563 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f14e00391c0 0x7f14e003b680 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:52.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:52.816+0000 7f14ffcd7700 1 --2- 172.21.15.60:0/3294248563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14f8101ca0 0x7f14f8075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:52.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:52.816+0000 7f14ffcd7700 1 --2- 172.21.15.60:0/3294248563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14f8102ea0 0x7f14f8074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:52.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:52.816+0000 7f14ffcd7700 1 -- 172.21.15.60:0/3294248563 >> 172.21.15.60:0/3294248563 conn(0x7f14f80fd280 msgr2=0x7f14f8106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:49:52.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:52.816+0000 7f14ffcd7700 1 -- 172.21.15.60:0/3294248563 shutdown_connections 2024-02-26T04:49:52.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:49:52.816+0000 7f14ffcd7700 1 -- 172.21.15.60:0/3294248563 wait complete. 2024-02-26T04:49:53.479 DEBUG:teuthology.orchestra.run.smithi060:osd.0> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.0.service 2024-02-26T04:49:53.482 INFO:tasks.cephadm:Deploying osd.1 on smithi156 with /dev/vg_nvme/lv_4... 2024-02-26T04:49:53.482 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-26T04:49:53.725 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T04:49:53.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:53 smithi060 ceph-mon[32382]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:53.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:53 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:53.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:53 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:53.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:53 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:53.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:53 smithi060 ceph-mon[32382]: from='osd.0 [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T04:49:53.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:53 smithi060 ceph-mon[32382]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T04:49:54.084 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:53 smithi156 ceph-mon[35805]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:54.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:53 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:54.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:53 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:54.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:53 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:54.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:53 smithi156 ceph-mon[35805]: from='osd.0 [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T04:49:54.085 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:53 smithi156 ceph-mon[35805]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T04:49:54.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:53 smithi191 ceph-mon[36352]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:54.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:53 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:54.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:53 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:54.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:53 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:49:54.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:53 smithi191 ceph-mon[36352]: from='osd.0 [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T04:49:54.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:53 smithi191 ceph-mon[36352]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-26T04:49:54.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:54 smithi060 ceph-mon[32382]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-26T04:49:54.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:54 smithi060 ceph-mon[32382]: osdmap e6: 1 total, 0 up, 1 in 2024-02-26T04:49:54.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:54.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:54 smithi060 ceph-mon[32382]: from='osd.0 [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:49:54.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:54 smithi060 ceph-mon[32382]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:49:54.991 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:54 smithi156 ceph-mon[35805]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-26T04:49:54.991 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:54 smithi156 ceph-mon[35805]: osdmap e6: 1 total, 0 up, 1 in 2024-02-26T04:49:54.991 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:54.991 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:54 smithi156 ceph-mon[35805]: from='osd.0 [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:49:54.991 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:54 smithi156 ceph-mon[35805]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:49:55.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:54 smithi191 ceph-mon[36352]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-26T04:49:55.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:54 smithi191 ceph-mon[36352]: osdmap e6: 1 total, 0 up, 1 in 2024-02-26T04:49:55.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:55.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:54 smithi191 ceph-mon[36352]: from='osd.0 [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:49:55.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:54 smithi191 ceph-mon[36352]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:49:55.306 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 04:49:55 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-0[44691]: 2024-02-26T04:49:55.268+0000 7fc3c723d700 -1 osd.0 0 waiting for initial osdmap 2024-02-26T04:49:55.306 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 04:49:55 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-0[44691]: 2024-02-26T04:49:55.272+0000 7fc3be827700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T04:49:55.523 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-26T04:49:55.523 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-26T04:49:55.524 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-02-26T04:49:55.524 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-02-26T04:49:55.524 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00884493 s, 1.2 GB/s 2024-02-26T04:49:55.524 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-02-26T04:49:56.102 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:55 smithi156 ceph-mon[35805]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:56.103 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:55 smithi156 ceph-mon[35805]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T04:49:56.103 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:55 smithi156 ceph-mon[35805]: osdmap e7: 1 total, 0 up, 1 in 2024-02-26T04:49:56.103 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:55 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:56.103 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:55 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:56.103 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:55 smithi156 ceph-mon[35805]: from='osd.0 ' entity='osd.0' 2024-02-26T04:49:56.133 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_4 2024-02-26T04:49:56.167 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:55 smithi060 ceph-mon[32382]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:56.167 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:55 smithi060 ceph-mon[32382]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T04:49:56.167 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:55 smithi060 ceph-mon[32382]: osdmap e7: 1 total, 0 up, 1 in 2024-02-26T04:49:56.167 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:55 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:56.167 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:55 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:56.168 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:55 smithi060 ceph-mon[32382]: from='osd.0 ' entity='osd.0' 2024-02-26T04:49:56.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:55 smithi191 ceph-mon[36352]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:56.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:55 smithi191 ceph-mon[36352]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T04:49:56.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:55 smithi191 ceph-mon[36352]: osdmap e7: 1 total, 0 up, 1 in 2024-02-26T04:49:56.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:55 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:56.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:55 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:56.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:55 smithi191 ceph-mon[36352]: from='osd.0 ' entity='osd.0' 2024-02-26T04:49:56.389 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T04:49:57.160 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:56 smithi191 ceph-mon[36352]: purged_snaps scrub starts 2024-02-26T04:49:57.160 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:56 smithi191 ceph-mon[36352]: purged_snaps scrub ok 2024-02-26T04:49:57.161 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:57.161 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:56 smithi191 ceph-mon[36352]: osd.0 [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016] boot 2024-02-26T04:49:57.161 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:56 smithi191 ceph-mon[36352]: osdmap e8: 1 total, 1 up, 1 in 2024-02-26T04:49:57.161 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:57.161 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:57.161 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:57.161 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:49:57.161 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:57.161 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:57.161 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:57.161 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:57.166 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:56 smithi156 ceph-mon[35805]: purged_snaps scrub starts 2024-02-26T04:49:57.166 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:56 smithi156 ceph-mon[35805]: purged_snaps scrub ok 2024-02-26T04:49:57.166 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:57.166 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:56 smithi156 ceph-mon[35805]: osd.0 [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016] boot 2024-02-26T04:49:57.166 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:56 smithi156 ceph-mon[35805]: osdmap e8: 1 total, 1 up, 1 in 2024-02-26T04:49:57.166 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:57.166 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:57.166 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:57.166 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:49:57.167 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:57.167 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:57.167 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:57.167 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:57.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:56 smithi060 ceph-mon[32382]: purged_snaps scrub starts 2024-02-26T04:49:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:56 smithi060 ceph-mon[32382]: purged_snaps scrub ok 2024-02-26T04:49:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:56 smithi060 ceph-mon[32382]: osd.0 [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016] boot 2024-02-26T04:49:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:56 smithi060 ceph-mon[32382]: osdmap e8: 1 total, 1 up, 1 in 2024-02-26T04:49:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-26T04:49:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:49:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:49:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:49:57.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.886+0000 7f5dd51d7700 1 -- 172.21.15.156:0/2556995021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd0100c10 msgr2=0x7f5dd0101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:57.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.886+0000 7f5dd51d7700 1 --2- 172.21.15.156:0/2556995021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd0100c10 0x7f5dd0101030 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40099c0 tx=0x7f5dc4009cd0 comp rx=0 tx=0).stop 2024-02-26T04:49:57.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.886+0000 7f5dd51d7700 1 -- 172.21.15.156:0/2556995021 shutdown_connections 2024-02-26T04:49:57.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.886+0000 7f5dd51d7700 1 --2- 172.21.15.156:0/2556995021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd01027b0 0x7f5dd0109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:57.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.886+0000 7f5dd51d7700 1 --2- 172.21.15.156:0/2556995021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd0100c10 0x7f5dd0101030 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:57.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.886+0000 7f5dd51d7700 1 --2- 172.21.15.156:0/2556995021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dd0101e10 0x7f5dd0102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:57.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.886+0000 7f5dd51d7700 1 -- 172.21.15.156:0/2556995021 >> 172.21.15.156:0/2556995021 conn(0x7f5dd00fc1d0 msgr2=0x7f5dd00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:49:57.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.886+0000 7f5dd51d7700 1 -- 172.21.15.156:0/2556995021 shutdown_connections 2024-02-26T04:49:57.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.886+0000 7f5dd51d7700 1 -- 172.21.15.156:0/2556995021 wait complete. 2024-02-26T04:49:57.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.887+0000 7f5dd51d7700 1 Processor -- start 2024-02-26T04:49:57.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.887+0000 7f5dd51d7700 1 -- start start 2024-02-26T04:49:57.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.887+0000 7f5dd51d7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dd0100c10 0x7f5dd0197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:57.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.887+0000 7f5dd51d7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd0101e10 0x7f5dd01982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:57.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.887+0000 7f5dd51d7700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd01027b0 0x7f5dd019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:57.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.887+0000 7f5dd51d7700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd019f860 con 0x7f5dd01027b0 2024-02-26T04:49:57.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.887+0000 7f5dd51d7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd019f9a0 con 0x7f5dd0101e10 2024-02-26T04:49:57.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.887+0000 7f5dd51d7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd019fb60 con 0x7f5dd0100c10 2024-02-26T04:49:57.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5dce59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd0101e10 0x7f5dd01982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:57.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5dce59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd0101e10 0x7f5dd01982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56570/0 (socket says 172.21.15.156:56570) 2024-02-26T04:49:57.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5dce59c700 1 -- 172.21.15.156:0/820644150 learned_addr learned my addr 172.21.15.156:0/820644150 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-02-26T04:49:57.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5dced9d700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dd0100c10 0x7f5dd0197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:57.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5dcf59e700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd01027b0 0x7f5dd019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:57.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5dce59c700 1 -- 172.21.15.156:0/820644150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dd0100c10 msgr2=0x7f5dd0197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:57.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5dce59c700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dd0100c10 0x7f5dd0197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:57.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5dce59c700 1 -- 172.21.15.156:0/820644150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd01027b0 msgr2=0x7f5dd019f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:49:57.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5dce59c700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd01027b0 0x7f5dd019f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:49:57.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5dce59c700 1 -- 172.21.15.156:0/820644150 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40095f0 con 0x7f5dd0101e10 2024-02-26T04:49:57.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5dcf59e700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd01027b0 0x7f5dd019f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:49:57.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5dce59c700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd0101e10 0x7f5dd01982a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000e9a0 tx=0x7f5dc000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:49:57.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5db7fff700 1 -- 172.21.15.156:0/820644150 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0009790 con 0x7f5dd0101e10 2024-02-26T04:49:57.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5db7fff700 1 -- 172.21.15.156:0/820644150 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5dc000cbd0 con 0x7f5dd0101e10 2024-02-26T04:49:57.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5dd51d7700 1 -- 172.21.15.156:0/820644150 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd019fe40 con 0x7f5dd0101e10 2024-02-26T04:49:57.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5db7fff700 1 -- 172.21.15.156:0/820644150 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0018680 con 0x7f5dd0101e10 2024-02-26T04:49:57.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.888+0000 7f5dd51d7700 1 -- 172.21.15.156:0/820644150 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd01a02d0 con 0x7f5dd0101e10 2024-02-26T04:49:57.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.889+0000 7f5dd51d7700 1 -- 172.21.15.156:0/820644150 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd010c5b0 con 0x7f5dd0101e10 2024-02-26T04:49:57.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.890+0000 7f5db7fff700 1 -- 172.21.15.156:0/820644150 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7f5dc000cd40 con 0x7f5dd0101e10 2024-02-26T04:49:57.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.891+0000 7f5db7fff700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5db80a20c0 0x7f5db80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:49:57.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.891+0000 7f5db7fff700 1 -- 172.21.15.156:0/820644150 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f5dc0014070 con 0x7f5dd0101e10 2024-02-26T04:49:57.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.891+0000 7f5dced9d700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5db80a20c0 0x7f5db80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:49:57.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.891+0000 7f5dced9d700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5db80a20c0 0x7f5db80a4580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009990 tx=0x7f5dc4003830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:49:57.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:57.895+0000 7f5db7fff700 1 -- 172.21.15.156:0/820644150 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5dc0052d20 con 0x7f5dd0101e10 2024-02-26T04:49:57.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:57 smithi156 ceph-mon[35805]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:57.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:57 smithi156 ceph-mon[35805]: Detected new or changed devices on smithi060 2024-02-26T04:49:57.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:57 smithi156 ceph-mon[35805]: Adjusting osd_memory_target on smithi060 to 17093M 2024-02-26T04:49:58.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:49:58.098+0000 7f5dd51d7700 1 -- 172.21.15.156:0/820644150 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5dd00fe350 con 0x7f5db80a20c0 2024-02-26T04:49:58.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:57 smithi060 ceph-mon[32382]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:58.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:57 smithi060 ceph-mon[32382]: Detected new or changed devices on smithi060 2024-02-26T04:49:58.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:57 smithi060 ceph-mon[32382]: Adjusting osd_memory_target on smithi060 to 17093M 2024-02-26T04:49:58.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:57 smithi191 ceph-mon[36352]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-26T04:49:58.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:57 smithi191 ceph-mon[36352]: Detected new or changed devices on smithi060 2024-02-26T04:49:58.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:57 smithi191 ceph-mon[36352]: Adjusting osd_memory_target on smithi060 to 17093M 2024-02-26T04:49:58.989 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:58 smithi156 ceph-mon[35805]: osdmap e9: 1 total, 1 up, 1 in 2024-02-26T04:49:58.989 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:58 smithi156 ceph-mon[35805]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:49:58.989 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:58 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:49:58.989 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:58 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:49:58.989 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:58 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:59.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:58 smithi060 ceph-mon[32382]: osdmap e9: 1 total, 1 up, 1 in 2024-02-26T04:49:59.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:58 smithi060 ceph-mon[32382]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:49:59.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:49:59.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:49:59.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:59.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:58 smithi191 ceph-mon[36352]: osdmap e9: 1 total, 1 up, 1 in 2024-02-26T04:49:59.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:58 smithi191 ceph-mon[36352]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:49:59.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:58 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:49:59.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:58 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:49:59.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:58 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:49:59.883 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:49:59 smithi156 ceph-mon[35805]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:00.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:49:59 smithi060 ceph-mon[32382]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:00.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:49:59 smithi191 ceph-mon[36352]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:01.062 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:00 smithi156 ceph-mon[35805]: overall HEALTH_OK 2024-02-26T04:50:01.062 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:00 smithi156 ceph-mon[35805]: from='client.? 172.21.15.156:0/2142722129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c43bde4b-f31d-4d09-8410-583371866db3"}]: dispatch 2024-02-26T04:50:01.062 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:00 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c43bde4b-f31d-4d09-8410-583371866db3"}]: dispatch 2024-02-26T04:50:01.062 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:00 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c43bde4b-f31d-4d09-8410-583371866db3"}]': finished 2024-02-26T04:50:01.062 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:00 smithi156 ceph-mon[35805]: osdmap e10: 2 total, 1 up, 2 in 2024-02-26T04:50:01.062 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:00 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:01.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:00 smithi060 ceph-mon[32382]: overall HEALTH_OK 2024-02-26T04:50:01.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:00 smithi060 ceph-mon[32382]: from='client.? 172.21.15.156:0/2142722129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c43bde4b-f31d-4d09-8410-583371866db3"}]: dispatch 2024-02-26T04:50:01.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:00 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c43bde4b-f31d-4d09-8410-583371866db3"}]: dispatch 2024-02-26T04:50:01.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:00 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c43bde4b-f31d-4d09-8410-583371866db3"}]': finished 2024-02-26T04:50:01.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:00 smithi060 ceph-mon[32382]: osdmap e10: 2 total, 1 up, 2 in 2024-02-26T04:50:01.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:00 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:01.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:00 smithi191 ceph-mon[36352]: overall HEALTH_OK 2024-02-26T04:50:01.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:00 smithi191 ceph-mon[36352]: from='client.? 172.21.15.156:0/2142722129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c43bde4b-f31d-4d09-8410-583371866db3"}]: dispatch 2024-02-26T04:50:01.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:00 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c43bde4b-f31d-4d09-8410-583371866db3"}]: dispatch 2024-02-26T04:50:01.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:00 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c43bde4b-f31d-4d09-8410-583371866db3"}]': finished 2024-02-26T04:50:01.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:00 smithi191 ceph-mon[36352]: osdmap e10: 2 total, 1 up, 2 in 2024-02-26T04:50:01.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:00 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:02.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:01 smithi156 ceph-mon[35805]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:02.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:01 smithi156 ceph-mon[35805]: from='client.? 172.21.15.156:0/3387253855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:50:02.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:01 smithi060 ceph-mon[32382]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:02.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:01 smithi060 ceph-mon[32382]: from='client.? 172.21.15.156:0/3387253855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:50:02.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:01 smithi191 ceph-mon[36352]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:02.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:01 smithi191 ceph-mon[36352]: from='client.? 172.21.15.156:0/3387253855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:50:04.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:03 smithi156 ceph-mon[35805]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:04.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:03 smithi060 ceph-mon[32382]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:04.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:03 smithi191 ceph-mon[36352]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:05.919 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:05 smithi156 ceph-mon[35805]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:06.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:05 smithi060 ceph-mon[32382]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:05 smithi191 ceph-mon[36352]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:08.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:07 smithi156 ceph-mon[35805]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:08.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:07 smithi060 ceph-mon[32382]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:08.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:07 smithi191 ceph-mon[36352]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:09.025 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:08 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-26T04:50:09.025 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:08 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:09.025 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:08 smithi156 ceph-mon[35805]: Deploying daemon osd.1 on smithi156 2024-02-26T04:50:09.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-26T04:50:09.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:09.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:08 smithi060 ceph-mon[32382]: Deploying daemon osd.1 on smithi156 2024-02-26T04:50:09.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-26T04:50:09.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:09.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:08 smithi191 ceph-mon[36352]: Deploying daemon osd.1 on smithi156 2024-02-26T04:50:09.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:09 smithi156 ceph-mon[35805]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:10.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:09 smithi060 ceph-mon[32382]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:10.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:09 smithi191 ceph-mon[36352]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:12.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:11 smithi156 ceph-mon[35805]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:12.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:11 smithi060 ceph-mon[32382]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:12.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:11 smithi191 ceph-mon[36352]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:14.113 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:14 smithi156 ceph-mon[35805]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:14.113 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:14 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:14.113 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:14 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:14.113 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:14 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:50:14.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:14 smithi060 ceph-mon[32382]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:14.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:14 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:14.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:14 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:14.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:14 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:50:14.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:14 smithi191 ceph-mon[36352]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:14.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:14 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:14.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:14 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:14.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:14 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:50:16.408 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:16 smithi156 ceph-mon[35805]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:16.409 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:16 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:16.409 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:16 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:16.409 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:16 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:16.409 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:16 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:50:16.409 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:16 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:16.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:16 smithi060 ceph-mon[32382]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:16.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:16.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:16.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:16.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:50:16.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:16.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:16 smithi191 ceph-mon[36352]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:16.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:16.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:16.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:16.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:50:16.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:16.765 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:50:16.763+0000 7f5db7fff700 1 -- 172.21.15.156:0/820644150 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5dd00fe350 con 0x7f5db80a20c0 2024-02-26T04:50:16.766 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 1 on host 'smithi156' 2024-02-26T04:50:16.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:50:16.765+0000 7f5dd51d7700 1 -- 172.21.15.156:0/820644150 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5db80a20c0 msgr2=0x7f5db80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:50:16.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:50:16.765+0000 7f5dd51d7700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5db80a20c0 0x7f5db80a4580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009990 tx=0x7f5dc4003830 comp rx=0 tx=0).stop 2024-02-26T04:50:16.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:50:16.765+0000 7f5dd51d7700 1 -- 172.21.15.156:0/820644150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd0101e10 msgr2=0x7f5dd01982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:50:16.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:50:16.765+0000 7f5dd51d7700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd0101e10 0x7f5dd01982a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000e9a0 tx=0x7f5dc000ecb0 comp rx=0 tx=0).stop 2024-02-26T04:50:16.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:50:16.765+0000 7f5dd51d7700 1 -- 172.21.15.156:0/820644150 shutdown_connections 2024-02-26T04:50:16.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:50:16.765+0000 7f5dd51d7700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd01027b0 0x7f5dd019f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:16.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:50:16.765+0000 7f5dd51d7700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dd0100c10 0x7f5dd0197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:16.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:50:16.765+0000 7f5dd51d7700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5db80a20c0 0x7f5db80a4580 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:16.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:50:16.765+0000 7f5dd51d7700 1 --2- 172.21.15.156:0/820644150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd0101e10 0x7f5dd01982a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:16.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:50:16.765+0000 7f5dd51d7700 1 -- 172.21.15.156:0/820644150 >> 172.21.15.156:0/820644150 conn(0x7f5dd00fc1d0 msgr2=0x7f5dd00fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:50:16.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:50:16.766+0000 7f5dd51d7700 1 -- 172.21.15.156:0/820644150 shutdown_connections 2024-02-26T04:50:16.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:50:16.766+0000 7f5dd51d7700 1 -- 172.21.15.156:0/820644150 wait complete. 2024-02-26T04:50:17.382 DEBUG:teuthology.orchestra.run.smithi156:osd.1> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.1.service 2024-02-26T04:50:17.385 INFO:tasks.cephadm:Deploying osd.2 on smithi191 with /dev/vg_nvme/lv_4... 2024-02-26T04:50:17.385 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-26T04:50:17.630 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T04:50:17.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:17 smithi060 ceph-mon[32382]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:17.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:17.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:17.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:50:17.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:17 smithi060 ceph-mon[32382]: from='osd.1 [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T04:50:17.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:17 smithi060 ceph-mon[32382]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T04:50:18.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:17 smithi191 ceph-mon[36352]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:18.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:18.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:18.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:50:18.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:17 smithi191 ceph-mon[36352]: from='osd.1 [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T04:50:18.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:17 smithi191 ceph-mon[36352]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T04:50:18.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:17 smithi156 ceph-mon[35805]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:18.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:17 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:18.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:17 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:18.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:17 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:50:18.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:17 smithi156 ceph-mon[35805]: from='osd.1 [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T04:50:18.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:17 smithi156 ceph-mon[35805]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-26T04:50:18.914 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:18 smithi156 ceph-mon[35805]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-26T04:50:18.914 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:18 smithi156 ceph-mon[35805]: from='osd.1 [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:50:18.914 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:18 smithi156 ceph-mon[35805]: osdmap e11: 2 total, 1 up, 2 in 2024-02-26T04:50:18.914 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:18.914 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:18 smithi156 ceph-mon[35805]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:50:18.914 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 04:50:18 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-1[40415]: 2024-02-26T04:50:18.733+0000 7f877b1ae700 -1 osd.1 0 waiting for initial osdmap 2024-02-26T04:50:18.914 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 04:50:18 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-1[40415]: 2024-02-26T04:50:18.738+0000 7f8776fa1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T04:50:18.951 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:18 smithi191 ceph-mon[36352]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-26T04:50:18.952 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:18 smithi191 ceph-mon[36352]: from='osd.1 [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:50:18.952 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:18 smithi191 ceph-mon[36352]: osdmap e11: 2 total, 1 up, 2 in 2024-02-26T04:50:18.952 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:18 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:18.952 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:18 smithi191 ceph-mon[36352]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:50:19.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:18 smithi060 ceph-mon[32382]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-26T04:50:19.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:18 smithi060 ceph-mon[32382]: from='osd.1 [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:50:19.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:18 smithi060 ceph-mon[32382]: osdmap e11: 2 total, 1 up, 2 in 2024-02-26T04:50:19.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:18 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:19.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:18 smithi060 ceph-mon[32382]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:50:19.488 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-26T04:50:19.488 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-26T04:50:19.488 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-02-26T04:50:19.488 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-02-26T04:50:19.488 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124204 s, 844 MB/s 2024-02-26T04:50:19.489 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-02-26T04:50:19.861 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:19 smithi191 ceph-mon[36352]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:19.861 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:19 smithi191 ceph-mon[36352]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-26T04:50:19.861 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:19 smithi191 ceph-mon[36352]: osdmap e12: 2 total, 1 up, 2 in 2024-02-26T04:50:19.861 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:19.861 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:19.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:19 smithi156 ceph-mon[35805]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:19.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:19 smithi156 ceph-mon[35805]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-26T04:50:19.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:19 smithi156 ceph-mon[35805]: osdmap e12: 2 total, 1 up, 2 in 2024-02-26T04:50:19.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:19.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:20.143 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_4 2024-02-26T04:50:20.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:19 smithi060 ceph-mon[32382]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:19 smithi060 ceph-mon[32382]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-26T04:50:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:19 smithi060 ceph-mon[32382]: osdmap e12: 2 total, 1 up, 2 in 2024-02-26T04:50:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:20.386 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T04:50:21.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:20 smithi191 ceph-mon[36352]: purged_snaps scrub starts 2024-02-26T04:50:21.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:20 smithi191 ceph-mon[36352]: purged_snaps scrub ok 2024-02-26T04:50:21.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:20 smithi191 ceph-mon[36352]: osd.1 [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364] boot 2024-02-26T04:50:21.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:20 smithi191 ceph-mon[36352]: osdmap e13: 2 total, 2 up, 2 in 2024-02-26T04:50:21.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:21.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:21.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:21.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:50:21.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:21.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:21.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:50:21.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:21.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:20 smithi156 ceph-mon[35805]: purged_snaps scrub starts 2024-02-26T04:50:21.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:20 smithi156 ceph-mon[35805]: purged_snaps scrub ok 2024-02-26T04:50:21.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:20 smithi156 ceph-mon[35805]: osd.1 [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364] boot 2024-02-26T04:50:21.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:20 smithi156 ceph-mon[35805]: osdmap e13: 2 total, 2 up, 2 in 2024-02-26T04:50:21.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:21.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:21.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:21.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:50:21.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:21.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:21.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:50:21.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:21.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:20 smithi060 ceph-mon[32382]: purged_snaps scrub starts 2024-02-26T04:50:21.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:20 smithi060 ceph-mon[32382]: purged_snaps scrub ok 2024-02-26T04:50:21.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:20 smithi060 ceph-mon[32382]: osd.1 [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364] boot 2024-02-26T04:50:21.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:20 smithi060 ceph-mon[32382]: osdmap e13: 2 total, 2 up, 2 in 2024-02-26T04:50:21.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-26T04:50:21.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:21.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:21.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:50:21.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:21.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:21.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:50:21.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.942+0000 7f9bcba37700 1 -- 172.21.15.191:0/1039913019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc41027b0 msgr2=0x7f9bc4109060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:50:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.942+0000 7f9bcba37700 1 --2- 172.21.15.191:0/1039913019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc41027b0 0x7f9bc4109060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009990 tx=0x7f9bbc009ca0 comp rx=0 tx=0).stop 2024-02-26T04:50:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.942+0000 7f9bcba37700 1 -- 172.21.15.191:0/1039913019 shutdown_connections 2024-02-26T04:50:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.942+0000 7f9bcba37700 1 --2- 172.21.15.191:0/1039913019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc41027b0 0x7f9bc4109060 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.942+0000 7f9bcba37700 1 --2- 172.21.15.191:0/1039913019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc4100c10 0x7f9bc4101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.942+0000 7f9bcba37700 1 --2- 172.21.15.191:0/1039913019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc4101e10 0x7f9bc4102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.942+0000 7f9bcba37700 1 -- 172.21.15.191:0/1039913019 >> 172.21.15.191:0/1039913019 conn(0x7f9bc40fc1d0 msgr2=0x7f9bc40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:50:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.942+0000 7f9bcba37700 1 -- 172.21.15.191:0/1039913019 shutdown_connections 2024-02-26T04:50:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.942+0000 7f9bcba37700 1 -- 172.21.15.191:0/1039913019 wait complete. 2024-02-26T04:50:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bcba37700 1 Processor -- start 2024-02-26T04:50:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bcba37700 1 -- start start 2024-02-26T04:50:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bcba37700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc4100c10 0x7f9bc4196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:50:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bcba37700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc4101e10 0x7f9bc4196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:50:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bcba37700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc41027b0 0x7f9bc419e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:50:21.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bcba37700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc419e620 con 0x7f9bc41027b0 2024-02-26T04:50:21.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bcba37700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc419e760 con 0x7f9bc4101e10 2024-02-26T04:50:21.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bcba37700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc419e8f0 con 0x7f9bc4100c10 2024-02-26T04:50:21.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bc97d3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc4100c10 0x7f9bc4196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:50:21.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bc97d3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc4100c10 0x7f9bc4196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:51844/0 (socket says 172.21.15.191:51844) 2024-02-26T04:50:21.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bc97d3700 1 -- 172.21.15.191:0/569285182 learned_addr learned my addr 172.21.15.191:0/569285182 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-26T04:50:21.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bc97d3700 1 -- 172.21.15.191:0/569285182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc4101e10 msgr2=0x7f9bc4196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:50:21.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bc97d3700 1 --2- 172.21.15.191:0/569285182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc4101e10 0x7f9bc4196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:21.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bc97d3700 1 -- 172.21.15.191:0/569285182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc41027b0 msgr2=0x7f9bc419e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:50:21.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bc97d3700 1 --2- 172.21.15.191:0/569285182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc41027b0 0x7f9bc419e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:21.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bc97d3700 1 -- 172.21.15.191:0/569285182 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc0095f0 con 0x7f9bc4100c10 2024-02-26T04:50:21.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.943+0000 7f9bc97d3700 1 --2- 172.21.15.191:0/569285182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc4100c10 0x7f9bc4196a90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0007f10 tx=0x7f9bc000f5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:50:21.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.944+0000 7f9bba7fc700 1 -- 172.21.15.191:0/569285182 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000fea0 con 0x7f9bc4100c10 2024-02-26T04:50:21.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.944+0000 7f9bba7fc700 1 -- 172.21.15.191:0/569285182 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bc0004020 con 0x7f9bc4100c10 2024-02-26T04:50:21.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.944+0000 7f9bcba37700 1 -- 172.21.15.191:0/569285182 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc406a3d0 con 0x7f9bc4100c10 2024-02-26T04:50:21.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.944+0000 7f9bba7fc700 1 -- 172.21.15.191:0/569285182 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00126c0 con 0x7f9bc4100c10 2024-02-26T04:50:21.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.944+0000 7f9bcba37700 1 -- 172.21.15.191:0/569285182 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc406a8d0 con 0x7f9bc4100c10 2024-02-26T04:50:21.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.945+0000 7f9bcba37700 1 -- 172.21.15.191:0/569285182 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc4068930 con 0x7f9bc4100c10 2024-02-26T04:50:21.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.946+0000 7f9bba7fc700 1 -- 172.21.15.191:0/569285182 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7f9bc0004190 con 0x7f9bc4100c10 2024-02-26T04:50:21.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.946+0000 7f9bba7fc700 1 --2- 172.21.15.191:0/569285182 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9bb00a20c0 0x7f9bb00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:50:21.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.946+0000 7f9bba7fc700 1 -- 172.21.15.191:0/569285182 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f9bc00cebb0 con 0x7f9bc4100c10 2024-02-26T04:50:21.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.946+0000 7f9bc8fd2700 1 --2- 172.21.15.191:0/569285182 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9bb00a20c0 0x7f9bb00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:50:21.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.947+0000 7f9bc8fd2700 1 --2- 172.21.15.191:0/569285182 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9bb00a20c0 0x7f9bb00a4580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4005950 tx=0x7f9bb40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:50:21.954 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:21.950+0000 7f9bba7fc700 1 -- 172.21.15.191:0/569285182 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9bc00993b0 con 0x7f9bc4100c10 2024-02-26T04:50:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:21 smithi191 ceph-mon[36352]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:21 smithi191 ceph-mon[36352]: Detected new or changed devices on smithi156 2024-02-26T04:50:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:21 smithi191 ceph-mon[36352]: Adjusting osd_memory_target on smithi156 to 17080M 2024-02-26T04:50:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:21 smithi191 ceph-mon[36352]: osdmap e14: 2 total, 2 up, 2 in 2024-02-26T04:50:22.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:22.153+0000 7f9bcba37700 1 -- 172.21.15.191:0/569285182 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bc4107cd0 con 0x7f9bb00a20c0 2024-02-26T04:50:22.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:21 smithi156 ceph-mon[35805]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:22.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:21 smithi156 ceph-mon[35805]: Detected new or changed devices on smithi156 2024-02-26T04:50:22.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:21 smithi156 ceph-mon[35805]: Adjusting osd_memory_target on smithi156 to 17080M 2024-02-26T04:50:22.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:21 smithi156 ceph-mon[35805]: osdmap e14: 2 total, 2 up, 2 in 2024-02-26T04:50:22.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:21 smithi060 ceph-mon[32382]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-26T04:50:22.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:21 smithi060 ceph-mon[32382]: Detected new or changed devices on smithi156 2024-02-26T04:50:22.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:21 smithi060 ceph-mon[32382]: Adjusting osd_memory_target on smithi156 to 17080M 2024-02-26T04:50:22.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:21 smithi060 ceph-mon[32382]: osdmap e14: 2 total, 2 up, 2 in 2024-02-26T04:50:23.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:22 smithi191 ceph-mon[36352]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:50:23.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:22 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:50:23.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:22 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:50:23.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:22 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:23.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:22 smithi156 ceph-mon[35805]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:50:23.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:22 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:50:23.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:22 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:50:23.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:22 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:23.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:22 smithi060 ceph-mon[32382]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:50:23.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:22 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:50:23.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:22 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:50:23.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:22 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:23.967 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:23 smithi191 ceph-mon[36352]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:24.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:23 smithi156 ceph-mon[35805]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:24.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:23 smithi060 ceph-mon[32382]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:25.099 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:24 smithi191 ceph-mon[36352]: from='client.? 172.21.15.191:0/505136916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e"}]: dispatch 2024-02-26T04:50:25.100 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:24 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e"}]: dispatch 2024-02-26T04:50:25.100 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:24 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e"}]': finished 2024-02-26T04:50:25.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:24 smithi156 ceph-mon[35805]: from='client.? 172.21.15.191:0/505136916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e"}]: dispatch 2024-02-26T04:50:25.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:24 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e"}]: dispatch 2024-02-26T04:50:25.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:24 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e"}]': finished 2024-02-26T04:50:25.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:24 smithi060 ceph-mon[32382]: from='client.? 172.21.15.191:0/505136916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e"}]: dispatch 2024-02-26T04:50:25.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:24 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e"}]: dispatch 2024-02-26T04:50:25.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:24 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e"}]': finished 2024-02-26T04:50:26.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:25 smithi156 ceph-mon[35805]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:26.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:25 smithi156 ceph-mon[35805]: osdmap e15: 3 total, 2 up, 3 in 2024-02-26T04:50:26.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:25 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:26.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:25 smithi156 ceph-mon[35805]: from='client.? 172.21.15.191:0/2284169613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:50:26.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:25 smithi060 ceph-mon[32382]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:26.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:25 smithi060 ceph-mon[32382]: osdmap e15: 3 total, 2 up, 3 in 2024-02-26T04:50:26.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:25 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:26.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:25 smithi060 ceph-mon[32382]: from='client.? 172.21.15.191:0/2284169613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:50:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:25 smithi191 ceph-mon[36352]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:25 smithi191 ceph-mon[36352]: osdmap e15: 3 total, 2 up, 3 in 2024-02-26T04:50:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:25 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:25 smithi191 ceph-mon[36352]: from='client.? 172.21.15.191:0/2284169613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:50:28.061 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:27 smithi191 ceph-mon[36352]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:28.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:27 smithi156 ceph-mon[35805]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:28.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:27 smithi060 ceph-mon[32382]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:29.921 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:29 smithi191 ceph-mon[36352]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:30.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:29 smithi156 ceph-mon[35805]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:30.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:29 smithi060 ceph-mon[32382]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:32.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:31 smithi191 ceph-mon[36352]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:32.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:31 smithi156 ceph-mon[35805]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:32.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:31 smithi060 ceph-mon[32382]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:33.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:32 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-26T04:50:33.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:32 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:33.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:32 smithi191 ceph-mon[36352]: Deploying daemon osd.2 on smithi191 2024-02-26T04:50:33.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:32 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-26T04:50:33.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:32 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:33.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:32 smithi156 ceph-mon[35805]: Deploying daemon osd.2 on smithi191 2024-02-26T04:50:33.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:32 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-26T04:50:33.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:32 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:33.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:32 smithi060 ceph-mon[32382]: Deploying daemon osd.2 on smithi191 2024-02-26T04:50:34.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:33 smithi156 ceph-mon[35805]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:34.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:33 smithi060 ceph-mon[32382]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:34.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:33 smithi191 ceph-mon[36352]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:35.917 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:35 smithi191 ceph-mon[36352]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:36.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:35 smithi156 ceph-mon[35805]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:36.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:35 smithi060 ceph-mon[32382]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:38.104 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:37 smithi191 ceph-mon[36352]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:38.104 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:37 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:50:38.104 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:37 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:38.104 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:37 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:38.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:37 smithi156 ceph-mon[35805]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:38.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:37 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:50:38.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:37 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:38.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:37 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:38.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:37 smithi060 ceph-mon[32382]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:38.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:37 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:50:38.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:37 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:38.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:37 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:39.983 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:39 smithi191 ceph-mon[36352]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:40.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:39 smithi156 ceph-mon[35805]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:40.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:39 smithi060 ceph-mon[32382]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:40.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:40.668+0000 7f9bba7fc700 1 -- 172.21.15.191:0/569285182 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9bc4107cd0 con 0x7f9bb00a20c0 2024-02-26T04:50:40.671 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 2 on host 'smithi191' 2024-02-26T04:50:40.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:40.671+0000 7f9bcba37700 1 -- 172.21.15.191:0/569285182 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9bb00a20c0 msgr2=0x7f9bb00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:50:40.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:40.671+0000 7f9bcba37700 1 --2- 172.21.15.191:0/569285182 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9bb00a20c0 0x7f9bb00a4580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4005950 tx=0x7f9bb40058e0 comp rx=0 tx=0).stop 2024-02-26T04:50:40.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:40.671+0000 7f9bcba37700 1 -- 172.21.15.191:0/569285182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc4100c10 msgr2=0x7f9bc4196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:50:40.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:40.671+0000 7f9bcba37700 1 --2- 172.21.15.191:0/569285182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc4100c10 0x7f9bc4196a90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0007f10 tx=0x7f9bc000f5c0 comp rx=0 tx=0).stop 2024-02-26T04:50:40.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:40.671+0000 7f9bcba37700 1 -- 172.21.15.191:0/569285182 shutdown_connections 2024-02-26T04:50:40.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:40.671+0000 7f9bcba37700 1 --2- 172.21.15.191:0/569285182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc41027b0 0x7f9bc419e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:40.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:40.671+0000 7f9bcba37700 1 --2- 172.21.15.191:0/569285182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc4100c10 0x7f9bc4196a90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:40.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:40.671+0000 7f9bcba37700 1 --2- 172.21.15.191:0/569285182 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9bb00a20c0 0x7f9bb00a4580 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:40.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:40.671+0000 7f9bcba37700 1 --2- 172.21.15.191:0/569285182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9bc4101e10 0x7f9bc4196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:40.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:40.671+0000 7f9bcba37700 1 -- 172.21.15.191:0/569285182 >> 172.21.15.191:0/569285182 conn(0x7f9bc40fc1d0 msgr2=0x7f9bc4105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:50:40.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:40.671+0000 7f9bcba37700 1 -- 172.21.15.191:0/569285182 shutdown_connections 2024-02-26T04:50:40.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:50:40.671+0000 7f9bcba37700 1 -- 172.21.15.191:0/569285182 wait complete. 2024-02-26T04:50:40.961 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:40 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:40.961 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:40 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:40.961 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:40 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:40.961 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:40 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:50:40.962 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:40 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:40.962 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:40 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:40.962 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:40 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:40.962 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:40 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:50:41.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:40 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:41.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:40 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:41.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:40 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:41.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:40 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:50:41.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:40 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:41.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:40 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:41.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:40 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:41.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:40 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:50:41.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:40 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:41.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:40 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:41.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:40 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:41.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:40 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:50:41.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:40 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:41.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:40 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:41.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:40 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:41.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:40 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:50:41.317 DEBUG:teuthology.orchestra.run.smithi191:osd.2> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.2.service 2024-02-26T04:50:41.319 INFO:tasks.cephadm:Deploying osd.3 on smithi060 with /dev/vg_nvme/lv_3... 2024-02-26T04:50:41.319 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-26T04:50:41.569 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:50:41.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:41 smithi060 ceph-mon[32382]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:41.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:41 smithi060 ceph-mon[32382]: from='osd.2 [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T04:50:41.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:41 smithi060 ceph-mon[32382]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T04:50:42.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:41 smithi191 ceph-mon[36352]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:42.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:41 smithi191 ceph-mon[36352]: from='osd.2 [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T04:50:42.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:41 smithi191 ceph-mon[36352]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T04:50:42.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:41 smithi156 ceph-mon[35805]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:42.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:41 smithi156 ceph-mon[35805]: from='osd.2 [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T04:50:42.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:41 smithi156 ceph-mon[35805]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-26T04:50:43.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:42 smithi060 ceph-mon[32382]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-26T04:50:43.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:42 smithi060 ceph-mon[32382]: osdmap e16: 3 total, 2 up, 3 in 2024-02-26T04:50:43.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:42 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:43.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:42 smithi060 ceph-mon[32382]: from='osd.2 [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:50:43.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:42 smithi060 ceph-mon[32382]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:50:43.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:42 smithi156 ceph-mon[35805]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-26T04:50:43.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:42 smithi156 ceph-mon[35805]: osdmap e16: 3 total, 2 up, 3 in 2024-02-26T04:50:43.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:42 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:43.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:42 smithi156 ceph-mon[35805]: from='osd.2 [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:50:43.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:42 smithi156 ceph-mon[35805]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:50:43.245 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:42 smithi191 ceph-mon[36352]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-26T04:50:43.246 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:42 smithi191 ceph-mon[36352]: osdmap e16: 3 total, 2 up, 3 in 2024-02-26T04:50:43.246 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:42 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:43.246 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:42 smithi191 ceph-mon[36352]: from='osd.2 [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:50:43.246 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:42 smithi191 ceph-mon[36352]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:50:43.509 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 04:50:43 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-2[41516]: 2024-02-26T04:50:43.241+0000 7f9404ade700 -1 osd.2 0 waiting for initial osdmap 2024-02-26T04:50:43.509 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 04:50:43 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-2[41516]: 2024-02-26T04:50:43.246+0000 7f93fc8c9700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T04:50:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-26T04:50:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-26T04:50:43.766 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-02-26T04:50:43.767 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-02-26T04:50:43.767 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00845459 s, 1.2 GB/s 2024-02-26T04:50:43.767 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-02-26T04:50:44.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:43 smithi191 ceph-mon[36352]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:44.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:43 smithi191 ceph-mon[36352]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-26T04:50:44.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:43 smithi191 ceph-mon[36352]: osdmap e17: 3 total, 2 up, 3 in 2024-02-26T04:50:44.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:43 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:44.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:43 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:44.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:43 smithi191 ceph-mon[36352]: from='osd.2 ' entity='osd.2' 2024-02-26T04:50:44.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:43 smithi156 ceph-mon[35805]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:44.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:43 smithi156 ceph-mon[35805]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-26T04:50:44.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:43 smithi156 ceph-mon[35805]: osdmap e17: 3 total, 2 up, 3 in 2024-02-26T04:50:44.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:43 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:44.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:43 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:44.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:43 smithi156 ceph-mon[35805]: from='osd.2 ' entity='osd.2' 2024-02-26T04:50:44.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:43 smithi060 ceph-mon[32382]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:44.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:43 smithi060 ceph-mon[32382]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-26T04:50:44.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:43 smithi060 ceph-mon[32382]: osdmap e17: 3 total, 2 up, 3 in 2024-02-26T04:50:44.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:43 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:44.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:43 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:44.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:43 smithi060 ceph-mon[32382]: from='osd.2 ' entity='osd.2' 2024-02-26T04:50:44.588 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi060:vg_nvme/lv_3 2024-02-26T04:50:44.821 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:44 smithi060 ceph-mon[32382]: purged_snaps scrub starts 2024-02-26T04:50:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:44 smithi060 ceph-mon[32382]: purged_snaps scrub ok 2024-02-26T04:50:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:44 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:44 smithi060 ceph-mon[32382]: osd.2 [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954] boot 2024-02-26T04:50:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:44 smithi060 ceph-mon[32382]: osdmap e18: 3 total, 3 up, 3 in 2024-02-26T04:50:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:44 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:44 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-26T04:50:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:44 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:44 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:44 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:50:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:44 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:44 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:44 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:50:44.822 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:44 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:44.833 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:50:45.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:44 smithi156 ceph-mon[35805]: purged_snaps scrub starts 2024-02-26T04:50:45.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:44 smithi156 ceph-mon[35805]: purged_snaps scrub ok 2024-02-26T04:50:45.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:44 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:45.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:44 smithi156 ceph-mon[35805]: osd.2 [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954] boot 2024-02-26T04:50:45.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:44 smithi156 ceph-mon[35805]: osdmap e18: 3 total, 3 up, 3 in 2024-02-26T04:50:45.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:44 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:45.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:44 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-26T04:50:45.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:44 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:45.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:44 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:45.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:44 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:50:45.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:44 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:45.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:44 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:45.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:44 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:50:45.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:44 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:45.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:44 smithi191 ceph-mon[36352]: purged_snaps scrub starts 2024-02-26T04:50:45.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:44 smithi191 ceph-mon[36352]: purged_snaps scrub ok 2024-02-26T04:50:45.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:44 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:45.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:44 smithi191 ceph-mon[36352]: osd.2 [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954] boot 2024-02-26T04:50:45.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:44 smithi191 ceph-mon[36352]: osdmap e18: 3 total, 3 up, 3 in 2024-02-26T04:50:45.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:44 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-26T04:50:45.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:44 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-26T04:50:45.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:44 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:45.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:44 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:45.264 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:44 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:50:45.264 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:44 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:45.264 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:44 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:45.264 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:44 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:50:45.264 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:44 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:46.477 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:46 smithi060 ceph-mon[32382]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:46.477 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:46 smithi060 ceph-mon[32382]: Detected new or changed devices on smithi191 2024-02-26T04:50:46.477 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:46 smithi060 ceph-mon[32382]: Adjusting osd_memory_target on smithi191 to 17080M 2024-02-26T04:50:46.478 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-26T04:50:46.478 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:46 smithi060 ceph-mon[32382]: osdmap e19: 3 total, 3 up, 3 in 2024-02-26T04:50:46.478 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T04:50:46.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:46 smithi191 ceph-mon[36352]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:46.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:46 smithi191 ceph-mon[36352]: Detected new or changed devices on smithi191 2024-02-26T04:50:46.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:46 smithi191 ceph-mon[36352]: Adjusting osd_memory_target on smithi191 to 17080M 2024-02-26T04:50:46.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-26T04:50:46.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:46 smithi191 ceph-mon[36352]: osdmap e19: 3 total, 3 up, 3 in 2024-02-26T04:50:46.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T04:50:46.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.581+0000 7f1597d2d700 1 -- 172.21.15.60:0/1103815238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1590100c10 msgr2=0x7f1590101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:50:46.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.581+0000 7f1597d2d700 1 --2- 172.21.15.60:0/1103815238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1590100c10 0x7f1590101030 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f158c0099c0 tx=0x7f158c009cd0 comp rx=0 tx=0).stop 2024-02-26T04:50:46.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.581+0000 7f1597d2d700 1 -- 172.21.15.60:0/1103815238 shutdown_connections 2024-02-26T04:50:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.581+0000 7f1597d2d700 1 --2- 172.21.15.60:0/1103815238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15901027b0 0x7f1590109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.581+0000 7f1597d2d700 1 --2- 172.21.15.60:0/1103815238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1590100c10 0x7f1590101030 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.581+0000 7f1597d2d700 1 --2- 172.21.15.60:0/1103815238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1590101e10 0x7f1590102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.581+0000 7f1597d2d700 1 -- 172.21.15.60:0/1103815238 >> 172.21.15.60:0/1103815238 conn(0x7f15900fc1d0 msgr2=0x7f15900fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:50:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.581+0000 7f1597d2d700 1 -- 172.21.15.60:0/1103815238 shutdown_connections 2024-02-26T04:50:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.581+0000 7f1597d2d700 1 -- 172.21.15.60:0/1103815238 wait complete. 2024-02-26T04:50:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f1597d2d700 1 Processor -- start 2024-02-26T04:50:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f1597d2d700 1 -- start start 2024-02-26T04:50:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f1597d2d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1590100c10 0x7f159006aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:50:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f1597d2d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1590101e10 0x7f159019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:50:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f1597d2d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15901027b0 0x7f159019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:50:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f1597d2d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159019f730 con 0x7f1590101e10 2024-02-26T04:50:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f1597d2d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159019f8a0 con 0x7f1590100c10 2024-02-26T04:50:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f1597d2d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159019fa60 con 0x7f15901027b0 2024-02-26T04:50:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f15952c8700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1590101e10 0x7f159019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:50:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f15952c8700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1590101e10 0x7f159019c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50554/0 (socket says 172.21.15.60:50554) 2024-02-26T04:50:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f15952c8700 1 -- 172.21.15.60:0/3792698045 learned_addr learned my addr 172.21.15.60:0/3792698045 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:50:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f1595ac9700 1 --2- 172.21.15.60:0/3792698045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1590100c10 0x7f159006aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:50:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f15962ca700 1 --2- 172.21.15.60:0/3792698045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15901027b0 0x7f159019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:50:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f15952c8700 1 -- 172.21.15.60:0/3792698045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15901027b0 msgr2=0x7f159019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:50:46.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f15952c8700 1 --2- 172.21.15.60:0/3792698045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15901027b0 0x7f159019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f15952c8700 1 -- 172.21.15.60:0/3792698045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1590100c10 msgr2=0x7f159006aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:50:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f15952c8700 1 --2- 172.21.15.60:0/3792698045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1590100c10 0x7f159006aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:50:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.582+0000 7f15952c8700 1 -- 172.21.15.60:0/3792698045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c0095f0 con 0x7f1590101e10 2024-02-26T04:50:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.583+0000 7f15952c8700 1 --2- 172.21.15.60:0/3792698045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1590101e10 0x7f159019c950 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f158000d950 tx=0x7f158000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:50:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.583+0000 7f1586ffd700 1 -- 172.21.15.60:0/3792698045 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580009940 con 0x7f1590101e10 2024-02-26T04:50:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.583+0000 7f1586ffd700 1 -- 172.21.15.60:0/3792698045 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1580010460 con 0x7f1590101e10 2024-02-26T04:50:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.583+0000 7f1597d2d700 1 -- 172.21.15.60:0/3792698045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159019fd40 con 0x7f1590101e10 2024-02-26T04:50:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.583+0000 7f1586ffd700 1 -- 172.21.15.60:0/3792698045 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158000f620 con 0x7f1590101e10 2024-02-26T04:50:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.583+0000 7f1597d2d700 1 -- 172.21.15.60:0/3792698045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15901a0230 con 0x7f1590101e10 2024-02-26T04:50:46.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.584+0000 7f1597d2d700 1 -- 172.21.15.60:0/3792698045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1590068930 con 0x7f1590101e10 2024-02-26T04:50:46.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.585+0000 7f1586ffd700 1 -- 172.21.15.60:0/3792698045 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7f15800105d0 con 0x7f1590101e10 2024-02-26T04:50:46.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.586+0000 7f1586ffd700 1 --2- 172.21.15.60:0/3792698045 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f157c0a21c0 0x7f157c0a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:50:46.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.586+0000 7f1595ac9700 1 --2- 172.21.15.60:0/3792698045 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f157c0a21c0 0x7f157c0a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:50:46.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.586+0000 7f1586ffd700 1 -- 172.21.15.60:0/3792698045 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f15800cca40 con 0x7f1590101e10 2024-02-26T04:50:46.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.586+0000 7f1595ac9700 1 --2- 172.21.15.60:0/3792698045 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f157c0a21c0 0x7f157c0a4680 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f158c0038c0 tx=0x7f158c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:50:46.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.589+0000 7f1586ffd700 1 -- 172.21.15.60:0/3792698045 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f158009a950 con 0x7f1590101e10 2024-02-26T04:50:46.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:46 smithi156 ceph-mon[35805]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-26T04:50:46.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:46 smithi156 ceph-mon[35805]: Detected new or changed devices on smithi191 2024-02-26T04:50:46.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:46 smithi156 ceph-mon[35805]: Adjusting osd_memory_target on smithi191 to 17080M 2024-02-26T04:50:46.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-26T04:50:46.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:46 smithi156 ceph-mon[35805]: osdmap e19: 3 total, 3 up, 3 in 2024-02-26T04:50:46.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T04:50:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:46.802+0000 7f1597d2d700 1 -- 172.21.15.60:0/3792698045 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f159002d910 con 0x7f157c0a21c0 2024-02-26T04:50:47.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:50:47.266+0000 7f1586ffd700 1 -- 172.21.15.60:0/3792698045 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f15800971f0 con 0x7f1590101e10 2024-02-26T04:50:47.479 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 04:50:47 smithi060 sudo[49023]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T04:50:47.480 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 04:50:47 smithi060 sudo[49023]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T04:50:47.480 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 04:50:47 smithi060 sudo[49023]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T04:50:47.480 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 04:50:47 smithi060 sudo[49023]: pam_unix(sudo:session): session closed for user root 2024-02-26T04:50:47.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:47 smithi060 ceph-mon[32382]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:47.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-26T04:50:47.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:47 smithi060 ceph-mon[32382]: osdmap e20: 3 total, 3 up, 3 in 2024-02-26T04:50:47.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:47 smithi060 ceph-mon[32382]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:50:47.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:50:47.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:50:47.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:47.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:47 smithi060 ceph-mon[32382]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T04:50:47.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:47 smithi191 ceph-mon[36352]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:47.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-26T04:50:47.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:47 smithi191 ceph-mon[36352]: osdmap e20: 3 total, 3 up, 3 in 2024-02-26T04:50:47.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:47 smithi191 ceph-mon[36352]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:50:47.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:50:47.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:50:47.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:47.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:47 smithi191 ceph-mon[36352]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T04:50:47.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:47 smithi156 ceph-mon[35805]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:47.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-26T04:50:47.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:47 smithi156 ceph-mon[35805]: osdmap e20: 3 total, 3 up, 3 in 2024-02-26T04:50:47.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:47 smithi156 ceph-mon[35805]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:50:47.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:50:47.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:50:47.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:47.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:47 smithi156 ceph-mon[35805]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T04:50:47.979 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 04:50:47 smithi060 sudo[49080]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T04:50:47.979 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 04:50:47 smithi060 sudo[49080]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T04:50:47.979 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 04:50:47 smithi060 sudo[49080]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T04:50:47.980 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 04:50:47 smithi060 sudo[49080]: pam_unix(sudo:session): session closed for user root 2024-02-26T04:50:48.000 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 04:50:47 smithi156 sudo[44153]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T04:50:48.000 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 04:50:47 smithi156 sudo[44153]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T04:50:48.001 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 04:50:47 smithi156 sudo[44153]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T04:50:48.001 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 04:50:47 smithi156 sudo[44153]: pam_unix(sudo:session): session closed for user root 2024-02-26T04:50:48.264 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 04:50:47 smithi156 sudo[44156]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T04:50:48.264 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 04:50:47 smithi156 sudo[44156]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T04:50:48.264 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 04:50:47 smithi156 sudo[44156]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T04:50:48.265 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 04:50:48 smithi156 sudo[44156]: pam_unix(sudo:session): session closed for user root 2024-02-26T04:50:48.519 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:48 smithi191 ceph-mon[36352]: osdmap e21: 3 total, 3 up, 3 in 2024-02-26T04:50:48.519 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:48 smithi191 ceph-mon[36352]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-26T04:50:48.519 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:48 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:48.519 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:48 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:48.519 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 04:50:48 smithi191 sudo[45160]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T04:50:48.519 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 04:50:48 smithi191 sudo[45160]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T04:50:48.519 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 04:50:48 smithi191 sudo[45160]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T04:50:48.520 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 04:50:48 smithi191 sudo[45160]: pam_unix(sudo:session): session closed for user root 2024-02-26T04:50:48.659 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:48 smithi060 ceph-mon[32382]: osdmap e21: 3 total, 3 up, 3 in 2024-02-26T04:50:48.660 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:48 smithi060 ceph-mon[32382]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-26T04:50:48.660 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:48 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:48.660 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:48 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:48.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:48 smithi156 ceph-mon[35805]: osdmap e21: 3 total, 3 up, 3 in 2024-02-26T04:50:48.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:48 smithi156 ceph-mon[35805]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-26T04:50:48.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:48 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:48.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:48 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:48.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:48 smithi060 sudo[49130]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T04:50:48.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:48 smithi060 sudo[49130]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T04:50:48.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:48 smithi060 sudo[49130]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T04:50:48.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:48 smithi060 sudo[49130]: pam_unix(sudo:session): session closed for user root 2024-02-26T04:50:49.009 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 04:50:48 smithi191 sudo[45163]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T04:50:49.009 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 04:50:48 smithi191 sudo[45163]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T04:50:49.009 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 04:50:48 smithi191 sudo[45163]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T04:50:49.009 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 04:50:48 smithi191 sudo[45163]: pam_unix(sudo:session): session closed for user root 2024-02-26T04:50:49.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 sudo[49138]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T04:50:49.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 sudo[49138]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T04:50:49.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 sudo[49138]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T04:50:49.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 sudo[49138]: pam_unix(sudo:session): session closed for user root 2024-02-26T04:50:49.572 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 sudo[45166]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T04:50:49.572 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 sudo[45166]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T04:50:49.572 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 sudo[45166]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T04:50:49.572 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 sudo[45166]: pam_unix(sudo:session): session closed for user root 2024-02-26T04:50:49.828 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 ceph-mon[35805]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:49.828 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:49.828 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 ceph-mon[35805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T04:50:49.828 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 ceph-mon[35805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T04:50:49.828 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 ceph-mon[35805]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T04:50:49.828 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 ceph-mon[35805]: Cluster is now healthy 2024-02-26T04:50:49.828 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:50:49.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:50:49.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:50:49.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:50:49.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:50:49.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:50:49.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 sudo[44159]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-26T04:50:49.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 sudo[44159]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T04:50:49.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 sudo[44159]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T04:50:49.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:49 smithi156 sudo[44159]: pam_unix(sudo:session): session closed for user root 2024-02-26T04:50:49.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 ceph-mon[32382]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:49.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:49.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 ceph-mon[32382]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T04:50:49.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 ceph-mon[32382]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T04:50:49.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 ceph-mon[32382]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T04:50:49.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 ceph-mon[32382]: Cluster is now healthy 2024-02-26T04:50:49.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:50:49.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:50:49.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:50:49.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:50:49.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:50:49.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:49 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:50:50.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 sudo[45169]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T04:50:50.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 sudo[45169]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T04:50:50.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 sudo[45169]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T04:50:50.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 sudo[45169]: pam_unix(sudo:session): session closed for user root 2024-02-26T04:50:50.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 ceph-mon[36352]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:50.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:50:50.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 ceph-mon[36352]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T04:50:50.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 ceph-mon[36352]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T04:50:50.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 ceph-mon[36352]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T04:50:50.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 ceph-mon[36352]: Cluster is now healthy 2024-02-26T04:50:50.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:50:50.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:50:50.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:50:50.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:50:50.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:50:50.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:49 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:50:50.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 sudo[44162]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-26T04:50:50.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 sudo[44162]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-26T04:50:50.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 sudo[44162]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-26T04:50:50.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 sudo[44162]: pam_unix(sudo:session): session closed for user root 2024-02-26T04:50:50.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 ceph-mon[35805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T04:50:50.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 ceph-mon[35805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T04:50:50.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 ceph-mon[35805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T04:50:50.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:50:50.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:50:50.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:50:50.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2446853654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e2d7dc0-1002-44e9-9295-2447bf76a03a"}]: dispatch 2024-02-26T04:50:50.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 ceph-mon[35805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T04:50:50.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2446853654' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e2d7dc0-1002-44e9-9295-2447bf76a03a"}]': finished 2024-02-26T04:50:50.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 ceph-mon[35805]: osdmap e22: 4 total, 3 up, 4 in 2024-02-26T04:50:50.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:50 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:50:50.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:50 smithi060 ceph-mon[32382]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T04:50:50.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:50 smithi060 ceph-mon[32382]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T04:50:50.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:50 smithi060 ceph-mon[32382]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T04:50:50.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:50:50.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:50:50.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:50:50.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:50 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2446853654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e2d7dc0-1002-44e9-9295-2447bf76a03a"}]: dispatch 2024-02-26T04:50:50.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:50 smithi060 ceph-mon[32382]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T04:50:50.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:50 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2446853654' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e2d7dc0-1002-44e9-9295-2447bf76a03a"}]': finished 2024-02-26T04:50:50.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:50 smithi060 ceph-mon[32382]: osdmap e22: 4 total, 3 up, 4 in 2024-02-26T04:50:50.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:50:51.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:50 smithi191 ceph-mon[36352]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T04:50:51.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:50 smithi191 ceph-mon[36352]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T04:50:51.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:50 smithi191 ceph-mon[36352]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-26T04:50:51.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:50 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-26T04:50:51.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:50 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-26T04:50:51.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:50 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-26T04:50:51.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:50 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2446853654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e2d7dc0-1002-44e9-9295-2447bf76a03a"}]: dispatch 2024-02-26T04:50:51.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:50 smithi191 ceph-mon[36352]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-26T04:50:51.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:50 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2446853654' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e2d7dc0-1002-44e9-9295-2447bf76a03a"}]': finished 2024-02-26T04:50:51.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:50 smithi191 ceph-mon[36352]: osdmap e22: 4 total, 3 up, 4 in 2024-02-26T04:50:51.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:50 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:50:51.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:51 smithi156 ceph-mon[35805]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:51.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:51 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3929576972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:50:51.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:51 smithi060 ceph-mon[32382]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:51.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:51 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3929576972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:50:52.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:51 smithi191 ceph-mon[36352]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:52.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:51 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3929576972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:50:53.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:53 smithi060 ceph-mon[32382]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:53.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:53 smithi156 ceph-mon[35805]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:54.011 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:53 smithi191 ceph-mon[36352]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:55.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:55 smithi060 ceph-mon[32382]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:55.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:55 smithi156 ceph-mon[35805]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:56.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:55 smithi191 ceph-mon[36352]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:57.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:57 smithi156 ceph-mon[35805]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:57.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:57 smithi060 ceph-mon[32382]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:58.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:57 smithi191 ceph-mon[36352]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:58.902 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-26T04:50:58.902 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:58.902 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:58 smithi060 ceph-mon[32382]: Deploying daemon osd.3 on smithi060 2024-02-26T04:50:58.909 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:58 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-26T04:50:58.910 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:58 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:58.910 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:58 smithi191 ceph-mon[36352]: Deploying daemon osd.3 on smithi060 2024-02-26T04:50:58.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:58 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-26T04:50:58.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:58 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:50:58.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:58 smithi156 ceph-mon[35805]: Deploying daemon osd.3 on smithi060 2024-02-26T04:50:59.913 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:50:59 smithi060 ceph-mon[32382]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:50:59.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:50:59 smithi156 ceph-mon[35805]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:00.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:50:59 smithi191 ceph-mon[36352]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:01.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:01 smithi060 ceph-mon[32382]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:01.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:01 smithi156 ceph-mon[35805]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:02.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:01 smithi191 ceph-mon[36352]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:03.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:03 smithi156 ceph-mon[35805]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:03.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:03 smithi060 ceph-mon[32382]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:04.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:03 smithi191 ceph-mon[36352]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:04.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:04 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:04.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:04 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:04.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:04 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:04.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:04 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:04.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:04 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:04.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:04 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:05.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:04 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:05.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:04 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:05.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:04 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:05.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:05 smithi156 ceph-mon[35805]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:05.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:05 smithi060 ceph-mon[32382]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:06.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:05 smithi191 ceph-mon[36352]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:07.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:51:07.307+0000 7f1586ffd700 1 -- 172.21.15.60:0/3792698045 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f159002d910 con 0x7f157c0a21c0 2024-02-26T04:51:07.311 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 3 on host 'smithi060' 2024-02-26T04:51:07.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:51:07.311+0000 7f1597d2d700 1 -- 172.21.15.60:0/3792698045 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f157c0a21c0 msgr2=0x7f157c0a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:51:07.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:51:07.311+0000 7f1597d2d700 1 --2- 172.21.15.60:0/3792698045 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f157c0a21c0 0x7f157c0a4680 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f158c0038c0 tx=0x7f158c00b540 comp rx=0 tx=0).stop 2024-02-26T04:51:07.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:51:07.311+0000 7f1597d2d700 1 -- 172.21.15.60:0/3792698045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1590101e10 msgr2=0x7f159019c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:51:07.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:51:07.311+0000 7f1597d2d700 1 --2- 172.21.15.60:0/3792698045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1590101e10 0x7f159019c950 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f158000d950 tx=0x7f158000dc60 comp rx=0 tx=0).stop 2024-02-26T04:51:07.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:51:07.311+0000 7f1597d2d700 1 -- 172.21.15.60:0/3792698045 shutdown_connections 2024-02-26T04:51:07.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:51:07.311+0000 7f1597d2d700 1 --2- 172.21.15.60:0/3792698045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15901027b0 0x7f159019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:07.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:51:07.311+0000 7f1597d2d700 1 --2- 172.21.15.60:0/3792698045 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f157c0a21c0 0x7f157c0a4680 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:07.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:51:07.311+0000 7f1597d2d700 1 --2- 172.21.15.60:0/3792698045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1590100c10 0x7f159006aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:07.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:51:07.311+0000 7f1597d2d700 1 --2- 172.21.15.60:0/3792698045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1590101e10 0x7f159019c950 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:07.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:51:07.311+0000 7f1597d2d700 1 -- 172.21.15.60:0/3792698045 >> 172.21.15.60:0/3792698045 conn(0x7f15900fc1d0 msgr2=0x7f15901051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:51:07.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:51:07.311+0000 7f1597d2d700 1 -- 172.21.15.60:0/3792698045 shutdown_connections 2024-02-26T04:51:07.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:51:07.311+0000 7f1597d2d700 1 -- 172.21.15.60:0/3792698045 wait complete. 2024-02-26T04:51:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:07 smithi060 ceph-mon[32382]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:51:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:07.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:07 smithi156 ceph-mon[35805]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:07.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:07.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:51:07.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:07 smithi191 ceph-mon[36352]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:51:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:08.116 DEBUG:teuthology.orchestra.run.smithi060:osd.3> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.3.service 2024-02-26T04:51:08.118 INFO:tasks.cephadm:Deploying osd.4 on smithi156 with /dev/vg_nvme/lv_3... 2024-02-26T04:51:08.118 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-26T04:51:08.366 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T04:51:08.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:08 smithi156 ceph-mon[35805]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T04:51:08.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:08 smithi060 ceph-mon[32382]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T04:51:08.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:08 smithi191 ceph-mon[36352]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-26T04:51:09.540 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:09 smithi060 ceph-mon[32382]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:09.540 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:09 smithi060 ceph-mon[32382]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-26T04:51:09.541 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:09 smithi060 ceph-mon[32382]: osdmap e23: 4 total, 3 up, 4 in 2024-02-26T04:51:09.541 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:09 smithi060 ceph-mon[32382]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:51:09.541 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:09 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:51:09.684 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:09 smithi156 ceph-mon[35805]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:09.684 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:09 smithi156 ceph-mon[35805]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-26T04:51:09.684 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:09 smithi156 ceph-mon[35805]: osdmap e23: 4 total, 3 up, 4 in 2024-02-26T04:51:09.684 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:09 smithi156 ceph-mon[35805]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:51:09.684 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:09 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:51:09.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:09 smithi191 ceph-mon[36352]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:09.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:09 smithi191 ceph-mon[36352]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-26T04:51:09.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:09 smithi191 ceph-mon[36352]: osdmap e23: 4 total, 3 up, 4 in 2024-02-26T04:51:09.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:09 smithi191 ceph-mon[36352]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:51:09.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:09 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:51:10.230 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 04:51:09 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3[51193]: 2024-02-26T04:51:09.926+0000 7f4fca517700 -1 osd.3 0 waiting for initial osdmap 2024-02-26T04:51:10.230 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 04:51:09 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3[51193]: 2024-02-26T04:51:09.934+0000 7f4fc1b01700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T04:51:10.359 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-26T04:51:10.359 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-26T04:51:10.359 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-02-26T04:51:10.359 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-02-26T04:51:10.359 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00917641 s, 1.1 GB/s 2024-02-26T04:51:10.359 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-02-26T04:51:10.663 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:10 smithi156 ceph-mon[35805]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T04:51:10.663 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:10 smithi156 ceph-mon[35805]: osdmap e24: 4 total, 3 up, 4 in 2024-02-26T04:51:10.664 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:10 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:51:10.664 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:10 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:51:10.664 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:10 smithi156 ceph-mon[35805]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' 2024-02-26T04:51:10.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:10 smithi060 ceph-mon[32382]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T04:51:10.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:10 smithi060 ceph-mon[32382]: osdmap e24: 4 total, 3 up, 4 in 2024-02-26T04:51:10.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:10 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:51:10.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:10 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:51:10.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:10 smithi060 ceph-mon[32382]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' 2024-02-26T04:51:10.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:10 smithi191 ceph-mon[36352]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T04:51:10.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:10 smithi191 ceph-mon[36352]: osdmap e24: 4 total, 3 up, 4 in 2024-02-26T04:51:10.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:10 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:51:10.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:10 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:51:10.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:10 smithi191 ceph-mon[36352]: from='osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146]' entity='osd.3' 2024-02-26T04:51:11.023 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_3 2024-02-26T04:51:11.268 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T04:51:11.643 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: purged_snaps scrub starts 2024-02-26T04:51:11.643 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: purged_snaps scrub ok 2024-02-26T04:51:11.643 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:11.643 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146] boot 2024-02-26T04:51:11.643 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: osdmap e25: 4 total, 4 up, 4 in 2024-02-26T04:51:11.643 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:51:11.643 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: Detected new or changed devices on smithi060 2024-02-26T04:51:11.643 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:11.643 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:11.643 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:51:11.644 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:51:11.645 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: Adjusting osd_memory_target on smithi060 to 8546M 2024-02-26T04:51:11.645 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:11.645 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:11.645 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:51:11.645 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:11.645 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:11 smithi156 ceph-mon[35805]: osdmap e26: 4 total, 4 up, 4 in 2024-02-26T04:51:11.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: purged_snaps scrub starts 2024-02-26T04:51:11.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: purged_snaps scrub ok 2024-02-26T04:51:11.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:11.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146] boot 2024-02-26T04:51:11.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: osdmap e25: 4 total, 4 up, 4 in 2024-02-26T04:51:11.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:51:11.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: Detected new or changed devices on smithi060 2024-02-26T04:51:11.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:11.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:11.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:51:11.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:51:11.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: Adjusting osd_memory_target on smithi060 to 8546M 2024-02-26T04:51:11.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:11.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:11.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:51:11.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:11.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:11 smithi060 ceph-mon[32382]: osdmap e26: 4 total, 4 up, 4 in 2024-02-26T04:51:11.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: purged_snaps scrub starts 2024-02-26T04:51:11.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: purged_snaps scrub ok 2024-02-26T04:51:11.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-26T04:51:11.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: osd.3 [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146] boot 2024-02-26T04:51:11.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: osdmap e25: 4 total, 4 up, 4 in 2024-02-26T04:51:11.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-26T04:51:11.761 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: Detected new or changed devices on smithi060 2024-02-26T04:51:11.761 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:11.761 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:11.761 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:51:11.761 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:51:11.761 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: Adjusting osd_memory_target on smithi060 to 8546M 2024-02-26T04:51:11.761 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:11.761 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:11.761 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:51:11.761 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:11.762 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:11 smithi191 ceph-mon[36352]: osdmap e26: 4 total, 4 up, 4 in 2024-02-26T04:51:12.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.855+0000 7f3abf740700 1 -- 172.21.15.156:0/1141177219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab8102ea0 msgr2=0x7f3ab8103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:51:12.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.855+0000 7f3abf740700 1 --2- 172.21.15.156:0/1141177219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab8102ea0 0x7f3ab8103300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80099c0 tx=0x7f3aa8009cd0 comp rx=0 tx=0).stop 2024-02-26T04:51:12.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.855+0000 7f3abf740700 1 -- 172.21.15.156:0/1141177219 shutdown_connections 2024-02-26T04:51:12.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.855+0000 7f3abf740700 1 --2- 172.21.15.156:0/1141177219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab8103840 0x7f3ab810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:12.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.855+0000 7f3abf740700 1 --2- 172.21.15.156:0/1141177219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ab8101ca0 0x7f3ab81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:12.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.855+0000 7f3abf740700 1 --2- 172.21.15.156:0/1141177219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab8102ea0 0x7f3ab8103300 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:12.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.855+0000 7f3abf740700 1 -- 172.21.15.156:0/1141177219 >> 172.21.15.156:0/1141177219 conn(0x7f3ab80fd280 msgr2=0x7f3ab80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:51:12.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.855+0000 7f3abf740700 1 -- 172.21.15.156:0/1141177219 shutdown_connections 2024-02-26T04:51:12.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.855+0000 7f3abf740700 1 -- 172.21.15.156:0/1141177219 wait complete. 2024-02-26T04:51:12.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abf740700 1 Processor -- start 2024-02-26T04:51:12.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abf740700 1 -- start start 2024-02-26T04:51:12.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abf740700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab8101ca0 0x7f3ab8197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:51:12.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abf740700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ab8102ea0 0x7f3ab81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:51:12.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abf740700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab8103840 0x7f3ab819f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:51:12.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abf740700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab819f7c0 con 0x7f3ab8101ca0 2024-02-26T04:51:12.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abf740700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab819f900 con 0x7f3ab8103840 2024-02-26T04:51:12.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abf740700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab819fac0 con 0x7f3ab8102ea0 2024-02-26T04:51:12.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abdcdd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab8103840 0x7f3ab819f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:51:12.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abdcdd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab8103840 0x7f3ab819f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48664/0 (socket says 172.21.15.156:48664) 2024-02-26T04:51:12.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abdcdd700 1 -- 172.21.15.156:0/3100277261 learned_addr learned my addr 172.21.15.156:0/3100277261 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-02-26T04:51:12.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abccdb700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ab8102ea0 0x7f3ab81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:51:12.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abd4dc700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab8101ca0 0x7f3ab8197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:51:12.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abdcdd700 1 -- 172.21.15.156:0/3100277261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ab8102ea0 msgr2=0x7f3ab81981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:51:12.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abdcdd700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ab8102ea0 0x7f3ab81981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:12.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abdcdd700 1 -- 172.21.15.156:0/3100277261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab8101ca0 msgr2=0x7f3ab8197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:51:12.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abdcdd700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab8101ca0 0x7f3ab8197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:12.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abdcdd700 1 -- 172.21.15.156:0/3100277261 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa80095f0 con 0x7f3ab8103840 2024-02-26T04:51:12.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abd4dc700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab8101ca0 0x7f3ab8197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:51:12.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3abdcdd700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab8103840 0x7f3ab819f1f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0007d70 tx=0x7f3ab000a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:51:12.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.856+0000 7f3aae7fc700 1 -- 172.21.15.156:0/3100277261 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab000ad40 con 0x7f3ab8103840 2024-02-26T04:51:12.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.857+0000 7f3abf740700 1 -- 172.21.15.156:0/3100277261 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab819fda0 con 0x7f3ab8103840 2024-02-26T04:51:12.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.857+0000 7f3aae7fc700 1 -- 172.21.15.156:0/3100277261 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ab0012900 con 0x7f3ab8103840 2024-02-26T04:51:12.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.857+0000 7f3aae7fc700 1 -- 172.21.15.156:0/3100277261 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0011780 con 0x7f3ab8103840 2024-02-26T04:51:12.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.857+0000 7f3abf740700 1 -- 172.21.15.156:0/3100277261 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab81a01e0 con 0x7f3ab8103840 2024-02-26T04:51:12.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.857+0000 7f3abf740700 1 -- 172.21.15.156:0/3100277261 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab81090e0 con 0x7f3ab8103840 2024-02-26T04:51:12.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.859+0000 7f3aae7fc700 1 -- 172.21.15.156:0/3100277261 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3ab0012a70 con 0x7f3ab8103840 2024-02-26T04:51:12.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.860+0000 7f3aae7fc700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3aa40a2150 0x7f3aa40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:51:12.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.860+0000 7f3aae7fc700 1 -- 172.21.15.156:0/3100277261 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f3ab00cd170 con 0x7f3ab8103840 2024-02-26T04:51:12.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.860+0000 7f3abd4dc700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3aa40a2150 0x7f3aa40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:51:12.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.861+0000 7f3abd4dc700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3aa40a2150 0x7f3aa40a4610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8199150 tx=0x7f3ab4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:51:12.866 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:12.864+0000 7f3aae7fc700 1 -- 172.21.15.156:0/3100277261 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3ab0097340 con 0x7f3ab8103840 2024-02-26T04:51:13.066 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:13.063+0000 7f3abf740700 1 -- 172.21.15.156:0/3100277261 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ab802d990 con 0x7f3aa40a2150 2024-02-26T04:51:13.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:13 smithi060 ceph-mon[32382]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:13.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:13 smithi060 ceph-mon[32382]: osdmap e27: 4 total, 4 up, 4 in 2024-02-26T04:51:13.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:13 smithi060 ceph-mon[32382]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:51:13.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:13 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:51:13.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:13 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:51:13.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:13 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:13.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:13 smithi191 ceph-mon[36352]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:13.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:13 smithi191 ceph-mon[36352]: osdmap e27: 4 total, 4 up, 4 in 2024-02-26T04:51:13.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:13 smithi191 ceph-mon[36352]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:51:13.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:13 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:51:13.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:13 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:51:13.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:13 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:13.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:13 smithi156 ceph-mon[35805]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:13.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:13 smithi156 ceph-mon[35805]: osdmap e27: 4 total, 4 up, 4 in 2024-02-26T04:51:13.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:13 smithi156 ceph-mon[35805]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:51:13.760 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:13 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:51:13.761 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:13 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:51:13.761 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:13 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:14.659 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:14 smithi156 ceph-mon[35805]: osdmap e28: 4 total, 4 up, 4 in 2024-02-26T04:51:14.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:14 smithi060 ceph-mon[32382]: osdmap e28: 4 total, 4 up, 4 in 2024-02-26T04:51:14.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:14 smithi191 ceph-mon[36352]: osdmap e28: 4 total, 4 up, 4 in 2024-02-26T04:51:15.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:15 smithi060 ceph-mon[32382]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:15.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:15 smithi060 ceph-mon[32382]: osdmap e29: 4 total, 4 up, 4 in 2024-02-26T04:51:15.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:15 smithi191 ceph-mon[36352]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:15.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:15 smithi191 ceph-mon[36352]: osdmap e29: 4 total, 4 up, 4 in 2024-02-26T04:51:15.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:15 smithi156 ceph-mon[35805]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:15.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:15 smithi156 ceph-mon[35805]: osdmap e29: 4 total, 4 up, 4 in 2024-02-26T04:51:16.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:16 smithi060 ceph-mon[32382]: from='client.? 172.21.15.156:0/3191433342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d040e4b1-df8e-4f0c-9f84-767081c41939"}]: dispatch 2024-02-26T04:51:16.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:16 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d040e4b1-df8e-4f0c-9f84-767081c41939"}]: dispatch 2024-02-26T04:51:16.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:16 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d040e4b1-df8e-4f0c-9f84-767081c41939"}]': finished 2024-02-26T04:51:16.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:16 smithi060 ceph-mon[32382]: osdmap e30: 5 total, 4 up, 5 in 2024-02-26T04:51:16.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:16.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:16 smithi060 ceph-mon[32382]: from='client.? 172.21.15.156:0/2558230607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:51:16.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:16 smithi191 ceph-mon[36352]: from='client.? 172.21.15.156:0/3191433342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d040e4b1-df8e-4f0c-9f84-767081c41939"}]: dispatch 2024-02-26T04:51:16.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:16 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d040e4b1-df8e-4f0c-9f84-767081c41939"}]: dispatch 2024-02-26T04:51:16.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:16 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d040e4b1-df8e-4f0c-9f84-767081c41939"}]': finished 2024-02-26T04:51:16.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:16 smithi191 ceph-mon[36352]: osdmap e30: 5 total, 4 up, 5 in 2024-02-26T04:51:16.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:16.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:16 smithi191 ceph-mon[36352]: from='client.? 172.21.15.156:0/2558230607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:51:16.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:16 smithi156 ceph-mon[35805]: from='client.? 172.21.15.156:0/3191433342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d040e4b1-df8e-4f0c-9f84-767081c41939"}]: dispatch 2024-02-26T04:51:16.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:16 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d040e4b1-df8e-4f0c-9f84-767081c41939"}]: dispatch 2024-02-26T04:51:16.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:16 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d040e4b1-df8e-4f0c-9f84-767081c41939"}]': finished 2024-02-26T04:51:16.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:16 smithi156 ceph-mon[35805]: osdmap e30: 5 total, 4 up, 5 in 2024-02-26T04:51:16.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:16 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:16.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:16 smithi156 ceph-mon[35805]: from='client.? 172.21.15.156:0/2558230607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:51:18.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:17 smithi156 ceph-mon[35805]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T04:51:18.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:17 smithi060 ceph-mon[32382]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T04:51:18.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:17 smithi191 ceph-mon[36352]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T04:51:20.044 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:19 smithi156 ceph-mon[35805]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-02-26T04:51:20.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:19 smithi060 ceph-mon[32382]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-02-26T04:51:20.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:19 smithi191 ceph-mon[36352]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-02-26T04:51:21.987 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:21 smithi156 ceph-mon[35805]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-02-26T04:51:22.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:21 smithi060 ceph-mon[32382]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-02-26T04:51:22.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:21 smithi191 ceph-mon[36352]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-02-26T04:51:24.076 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:23 smithi156 ceph-mon[35805]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T04:51:24.076 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:23 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-26T04:51:24.076 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:23 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:24.076 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:23 smithi156 ceph-mon[35805]: Deploying daemon osd.4 on smithi156 2024-02-26T04:51:24.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:23 smithi060 ceph-mon[32382]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T04:51:24.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:23 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-26T04:51:24.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:23 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:24.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:23 smithi060 ceph-mon[32382]: Deploying daemon osd.4 on smithi156 2024-02-26T04:51:24.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:23 smithi191 ceph-mon[36352]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T04:51:24.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:23 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-26T04:51:24.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:23 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:24.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:23 smithi191 ceph-mon[36352]: Deploying daemon osd.4 on smithi156 2024-02-26T04:51:26.127 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:25 smithi156 ceph-mon[35805]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-26T04:51:26.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:25 smithi060 ceph-mon[32382]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-26T04:51:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:25 smithi191 ceph-mon[36352]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-26T04:51:28.077 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:27 smithi156 ceph-mon[35805]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:28.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:27 smithi060 ceph-mon[32382]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:28.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:27 smithi191 ceph-mon[36352]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:29.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:29 smithi156 ceph-mon[35805]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:29.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:29 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:29.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:29 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:29.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:29 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:29.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:29 smithi060 ceph-mon[32382]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:29.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:29 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:29.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:29 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:29.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:29 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:30.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:29 smithi191 ceph-mon[36352]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:30.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:29 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:30.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:29 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:30.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:29 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:31.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:31 smithi060 ceph-mon[32382]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:31.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:31.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:31.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:31.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:51:31.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:32.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:31 smithi191 ceph-mon[36352]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:32.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:31 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:32.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:31 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:32.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:31 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:32.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:31 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:51:32.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:31 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:32.100 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:31 smithi156 ceph-mon[35805]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:32.100 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:31 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:32.100 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:31 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:32.100 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:31 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:32.100 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:31 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:51:32.100 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:31 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:32.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:32.481+0000 7f3aae7fc700 1 -- 172.21.15.156:0/3100277261 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3ab802d990 con 0x7f3aa40a2150 2024-02-26T04:51:32.484 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 4 on host 'smithi156' 2024-02-26T04:51:32.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:32.484+0000 7f3abf740700 1 -- 172.21.15.156:0/3100277261 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3aa40a2150 msgr2=0x7f3aa40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:51:32.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:32.484+0000 7f3abf740700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3aa40a2150 0x7f3aa40a4610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8199150 tx=0x7f3ab4008040 comp rx=0 tx=0).stop 2024-02-26T04:51:32.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:32.484+0000 7f3abf740700 1 -- 172.21.15.156:0/3100277261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab8103840 msgr2=0x7f3ab819f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:51:32.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:32.484+0000 7f3abf740700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab8103840 0x7f3ab819f1f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0007d70 tx=0x7f3ab000a340 comp rx=0 tx=0).stop 2024-02-26T04:51:32.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:32.484+0000 7f3abf740700 1 -- 172.21.15.156:0/3100277261 shutdown_connections 2024-02-26T04:51:32.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:32.484+0000 7f3abf740700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3aa40a2150 0x7f3aa40a4610 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:32.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:32.484+0000 7f3abf740700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab8103840 0x7f3ab819f1f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:32.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:32.484+0000 7f3abf740700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab8101ca0 0x7f3ab8197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:32.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:32.484+0000 7f3abf740700 1 --2- 172.21.15.156:0/3100277261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ab8102ea0 0x7f3ab81981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:32.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:32.484+0000 7f3abf740700 1 -- 172.21.15.156:0/3100277261 >> 172.21.15.156:0/3100277261 conn(0x7f3ab80fd280 msgr2=0x7f3ab8106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:51:32.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:32.484+0000 7f3abf740700 1 -- 172.21.15.156:0/3100277261 shutdown_connections 2024-02-26T04:51:32.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:51:32.484+0000 7f3abf740700 1 -- 172.21.15.156:0/3100277261 wait complete. 2024-02-26T04:51:33.122 DEBUG:teuthology.orchestra.run.smithi156:osd.4> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.4.service 2024-02-26T04:51:33.125 INFO:tasks.cephadm:Deploying osd.5 on smithi191 with /dev/vg_nvme/lv_3... 2024-02-26T04:51:33.125 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-26T04:51:33.369 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T04:51:33.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:33 smithi060 ceph-mon[32382]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:33.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:33.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:33.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:33.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:33 smithi060 ceph-mon[32382]: from='osd.4 [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T04:51:33.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:33 smithi060 ceph-mon[32382]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T04:51:33.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:33 smithi191 ceph-mon[36352]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:33.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:33.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:33.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:33.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:33 smithi191 ceph-mon[36352]: from='osd.4 [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T04:51:33.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:33 smithi191 ceph-mon[36352]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T04:51:33.902 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:33 smithi156 ceph-mon[35805]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:33.902 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:33.902 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:33.902 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:33.902 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:33 smithi156 ceph-mon[35805]: from='osd.4 [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T04:51:33.903 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:33 smithi156 ceph-mon[35805]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-26T04:51:34.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:34 smithi060 ceph-mon[32382]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-26T04:51:34.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:34 smithi060 ceph-mon[32382]: from='osd.4 [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:51:34.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:34 smithi060 ceph-mon[32382]: osdmap e31: 5 total, 4 up, 5 in 2024-02-26T04:51:34.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:34 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:34.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:34 smithi060 ceph-mon[32382]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:51:34.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:34 smithi191 ceph-mon[36352]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-26T04:51:34.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:34 smithi191 ceph-mon[36352]: from='osd.4 [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:51:34.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:34 smithi191 ceph-mon[36352]: osdmap e31: 5 total, 4 up, 5 in 2024-02-26T04:51:34.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:34 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:34.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:34 smithi191 ceph-mon[36352]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:51:34.765 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:34 smithi156 ceph-mon[35805]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-26T04:51:34.765 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:34 smithi156 ceph-mon[35805]: from='osd.4 [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:51:34.765 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:34 smithi156 ceph-mon[35805]: osdmap e31: 5 total, 4 up, 5 in 2024-02-26T04:51:34.765 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:34 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:34.765 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:34 smithi156 ceph-mon[35805]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:51:34.765 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 04:51:34 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4[46970]: 2024-02-26T04:51:34.470+0000 7f8d7fd6b700 -1 osd.4 0 waiting for initial osdmap 2024-02-26T04:51:34.765 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 04:51:34 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4[46970]: 2024-02-26T04:51:34.475+0000 7f8d7bb5e700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T04:51:35.379 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-26T04:51:35.380 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-26T04:51:35.380 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-02-26T04:51:35.380 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-02-26T04:51:35.380 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134552 s, 779 MB/s 2024-02-26T04:51:35.380 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-02-26T04:51:35.531 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:35 smithi191 ceph-mon[36352]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:35.531 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:35 smithi191 ceph-mon[36352]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-26T04:51:35.531 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:35 smithi191 ceph-mon[36352]: osdmap e32: 5 total, 4 up, 5 in 2024-02-26T04:51:35.531 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:35 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:35.532 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:35 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:35.660 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:35 smithi156 ceph-mon[35805]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:35.660 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:35 smithi156 ceph-mon[35805]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-26T04:51:35.660 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:35 smithi156 ceph-mon[35805]: osdmap e32: 5 total, 4 up, 5 in 2024-02-26T04:51:35.660 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:35 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:35.660 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:35 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:35.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:35 smithi060 ceph-mon[32382]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-02-26T04:51:35.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:35 smithi060 ceph-mon[32382]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-26T04:51:35.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:35 smithi060 ceph-mon[32382]: osdmap e32: 5 total, 4 up, 5 in 2024-02-26T04:51:35.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:35 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:35.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:35 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:36.009 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_3 2024-02-26T04:51:36.255 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T04:51:36.636 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: purged_snaps scrub starts 2024-02-26T04:51:36.637 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: purged_snaps scrub ok 2024-02-26T04:51:36.637 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: osd.4 [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576] boot 2024-02-26T04:51:36.637 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: osdmap e33: 5 total, 5 up, 5 in 2024-02-26T04:51:36.637 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:36.637 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: Detected new or changed devices on smithi156 2024-02-26T04:51:36.637 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:36.637 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:36.637 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:51:36.637 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:51:36.637 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: Adjusting osd_memory_target on smithi156 to 8540M 2024-02-26T04:51:36.638 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:36.638 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:36.638 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:51:36.638 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:36 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:36.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: purged_snaps scrub starts 2024-02-26T04:51:36.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: purged_snaps scrub ok 2024-02-26T04:51:36.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: osd.4 [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576] boot 2024-02-26T04:51:36.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: osdmap e33: 5 total, 5 up, 5 in 2024-02-26T04:51:36.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:36.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: Detected new or changed devices on smithi156 2024-02-26T04:51:36.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:36.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:36.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:51:36.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:51:36.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: Adjusting osd_memory_target on smithi156 to 8540M 2024-02-26T04:51:36.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:36.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:36.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:51:36.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:36 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:36.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: purged_snaps scrub starts 2024-02-26T04:51:36.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: purged_snaps scrub ok 2024-02-26T04:51:36.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: osd.4 [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576] boot 2024-02-26T04:51:36.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: osdmap e33: 5 total, 5 up, 5 in 2024-02-26T04:51:36.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-26T04:51:36.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: Detected new or changed devices on smithi156 2024-02-26T04:51:36.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:36.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:36.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:51:36.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:51:36.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: Adjusting osd_memory_target on smithi156 to 8540M 2024-02-26T04:51:36.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:36.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:36.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:51:36.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:36 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:37.661 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:37 smithi191 ceph-mon[36352]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:37.661 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:37 smithi191 ceph-mon[36352]: osdmap e34: 5 total, 5 up, 5 in 2024-02-26T04:51:37.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.860+0000 7f81e4996700 1 -- 172.21.15.191:0/2453288148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0101520 msgr2=0x7f81e0101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:51:37.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.860+0000 7f81e4996700 1 --2- 172.21.15.191:0/2453288148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0101520 0x7f81e0101980 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009990 tx=0x7f81c8009ca0 comp rx=0 tx=0).stop 2024-02-26T04:51:37.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.860+0000 7f81e4996700 1 -- 172.21.15.191:0/2453288148 shutdown_connections 2024-02-26T04:51:37.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.860+0000 7f81e4996700 1 --2- 172.21.15.191:0/2453288148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e0101ec0 0x7f81e0108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:37.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.860+0000 7f81e4996700 1 --2- 172.21.15.191:0/2453288148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e0100320 0x7f81e0100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:37.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.860+0000 7f81e4996700 1 --2- 172.21.15.191:0/2453288148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0101520 0x7f81e0101980 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:37.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.860+0000 7f81e4996700 1 -- 172.21.15.191:0/2453288148 >> 172.21.15.191:0/2453288148 conn(0x7f81e00fb8e0 msgr2=0x7f81e00fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:51:37.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.860+0000 7f81e4996700 1 -- 172.21.15.191:0/2453288148 shutdown_connections 2024-02-26T04:51:37.864 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.860+0000 7f81e4996700 1 -- 172.21.15.191:0/2453288148 wait complete. 2024-02-26T04:51:37.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.861+0000 7f81e4996700 1 Processor -- start 2024-02-26T04:51:37.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.861+0000 7f81e4996700 1 -- start start 2024-02-26T04:51:37.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.861+0000 7f81e4996700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e0100320 0x7f81e0197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:51:37.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.861+0000 7f81e4996700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e0101520 0x7f81e0198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:51:37.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.861+0000 7f81e4996700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0101ec0 0x7f81e019f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:51:37.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.861+0000 7f81e4996700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e019f790 con 0x7f81e0101520 2024-02-26T04:51:37.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.861+0000 7f81e4996700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e019f8d0 con 0x7f81e0100320 2024-02-26T04:51:37.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.861+0000 7f81e4996700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e019fa90 con 0x7f81e0101ec0 2024-02-26T04:51:37.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.861+0000 7f81ded9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0101ec0 0x7f81e019f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:51:37.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.861+0000 7f81de59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e0100320 0x7f81e0197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:51:37.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.861+0000 7f81ded9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0101ec0 0x7f81e019f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:55098/0 (socket says 172.21.15.191:55098) 2024-02-26T04:51:37.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.861+0000 7f81ded9d700 1 -- 172.21.15.191:0/699086092 learned_addr learned my addr 172.21.15.191:0/699086092 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-26T04:51:37.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.861+0000 7f81ddd9b700 1 --2- 172.21.15.191:0/699086092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e0101520 0x7f81e0198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:51:37.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.862+0000 7f81ded9d700 1 -- 172.21.15.191:0/699086092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e0100320 msgr2=0x7f81e0197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:51:37.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.862+0000 7f81ded9d700 1 --2- 172.21.15.191:0/699086092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e0100320 0x7f81e0197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:37.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.862+0000 7f81ded9d700 1 -- 172.21.15.191:0/699086092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e0101520 msgr2=0x7f81e0198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:51:37.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.862+0000 7f81ded9d700 1 --2- 172.21.15.191:0/699086092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e0101520 0x7f81e0198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:37.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.862+0000 7f81ded9d700 1 -- 172.21.15.191:0/699086092 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c80095f0 con 0x7f81e0101ec0 2024-02-26T04:51:37.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.862+0000 7f81ded9d700 1 --2- 172.21.15.191:0/699086092 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0101ec0 0x7f81e019f1c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f81d800d980 tx=0x7f81d800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:51:37.867 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.862+0000 7f81d77fe700 1 -- 172.21.15.191:0/699086092 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8004a30 con 0x7f81e0101ec0 2024-02-26T04:51:37.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.862+0000 7f81d77fe700 1 -- 172.21.15.191:0/699086092 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81d8005500 con 0x7f81e0101ec0 2024-02-26T04:51:37.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.862+0000 7f81e4996700 1 -- 172.21.15.191:0/699086092 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e0109f60 con 0x7f81e0101ec0 2024-02-26T04:51:37.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.862+0000 7f81d77fe700 1 -- 172.21.15.191:0/699086092 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d800fc80 con 0x7f81e0101ec0 2024-02-26T04:51:37.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.862+0000 7f81e4996700 1 -- 172.21.15.191:0/699086092 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e010a410 con 0x7f81e0101ec0 2024-02-26T04:51:37.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.863+0000 7f81e4996700 1 -- 172.21.15.191:0/699086092 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e01076e0 con 0x7f81e0101ec0 2024-02-26T04:51:37.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.864+0000 7f81d77fe700 1 -- 172.21.15.191:0/699086092 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f81d8005000 con 0x7f81e0101ec0 2024-02-26T04:51:37.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.865+0000 7f81d77fe700 1 --2- 172.21.15.191:0/699086092 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f81cc0a2150 0x7f81cc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:51:37.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.865+0000 7f81d77fe700 1 -- 172.21.15.191:0/699086092 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f81d80cce60 con 0x7f81e0101ec0 2024-02-26T04:51:37.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.865+0000 7f81de59c700 1 --2- 172.21.15.191:0/699086092 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f81cc0a2150 0x7f81cc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:51:37.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.866+0000 7f81de59c700 1 --2- 172.21.15.191:0/699086092 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f81cc0a2150 0x7f81cc0a4610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009510 tx=0x7f81d0009230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:51:37.871 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:37.869+0000 7f81d77fe700 1 -- 172.21.15.191:0/699086092 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f81d8096ea0 con 0x7f81e0101ec0 2024-02-26T04:51:37.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:37 smithi156 ceph-mon[35805]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:37.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:37 smithi156 ceph-mon[35805]: osdmap e34: 5 total, 5 up, 5 in 2024-02-26T04:51:37.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:37 smithi060 ceph-mon[32382]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:37.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:37 smithi060 ceph-mon[32382]: osdmap e34: 5 total, 5 up, 5 in 2024-02-26T04:51:38.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:38.069+0000 7f81e4996700 1 -- 172.21.15.191:0/699086092 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f81e002d870 con 0x7f81cc0a2150 2024-02-26T04:51:38.736 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:38 smithi191 ceph-mon[36352]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:51:38.736 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:51:38.736 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:51:38.737 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:38.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:38 smithi060 ceph-mon[32382]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:51:38.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:51:38.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:51:38.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:38.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:38 smithi156 ceph-mon[35805]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:51:38.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:51:38.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:51:38.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:39.739 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:39 smithi191 ceph-mon[36352]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:39.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:39 smithi156 ceph-mon[35805]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:39.902 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:39 smithi060 ceph-mon[32382]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:41.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:41 smithi191 ceph-mon[36352]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:41.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:41 smithi191 ceph-mon[36352]: from='client.? 172.21.15.191:0/995276504' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7807402-620a-40f2-85b4-71f762a1a73b"}]: dispatch 2024-02-26T04:51:41.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:41 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7807402-620a-40f2-85b4-71f762a1a73b"}]: dispatch 2024-02-26T04:51:41.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:41 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7807402-620a-40f2-85b4-71f762a1a73b"}]': finished 2024-02-26T04:51:41.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:41 smithi191 ceph-mon[36352]: osdmap e35: 6 total, 5 up, 6 in 2024-02-26T04:51:41.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:41 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:51:41.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:41 smithi191 ceph-mon[36352]: from='client.? 172.21.15.191:0/1418591072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:51:41.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:41 smithi156 ceph-mon[35805]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:41.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:41 smithi156 ceph-mon[35805]: from='client.? 172.21.15.191:0/995276504' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7807402-620a-40f2-85b4-71f762a1a73b"}]: dispatch 2024-02-26T04:51:41.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:41 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7807402-620a-40f2-85b4-71f762a1a73b"}]: dispatch 2024-02-26T04:51:41.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:41 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7807402-620a-40f2-85b4-71f762a1a73b"}]': finished 2024-02-26T04:51:41.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:41 smithi156 ceph-mon[35805]: osdmap e35: 6 total, 5 up, 6 in 2024-02-26T04:51:41.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:41 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:51:41.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:41 smithi156 ceph-mon[35805]: from='client.? 172.21.15.191:0/1418591072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:51:41.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:41 smithi060 ceph-mon[32382]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:41.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:41 smithi060 ceph-mon[32382]: from='client.? 172.21.15.191:0/995276504' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7807402-620a-40f2-85b4-71f762a1a73b"}]: dispatch 2024-02-26T04:51:41.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:41 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7807402-620a-40f2-85b4-71f762a1a73b"}]: dispatch 2024-02-26T04:51:41.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:41 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7807402-620a-40f2-85b4-71f762a1a73b"}]': finished 2024-02-26T04:51:41.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:41 smithi060 ceph-mon[32382]: osdmap e35: 6 total, 5 up, 6 in 2024-02-26T04:51:41.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:41 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:51:41.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:41 smithi060 ceph-mon[32382]: from='client.? 172.21.15.191:0/1418591072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:51:43.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:43 smithi191 ceph-mon[36352]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:43.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:43 smithi156 ceph-mon[35805]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:43.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:43 smithi060 ceph-mon[32382]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:45.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:45 smithi191 ceph-mon[36352]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:45.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:45 smithi156 ceph-mon[35805]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:45.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:45 smithi060 ceph-mon[32382]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:47.679 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:47 smithi191 ceph-mon[36352]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:47.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:47 smithi156 ceph-mon[35805]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:47.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:47 smithi060 ceph-mon[32382]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:48.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:48 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-26T04:51:48.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:48 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:48.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:48 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-26T04:51:48.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:48 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:48.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:48 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-26T04:51:48.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:48 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:51:49.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:49 smithi191 ceph-mon[36352]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:49.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:49 smithi191 ceph-mon[36352]: Deploying daemon osd.5 on smithi191 2024-02-26T04:51:49.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:49 smithi156 ceph-mon[35805]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:49.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:49 smithi156 ceph-mon[35805]: Deploying daemon osd.5 on smithi191 2024-02-26T04:51:49.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:49 smithi060 ceph-mon[32382]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:49.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:49 smithi060 ceph-mon[32382]: Deploying daemon osd.5 on smithi191 2024-02-26T04:51:51.729 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:51 smithi191 ceph-mon[36352]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:51.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:51 smithi156 ceph-mon[35805]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:51.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:51 smithi060 ceph-mon[32382]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:53.749 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:53 smithi191 ceph-mon[36352]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:53.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:53 smithi156 ceph-mon[35805]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:53.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:53 smithi060 ceph-mon[32382]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:55.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:55 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:55.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:55 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:55.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:55 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:55.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:55 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:55.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:55 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:55.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:55 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:55.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:55 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:55.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:55 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:55.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:55 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:51:56.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:56 smithi191 ceph-mon[36352]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:56.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:56 smithi156 ceph-mon[35805]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:56 smithi060 ceph-mon[32382]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:57.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:57.635+0000 7f81d77fe700 1 -- 172.21.15.191:0/699086092 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f81e002d870 con 0x7f81cc0a2150 2024-02-26T04:51:57.639 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 5 on host 'smithi191' 2024-02-26T04:51:57.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:57.637+0000 7f81e4996700 1 -- 172.21.15.191:0/699086092 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f81cc0a2150 msgr2=0x7f81cc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:51:57.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:57.637+0000 7f81e4996700 1 --2- 172.21.15.191:0/699086092 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f81cc0a2150 0x7f81cc0a4610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009510 tx=0x7f81d0009230 comp rx=0 tx=0).stop 2024-02-26T04:51:57.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:57.637+0000 7f81e4996700 1 -- 172.21.15.191:0/699086092 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0101ec0 msgr2=0x7f81e019f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:51:57.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:57.637+0000 7f81e4996700 1 --2- 172.21.15.191:0/699086092 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0101ec0 0x7f81e019f1c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f81d800d980 tx=0x7f81d800dc90 comp rx=0 tx=0).stop 2024-02-26T04:51:57.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:57.637+0000 7f81e4996700 1 -- 172.21.15.191:0/699086092 shutdown_connections 2024-02-26T04:51:57.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:57.637+0000 7f81e4996700 1 --2- 172.21.15.191:0/699086092 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0101ec0 0x7f81e019f1c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:57.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:57.637+0000 7f81e4996700 1 --2- 172.21.15.191:0/699086092 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f81cc0a2150 0x7f81cc0a4610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:57.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:57.637+0000 7f81e4996700 1 --2- 172.21.15.191:0/699086092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e0100320 0x7f81e0197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:57.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:57.637+0000 7f81e4996700 1 --2- 172.21.15.191:0/699086092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e0101520 0x7f81e0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:51:57.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:57.637+0000 7f81e4996700 1 -- 172.21.15.191:0/699086092 >> 172.21.15.191:0/699086092 conn(0x7f81e00fb8e0 msgr2=0x7f81e0104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:51:57.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:57.637+0000 7f81e4996700 1 -- 172.21.15.191:0/699086092 shutdown_connections 2024-02-26T04:51:57.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:51:57.637+0000 7f81e4996700 1 -- 172.21.15.191:0/699086092 wait complete. 2024-02-26T04:51:58.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:58 smithi191 ceph-mon[36352]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:58.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:58 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:58.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:58 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:58.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:58 smithi191 ceph-mon[36352]: from='osd.5 [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T04:51:58.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:58 smithi191 ceph-mon[36352]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T04:51:58.331 DEBUG:teuthology.orchestra.run.smithi191:osd.5> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.5.service 2024-02-26T04:51:58.334 INFO:tasks.cephadm:Deploying osd.6 on smithi060 with /dev/vg_nvme/lv_2... 2024-02-26T04:51:58.334 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-26T04:51:58.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:58 smithi060 ceph-mon[32382]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:58.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:58.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:58.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:58 smithi060 ceph-mon[32382]: from='osd.5 [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T04:51:58.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:58 smithi060 ceph-mon[32382]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T04:51:58.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:58 smithi156 ceph-mon[35805]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:58.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:58 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:58.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:58 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:58.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:58 smithi156 ceph-mon[35805]: from='osd.5 [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T04:51:58.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:58 smithi156 ceph-mon[35805]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-26T04:51:58.588 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:51:59.681 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:59 smithi191 ceph-mon[36352]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:59.681 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:59 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:59.681 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:59 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:59.681 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:59 smithi191 ceph-mon[36352]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-26T04:51:59.682 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:59 smithi191 ceph-mon[36352]: osdmap e36: 6 total, 5 up, 6 in 2024-02-26T04:51:59.682 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:59 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:51:59.682 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:59 smithi191 ceph-mon[36352]: from='osd.5 [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:51:59.682 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:51:59 smithi191 ceph-mon[36352]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:51:59.682 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 04:51:59 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T04:51:59.634+0000 7fd9fb256700 -1 osd.5 0 waiting for initial osdmap 2024-02-26T04:51:59.682 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 04:51:59 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T04:51:59.640+0000 7fd9f7049700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T04:51:59.887 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:59 smithi060 ceph-mon[32382]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:59.887 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:59 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:59.887 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:59 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:59.887 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:59 smithi060 ceph-mon[32382]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-26T04:51:59.888 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:59 smithi060 ceph-mon[32382]: osdmap e36: 6 total, 5 up, 6 in 2024-02-26T04:51:59.888 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:59 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:51:59.888 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:59 smithi060 ceph-mon[32382]: from='osd.5 [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:51:59.888 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:51:59 smithi060 ceph-mon[32382]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:51:59.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:59 smithi156 ceph-mon[35805]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:51:59.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:59 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:59.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:59 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:51:59.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:59 smithi156 ceph-mon[35805]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-26T04:51:59.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:59 smithi156 ceph-mon[35805]: osdmap e36: 6 total, 5 up, 6 in 2024-02-26T04:51:59.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:59 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:51:59.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:59 smithi156 ceph-mon[35805]: from='osd.5 [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:51:59.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:51:59 smithi156 ceph-mon[35805]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:52:00.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:00 smithi191 ceph-mon[36352]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-26T04:52:00.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:00 smithi191 ceph-mon[36352]: osdmap e37: 6 total, 5 up, 6 in 2024-02-26T04:52:00.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:00 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:52:00.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:00 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:52:00.782 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:00 smithi060 ceph-mon[32382]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-26T04:52:00.782 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:00 smithi060 ceph-mon[32382]: osdmap e37: 6 total, 5 up, 6 in 2024-02-26T04:52:00.782 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:00 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:52:00.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:00 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:52:00.783 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-26T04:52:00.783 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-26T04:52:00.783 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-02-26T04:52:00.783 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-02-26T04:52:00.783 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00866023 s, 1.2 GB/s 2024-02-26T04:52:00.783 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-02-26T04:52:00.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:00 smithi156 ceph-mon[35805]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-26T04:52:00.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:00 smithi156 ceph-mon[35805]: osdmap e37: 6 total, 5 up, 6 in 2024-02-26T04:52:00.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:00 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:52:00.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:00 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:52:01.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi060:vg_nvme/lv_2 2024-02-26T04:52:01.703 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:52:01.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:01 smithi156 ceph-mon[35805]: purged_snaps scrub starts 2024-02-26T04:52:01.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:01 smithi156 ceph-mon[35805]: purged_snaps scrub ok 2024-02-26T04:52:01.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:01 smithi156 ceph-mon[35805]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:52:01.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:01 smithi156 ceph-mon[35805]: osd.5 [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417] boot 2024-02-26T04:52:01.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:01 smithi156 ceph-mon[35805]: osdmap e38: 6 total, 6 up, 6 in 2024-02-26T04:52:01.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:01 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:52:01.974 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:01 smithi060 ceph-mon[32382]: purged_snaps scrub starts 2024-02-26T04:52:01.974 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:01 smithi060 ceph-mon[32382]: purged_snaps scrub ok 2024-02-26T04:52:01.975 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:01 smithi060 ceph-mon[32382]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:52:01.975 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:01 smithi060 ceph-mon[32382]: osd.5 [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417] boot 2024-02-26T04:52:01.975 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:01 smithi060 ceph-mon[32382]: osdmap e38: 6 total, 6 up, 6 in 2024-02-26T04:52:01.975 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:01 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:52:01.993 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:01 smithi191 ceph-mon[36352]: purged_snaps scrub starts 2024-02-26T04:52:01.994 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:01 smithi191 ceph-mon[36352]: purged_snaps scrub ok 2024-02-26T04:52:01.994 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:01 smithi191 ceph-mon[36352]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-02-26T04:52:01.994 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:01 smithi191 ceph-mon[36352]: osd.5 [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417] boot 2024-02-26T04:52:01.994 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:01 smithi191 ceph-mon[36352]: osdmap e38: 6 total, 6 up, 6 in 2024-02-26T04:52:01.994 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:01 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-26T04:52:02.836 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: osdmap e39: 6 total, 6 up, 6 in 2024-02-26T04:52:02.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: Detected new or changed devices on smithi191 2024-02-26T04:52:02.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:02.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:02.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:02.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:02.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: Adjusting osd_memory_target on smithi191 to 8540M 2024-02-26T04:52:02.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:02.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:02.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:02.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:02.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:02.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:02.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:02.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:02.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:02 smithi060 ceph-mon[32382]: osdmap e40: 6 total, 6 up, 6 in 2024-02-26T04:52:02.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: osdmap e39: 6 total, 6 up, 6 in 2024-02-26T04:52:02.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: Detected new or changed devices on smithi191 2024-02-26T04:52:02.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:02.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:02.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:02.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:02.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: Adjusting osd_memory_target on smithi191 to 8540M 2024-02-26T04:52:02.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:02.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:02.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:02.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:02.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:02.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:02.930 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:02.930 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:02.930 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:02 smithi156 ceph-mon[35805]: osdmap e40: 6 total, 6 up, 6 in 2024-02-26T04:52:03.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: osdmap e39: 6 total, 6 up, 6 in 2024-02-26T04:52:03.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: Detected new or changed devices on smithi191 2024-02-26T04:52:03.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:03.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:03.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: Adjusting osd_memory_target on smithi191 to 8540M 2024-02-26T04:52:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:03.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:02 smithi191 ceph-mon[36352]: osdmap e40: 6 total, 6 up, 6 in 2024-02-26T04:52:03.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.392+0000 7f99250c2700 1 -- 172.21.15.60:0/834703605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9920100ba0 msgr2=0x7f9920100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:52:03.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.392+0000 7f99250c2700 1 --2- 172.21.15.60:0/834703605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9920100ba0 0x7f9920100fc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f99100099c0 tx=0x7f9910009cd0 comp rx=0 tx=0).stop 2024-02-26T04:52:03.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.392+0000 7f99250c2700 1 -- 172.21.15.60:0/834703605 shutdown_connections 2024-02-26T04:52:03.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.392+0000 7f99250c2700 1 --2- 172.21.15.60:0/834703605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9920102740 0x7f9920108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:03.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.392+0000 7f99250c2700 1 --2- 172.21.15.60:0/834703605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9920100ba0 0x7f9920100fc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:03.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.392+0000 7f99250c2700 1 --2- 172.21.15.60:0/834703605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9920101da0 0x7f9920102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:03.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.392+0000 7f99250c2700 1 -- 172.21.15.60:0/834703605 >> 172.21.15.60:0/834703605 conn(0x7f99200fc140 msgr2=0x7f99200fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:52:03.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.392+0000 7f99250c2700 1 -- 172.21.15.60:0/834703605 shutdown_connections 2024-02-26T04:52:03.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.392+0000 7f99250c2700 1 -- 172.21.15.60:0/834703605 wait complete. 2024-02-26T04:52:03.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.393+0000 7f99250c2700 1 Processor -- start 2024-02-26T04:52:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.393+0000 7f99250c2700 1 -- start start 2024-02-26T04:52:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.393+0000 7f99250c2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9920100ba0 0x7f992006ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:52:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.393+0000 7f99250c2700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9920101da0 0x7f992019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:52:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.393+0000 7f99250c2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9920102740 0x7f992019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:52:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.393+0000 7f99250c2700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992019f810 con 0x7f9920101da0 2024-02-26T04:52:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.393+0000 7f99250c2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992019f980 con 0x7f9920102740 2024-02-26T04:52:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.393+0000 7f99250c2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992019fb40 con 0x7f9920100ba0 2024-02-26T04:52:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.393+0000 7f991e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9920101da0 0x7f992019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:52:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.393+0000 7f991e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9920101da0 0x7f992019ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38980/0 (socket says 172.21.15.60:38980) 2024-02-26T04:52:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.393+0000 7f991e59c700 1 -- 172.21.15.60:0/3993780175 learned_addr learned my addr 172.21.15.60:0/3993780175 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:52:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.394+0000 7f991ed9d700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9920100ba0 0x7f992006ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:52:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.394+0000 7f991f59e700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9920102740 0x7f992019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:52:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.394+0000 7f991e59c700 1 -- 172.21.15.60:0/3993780175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9920100ba0 msgr2=0x7f992006ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:52:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.394+0000 7f991e59c700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9920100ba0 0x7f992006ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.394+0000 7f991e59c700 1 -- 172.21.15.60:0/3993780175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9920102740 msgr2=0x7f992019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:52:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.394+0000 7f991e59c700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9920102740 0x7f992019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.394+0000 7f991e59c700 1 -- 172.21.15.60:0/3993780175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99100095f0 con 0x7f9920101da0 2024-02-26T04:52:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.394+0000 7f991f59e700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9920102740 0x7f992019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:52:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.394+0000 7f991e59c700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9920101da0 0x7f992019ca30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f991400d950 tx=0x7f991400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:52:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.394+0000 7f9907fff700 1 -- 172.21.15.60:0/3993780175 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914009940 con 0x7f9920101da0 2024-02-26T04:52:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.394+0000 7f9907fff700 1 -- 172.21.15.60:0/3993780175 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9914010460 con 0x7f9920101da0 2024-02-26T04:52:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.394+0000 7f99250c2700 1 -- 172.21.15.60:0/3993780175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992019fe20 con 0x7f9920101da0 2024-02-26T04:52:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.394+0000 7f9907fff700 1 -- 172.21.15.60:0/3993780175 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991400f620 con 0x7f9920101da0 2024-02-26T04:52:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.394+0000 7f99250c2700 1 -- 172.21.15.60:0/3993780175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99201a0290 con 0x7f9920101da0 2024-02-26T04:52:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.395+0000 7f99250c2700 1 -- 172.21.15.60:0/3993780175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9920107f60 con 0x7f9920101da0 2024-02-26T04:52:03.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.396+0000 7f9907fff700 1 -- 172.21.15.60:0/3993780175 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f99140105d0 con 0x7f9920101da0 2024-02-26T04:52:03.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.397+0000 7f9907fff700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f99080a21d0 0x7f99080a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:52:03.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.397+0000 7f991ed9d700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f99080a21d0 0x7f99080a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:52:03.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.397+0000 7f9907fff700 1 -- 172.21.15.60:0/3993780175 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f99140ccd20 con 0x7f9920101da0 2024-02-26T04:52:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.397+0000 7f991ed9d700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f99080a21d0 0x7f99080a4690 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f99100038c0 tx=0x7f991000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:52:03.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.401+0000 7f9907fff700 1 -- 172.21.15.60:0/3993780175 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f991409a620 con 0x7f9920101da0 2024-02-26T04:52:03.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:03.613+0000 7f99250c2700 1 -- 172.21.15.60:0/3993780175 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f992002d870 con 0x7f99080a21d0 2024-02-26T04:52:04.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:04 smithi191 ceph-mon[36352]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:04.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:04 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:52:04.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:04 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:52:04.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:04 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:04.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:04 smithi191 ceph-mon[36352]: osdmap e41: 6 total, 6 up, 6 in 2024-02-26T04:52:04.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:04 smithi060 ceph-mon[32382]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:04.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:04 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:52:04.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:04 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:52:04.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:04 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:04.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:04 smithi060 ceph-mon[32382]: osdmap e41: 6 total, 6 up, 6 in 2024-02-26T04:52:04.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:04 smithi156 ceph-mon[35805]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:04.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:04 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:52:04.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:04 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:52:04.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:04 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:04.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:04 smithi156 ceph-mon[35805]: osdmap e41: 6 total, 6 up, 6 in 2024-02-26T04:52:05.601 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:05 smithi060 ceph-mon[32382]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:52:05.601 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:05 smithi060 ceph-mon[32382]: osdmap e42: 6 total, 6 up, 6 in 2024-02-26T04:52:05.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:05 smithi156 ceph-mon[35805]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:52:05.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:05 smithi156 ceph-mon[35805]: osdmap e42: 6 total, 6 up, 6 in 2024-02-26T04:52:05.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:05 smithi191 ceph-mon[36352]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:52:05.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:05 smithi191 ceph-mon[36352]: osdmap e42: 6 total, 6 up, 6 in 2024-02-26T04:52:06.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:06 smithi156 ceph-mon[35805]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:06.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:06 smithi060 ceph-mon[32382]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:06.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:06 smithi191 ceph-mon[36352]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:07.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:07 smithi156 ceph-mon[35805]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-02-26T04:52:07.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:07 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3252034601' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43d4456c-935e-4e9c-afe8-9a6c72a8a4ad"}]: dispatch 2024-02-26T04:52:07.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:07 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3252034601' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43d4456c-935e-4e9c-afe8-9a6c72a8a4ad"}]': finished 2024-02-26T04:52:07.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:07 smithi156 ceph-mon[35805]: osdmap e43: 7 total, 6 up, 7 in 2024-02-26T04:52:07.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:07.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:07 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4066689567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:52:07.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:07 smithi060 ceph-mon[32382]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-02-26T04:52:07.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:07 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3252034601' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43d4456c-935e-4e9c-afe8-9a6c72a8a4ad"}]: dispatch 2024-02-26T04:52:07.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:07 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3252034601' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43d4456c-935e-4e9c-afe8-9a6c72a8a4ad"}]': finished 2024-02-26T04:52:07.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:07 smithi060 ceph-mon[32382]: osdmap e43: 7 total, 6 up, 7 in 2024-02-26T04:52:07.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:07.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:07 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4066689567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:52:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:07 smithi191 ceph-mon[36352]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-02-26T04:52:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:07 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3252034601' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43d4456c-935e-4e9c-afe8-9a6c72a8a4ad"}]: dispatch 2024-02-26T04:52:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:07 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3252034601' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43d4456c-935e-4e9c-afe8-9a6c72a8a4ad"}]': finished 2024-02-26T04:52:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:07 smithi191 ceph-mon[36352]: osdmap e43: 7 total, 6 up, 7 in 2024-02-26T04:52:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:07.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:07 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4066689567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:52:09.531 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:09 smithi060 ceph-mon[32382]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-02-26T04:52:09.679 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:09 smithi156 ceph-mon[35805]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-02-26T04:52:09.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:09 smithi191 ceph-mon[36352]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-02-26T04:52:11.559 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:11 smithi060 ceph-mon[32382]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-02-26T04:52:11.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:11 smithi156 ceph-mon[35805]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-02-26T04:52:11.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:11 smithi191 ceph-mon[36352]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-02-26T04:52:13.673 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:13 smithi060 ceph-mon[32382]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T04:52:13.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:13 smithi156 ceph-mon[35805]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T04:52:13.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:13 smithi191 ceph-mon[36352]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-26T04:52:14.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:14 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-26T04:52:14.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:14 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:14.641 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:14 smithi060 ceph-mon[32382]: Deploying daemon osd.6 on smithi060 2024-02-26T04:52:14.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:14 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-26T04:52:14.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:14 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:14.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:14 smithi156 ceph-mon[35805]: Deploying daemon osd.6 on smithi060 2024-02-26T04:52:14.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:14 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-26T04:52:14.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:14 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:14.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:14 smithi191 ceph-mon[36352]: Deploying daemon osd.6 on smithi060 2024-02-26T04:52:15.670 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:15 smithi060 ceph-mon[32382]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-02-26T04:52:15.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:15 smithi156 ceph-mon[35805]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-02-26T04:52:15.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:15 smithi191 ceph-mon[36352]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-02-26T04:52:17.459 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:17 smithi060 ceph-mon[32382]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:17.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:17 smithi156 ceph-mon[35805]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:17.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:17 smithi191 ceph-mon[36352]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:19.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:19 smithi060 ceph-mon[32382]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:19.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:19 smithi156 ceph-mon[35805]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:19.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:19 smithi191 ceph-mon[36352]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:21.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:21.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:21.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:21.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:21.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:21.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:21.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:21.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:21.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:22.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:21 smithi156 ceph-mon[35805]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:22.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:21 smithi060 ceph-mon[32382]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:22.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:21 smithi191 ceph-mon[36352]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:23.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:23.723+0000 7f9907fff700 1 -- 172.21.15.60:0/3993780175 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f992002d870 con 0x7f99080a21d0 2024-02-26T04:52:23.726 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 6 on host 'smithi060' 2024-02-26T04:52:23.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:23.725+0000 7f99250c2700 1 -- 172.21.15.60:0/3993780175 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f99080a21d0 msgr2=0x7f99080a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:52:23.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:23.725+0000 7f99250c2700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f99080a21d0 0x7f99080a4690 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f99100038c0 tx=0x7f991000b540 comp rx=0 tx=0).stop 2024-02-26T04:52:23.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:23.725+0000 7f99250c2700 1 -- 172.21.15.60:0/3993780175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9920101da0 msgr2=0x7f992019ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:52:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:23.725+0000 7f99250c2700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9920101da0 0x7f992019ca30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f991400d950 tx=0x7f991400dc60 comp rx=0 tx=0).stop 2024-02-26T04:52:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:23.725+0000 7f99250c2700 1 -- 172.21.15.60:0/3993780175 shutdown_connections 2024-02-26T04:52:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:23.725+0000 7f99250c2700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f99080a21d0 0x7f99080a4690 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:23.725+0000 7f99250c2700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9920102740 0x7f992019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:23.725+0000 7f99250c2700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9920100ba0 0x7f992006ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:23.725+0000 7f99250c2700 1 --2- 172.21.15.60:0/3993780175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9920101da0 0x7f992019ca30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:23.725+0000 7f99250c2700 1 -- 172.21.15.60:0/3993780175 >> 172.21.15.60:0/3993780175 conn(0x7f99200fc140 msgr2=0x7f99201050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:52:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:23.725+0000 7f99250c2700 1 -- 172.21.15.60:0/3993780175 shutdown_connections 2024-02-26T04:52:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:52:23.725+0000 7f99250c2700 1 -- 172.21.15.60:0/3993780175 wait complete. 2024-02-26T04:52:23.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:23 smithi060 ceph-mon[32382]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:23.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:23 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:23.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:23 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:23.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:23 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:23.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:23 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:23.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:23 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:23.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:23 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:23.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:23 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:23.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:23 smithi060 ceph-mon[32382]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T04:52:23.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:23 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:24.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:23 smithi191 ceph-mon[36352]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:24.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:23 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:24.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:23 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:24.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:23 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:24.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:23 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:24.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:23 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:24.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:23 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:24.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:23 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:24.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:23 smithi191 ceph-mon[36352]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T04:52:24.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:23 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:24.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:23 smithi156 ceph-mon[35805]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:24.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:23 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:24.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:23 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:24.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:23 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:24.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:23 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:24.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:23 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:24.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:23 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:24.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:23 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:24.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:23 smithi156 ceph-mon[35805]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-26T04:52:24.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:23 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:24.447 DEBUG:teuthology.orchestra.run.smithi060:osd.6> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.6.service 2024-02-26T04:52:24.450 INFO:tasks.cephadm:Deploying osd.7 on smithi156 with /dev/vg_nvme/lv_2... 2024-02-26T04:52:24.450 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-26T04:52:24.696 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T04:52:25.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:25 smithi060 ceph-mon[32382]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:25.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:25 smithi060 ceph-mon[32382]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-26T04:52:25.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:25 smithi060 ceph-mon[32382]: osdmap e44: 7 total, 6 up, 7 in 2024-02-26T04:52:25.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:25 smithi060 ceph-mon[32382]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:52:25.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:25 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:26.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:25 smithi191 ceph-mon[36352]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:26.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:25 smithi191 ceph-mon[36352]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-26T04:52:26.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:25 smithi191 ceph-mon[36352]: osdmap e44: 7 total, 6 up, 7 in 2024-02-26T04:52:26.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:25 smithi191 ceph-mon[36352]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:52:26.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:25 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:26.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:25 smithi156 ceph-mon[35805]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:26.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:25 smithi156 ceph-mon[35805]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-26T04:52:26.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:25 smithi156 ceph-mon[35805]: osdmap e44: 7 total, 6 up, 7 in 2024-02-26T04:52:26.023 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:25 smithi156 ceph-mon[35805]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:52:26.024 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:25 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:26.480 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 04:52:26 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T04:52:26.227+0000 7f9d31a58700 -1 osd.6 0 waiting for initial osdmap 2024-02-26T04:52:26.480 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 04:52:26 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T04:52:26.237+0000 7f9d29042700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T04:52:26.887 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-26T04:52:26.887 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-26T04:52:26.887 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-02-26T04:52:26.887 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-02-26T04:52:26.887 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00904935 s, 1.2 GB/s 2024-02-26T04:52:26.887 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-02-26T04:52:26.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:26 smithi060 ceph-mon[32382]: purged_snaps scrub starts 2024-02-26T04:52:26.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:26 smithi060 ceph-mon[32382]: purged_snaps scrub ok 2024-02-26T04:52:26.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:26 smithi060 ceph-mon[32382]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T04:52:26.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:26 smithi060 ceph-mon[32382]: osdmap e45: 7 total, 6 up, 7 in 2024-02-26T04:52:26.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:26.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:26.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:26 smithi060 ceph-mon[32382]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' 2024-02-26T04:52:27.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:26 smithi191 ceph-mon[36352]: purged_snaps scrub starts 2024-02-26T04:52:27.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:26 smithi191 ceph-mon[36352]: purged_snaps scrub ok 2024-02-26T04:52:27.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:26 smithi191 ceph-mon[36352]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T04:52:27.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:26 smithi191 ceph-mon[36352]: osdmap e45: 7 total, 6 up, 7 in 2024-02-26T04:52:27.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:27.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:27.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:26 smithi191 ceph-mon[36352]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' 2024-02-26T04:52:27.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:26 smithi156 ceph-mon[35805]: purged_snaps scrub starts 2024-02-26T04:52:27.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:26 smithi156 ceph-mon[35805]: purged_snaps scrub ok 2024-02-26T04:52:27.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:26 smithi156 ceph-mon[35805]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T04:52:27.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:26 smithi156 ceph-mon[35805]: osdmap e45: 7 total, 6 up, 7 in 2024-02-26T04:52:27.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:27.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:27.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:26 smithi156 ceph-mon[35805]: from='osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804]' entity='osd.6' 2024-02-26T04:52:27.583 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_2 2024-02-26T04:52:27.828 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T04:52:27.863 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:27 smithi156 ceph-mon[35805]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:27.864 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:27 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:27.864 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:27 smithi156 ceph-mon[35805]: osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804] boot 2024-02-26T04:52:27.864 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:27 smithi156 ceph-mon[35805]: osdmap e46: 7 total, 7 up, 7 in 2024-02-26T04:52:27.864 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:27 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:27.864 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:27 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:27.864 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:27 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:27.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:27 smithi060 ceph-mon[32382]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:27.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:27 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:27.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:27 smithi060 ceph-mon[32382]: osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804] boot 2024-02-26T04:52:27.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:27 smithi060 ceph-mon[32382]: osdmap e46: 7 total, 7 up, 7 in 2024-02-26T04:52:27.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:27 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:27.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:27 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:27.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:27 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:28.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:27 smithi191 ceph-mon[36352]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-02-26T04:52:28.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:27 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:28.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:27 smithi191 ceph-mon[36352]: osd.6 [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804] boot 2024-02-26T04:52:28.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:27 smithi191 ceph-mon[36352]: osdmap e46: 7 total, 7 up, 7 in 2024-02-26T04:52:28.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:27 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-26T04:52:28.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:27 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:28.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:27 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:28.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:28 smithi060 ceph-mon[32382]: Detected new or changed devices on smithi060 2024-02-26T04:52:28.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:28 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:28.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:28 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:28.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:28 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:28.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:28 smithi060 ceph-mon[32382]: Adjusting osd_memory_target on smithi060 to 5697M 2024-02-26T04:52:28.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:28 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:28.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:28 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:28.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:28 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:28.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:28 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:28.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:28 smithi060 ceph-mon[32382]: osdmap e47: 7 total, 7 up, 7 in 2024-02-26T04:52:28.997 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:28 smithi156 ceph-mon[35805]: Detected new or changed devices on smithi060 2024-02-26T04:52:28.997 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:28 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:28.997 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:28 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:28.998 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:28 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:28.998 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:28 smithi156 ceph-mon[35805]: Adjusting osd_memory_target on smithi060 to 5697M 2024-02-26T04:52:28.998 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:28 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:28.998 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:28 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:28.998 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:28 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:28.998 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:28 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:28.998 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:28 smithi156 ceph-mon[35805]: osdmap e47: 7 total, 7 up, 7 in 2024-02-26T04:52:29.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:28 smithi191 ceph-mon[36352]: Detected new or changed devices on smithi060 2024-02-26T04:52:29.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:28 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:29.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:28 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:29.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:28 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:29.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:28 smithi191 ceph-mon[36352]: Adjusting osd_memory_target on smithi060 to 5697M 2024-02-26T04:52:29.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:28 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:29.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:28 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:29.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:28 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:29.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:28 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:29.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:28 smithi191 ceph-mon[36352]: osdmap e47: 7 total, 7 up, 7 in 2024-02-26T04:52:29.531 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.527+0000 7f38ca871700 1 -- 172.21.15.156:0/3071639570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c4105090 msgr2=0x7f38c4107480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:52:29.531 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.527+0000 7f38ca871700 1 --2- 172.21.15.156:0/3071639570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c4105090 0x7f38c4107480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009990 tx=0x7f38b8009ca0 comp rx=0 tx=0).stop 2024-02-26T04:52:29.531 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.527+0000 7f38ca871700 1 -- 172.21.15.156:0/3071639570 shutdown_connections 2024-02-26T04:52:29.531 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.527+0000 7f38ca871700 1 --2- 172.21.15.156:0/3071639570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c41079c0 0x7f38c4109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:29.531 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.527+0000 7f38ca871700 1 --2- 172.21.15.156:0/3071639570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38c4102760 0x7f38c4104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:29.532 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.527+0000 7f38ca871700 1 --2- 172.21.15.156:0/3071639570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c4105090 0x7f38c4107480 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:29.532 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.527+0000 7f38ca871700 1 -- 172.21.15.156:0/3071639570 >> 172.21.15.156:0/3071639570 conn(0x7f38c40fc190 msgr2=0x7f38c40fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:52:29.532 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.527+0000 7f38ca871700 1 -- 172.21.15.156:0/3071639570 shutdown_connections 2024-02-26T04:52:29.536 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.527+0000 7f38ca871700 1 -- 172.21.15.156:0/3071639570 wait complete. 2024-02-26T04:52:29.536 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.527+0000 7f38ca871700 1 Processor -- start 2024-02-26T04:52:29.536 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.527+0000 7f38ca871700 1 -- start start 2024-02-26T04:52:29.536 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38ca871700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c4102760 0x7f38c4197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:52:29.536 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38ca871700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38c4105090 0x7f38c4198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:52:29.536 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38ca871700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c41079c0 0x7f38c419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:52:29.536 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38ca871700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c419f790 con 0x7f38c4105090 2024-02-26T04:52:29.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38ca871700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c419f8d0 con 0x7f38c41079c0 2024-02-26T04:52:29.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38ca871700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c419fa90 con 0x7f38c4102760 2024-02-26T04:52:29.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c8e0e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c41079c0 0x7f38c419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:52:29.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c3fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c4102760 0x7f38c4197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:52:29.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c8e0e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c41079c0 0x7f38c419f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35632/0 (socket says 172.21.15.156:35632) 2024-02-26T04:52:29.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c3fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c4102760 0x7f38c4197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.156:33118/0 (socket says 172.21.15.156:33118) 2024-02-26T04:52:29.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c8e0e700 1 -- 172.21.15.156:0/812583786 learned_addr learned my addr 172.21.15.156:0/812583786 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-02-26T04:52:29.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c37fe700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38c4105090 0x7f38c4198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:52:29.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c8e0e700 1 -- 172.21.15.156:0/812583786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c4102760 msgr2=0x7f38c4197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:52:29.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c8e0e700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c4102760 0x7f38c4197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:29.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c8e0e700 1 -- 172.21.15.156:0/812583786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38c4105090 msgr2=0x7f38c4198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:52:29.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c8e0e700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38c4105090 0x7f38c4198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:29.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c8e0e700 1 -- 172.21.15.156:0/812583786 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b80095f0 con 0x7f38c41079c0 2024-02-26T04:52:29.538 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c37fe700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38c4105090 0x7f38c4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:52:29.538 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c3fff700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c4102760 0x7f38c4197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:52:29.538 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c8e0e700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c41079c0 0x7f38c419f1c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00ef90 tx=0x7f38bc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:52:29.538 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.528+0000 7f38c17fa700 1 -- 172.21.15.156:0/812583786 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc018070 con 0x7f38c41079c0 2024-02-26T04:52:29.538 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.529+0000 7f38ca871700 1 -- 172.21.15.156:0/812583786 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c419fd10 con 0x7f38c41079c0 2024-02-26T04:52:29.538 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.529+0000 7f38c17fa700 1 -- 172.21.15.156:0/812583786 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38bc00f040 con 0x7f38c41079c0 2024-02-26T04:52:29.538 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.529+0000 7f38c17fa700 1 -- 172.21.15.156:0/812583786 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc004020 con 0x7f38c41079c0 2024-02-26T04:52:29.538 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.529+0000 7f38ca871700 1 -- 172.21.15.156:0/812583786 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c41a01c0 con 0x7f38c41079c0 2024-02-26T04:52:29.539 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.530+0000 7f38ca871700 1 -- 172.21.15.156:0/812583786 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c40fdd10 con 0x7f38c41079c0 2024-02-26T04:52:29.539 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.531+0000 7f38c17fa700 1 -- 172.21.15.156:0/812583786 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f38bc004ad0 con 0x7f38c41079c0 2024-02-26T04:52:29.539 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.531+0000 7f38c17fa700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f38ac0a2150 0x7f38ac0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:52:29.539 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.531+0000 7f38c17fa700 1 -- 172.21.15.156:0/812583786 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f38bc0ccb90 con 0x7f38c41079c0 2024-02-26T04:52:29.539 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.531+0000 7f38c3fff700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f38ac0a2150 0x7f38ac0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:52:29.539 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.532+0000 7f38c3fff700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f38ac0a2150 0x7f38ac0a4610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f38b4005950 tx=0x7f38b400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:52:29.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.536+0000 7f38c17fa700 1 -- 172.21.15.156:0/812583786 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f38bc096900 con 0x7f38c41079c0 2024-02-26T04:52:29.740 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:29.737+0000 7f38ca871700 1 -- 172.21.15.156:0/812583786 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f38c402d7d0 con 0x7f38ac0a2150 2024-02-26T04:52:30.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:30 smithi191 ceph-mon[36352]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:30.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:30 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:52:30.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:30 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:52:30.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:30 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:30.595 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:30 smithi156 ceph-mon[35805]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:30.595 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:30 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:52:30.595 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:30 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:52:30.595 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:30 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:30.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:30 smithi060 ceph-mon[32382]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:30.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:30 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:52:30.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:30 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:52:30.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:30 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:31.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:31 smithi191 ceph-mon[36352]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:52:31.550 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:31 smithi156 ceph-mon[35805]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:52:31.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:31 smithi060 ceph-mon[32382]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:52:32.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:32 smithi191 ceph-mon[36352]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:32.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:32 smithi156 ceph-mon[35805]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:32.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:32 smithi060 ceph-mon[32382]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:33.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:33 smithi191 ceph-mon[36352]: from='client.? 172.21.15.156:0/3129404851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e158aaaf-5538-4711-ae58-2d9acc28c9f2"}]: dispatch 2024-02-26T04:52:33.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:33 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e158aaaf-5538-4711-ae58-2d9acc28c9f2"}]: dispatch 2024-02-26T04:52:33.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:33 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e158aaaf-5538-4711-ae58-2d9acc28c9f2"}]': finished 2024-02-26T04:52:33.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:33 smithi191 ceph-mon[36352]: osdmap e48: 8 total, 7 up, 8 in 2024-02-26T04:52:33.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:33 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:33.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:33 smithi191 ceph-mon[36352]: from='client.? 172.21.15.156:0/331811910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:52:33.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:33 smithi156 ceph-mon[35805]: from='client.? 172.21.15.156:0/3129404851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e158aaaf-5538-4711-ae58-2d9acc28c9f2"}]: dispatch 2024-02-26T04:52:33.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:33 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e158aaaf-5538-4711-ae58-2d9acc28c9f2"}]: dispatch 2024-02-26T04:52:33.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:33 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e158aaaf-5538-4711-ae58-2d9acc28c9f2"}]': finished 2024-02-26T04:52:33.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:33 smithi156 ceph-mon[35805]: osdmap e48: 8 total, 7 up, 8 in 2024-02-26T04:52:33.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:33 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:33.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:33 smithi156 ceph-mon[35805]: from='client.? 172.21.15.156:0/331811910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:52:33.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:33 smithi060 ceph-mon[32382]: from='client.? 172.21.15.156:0/3129404851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e158aaaf-5538-4711-ae58-2d9acc28c9f2"}]: dispatch 2024-02-26T04:52:33.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:33 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e158aaaf-5538-4711-ae58-2d9acc28c9f2"}]: dispatch 2024-02-26T04:52:33.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:33 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e158aaaf-5538-4711-ae58-2d9acc28c9f2"}]': finished 2024-02-26T04:52:33.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:33 smithi060 ceph-mon[32382]: osdmap e48: 8 total, 7 up, 8 in 2024-02-26T04:52:33.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:33 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:33.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:33 smithi060 ceph-mon[32382]: from='client.? 172.21.15.156:0/331811910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:52:34.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:34 smithi191 ceph-mon[36352]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:34.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:34 smithi156 ceph-mon[35805]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:34.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:34 smithi060 ceph-mon[32382]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:36.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:36 smithi156 ceph-mon[35805]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:36.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:36 smithi191 ceph-mon[36352]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:36.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:36 smithi060 ceph-mon[32382]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:38.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:38 smithi191 ceph-mon[36352]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:38.623 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:38 smithi156 ceph-mon[35805]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:38.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:38 smithi060 ceph-mon[32382]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:40.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:40 smithi191 ceph-mon[36352]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:40.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:40 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-26T04:52:40.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:40 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:40.531 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:40 smithi060 ceph-mon[32382]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:40.532 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:40 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-26T04:52:40.532 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:40 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:40.603 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:40 smithi156 ceph-mon[35805]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:40.603 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:40 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-26T04:52:40.603 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:40 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:41.523 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:41 smithi156 ceph-mon[35805]: Deploying daemon osd.7 on smithi156 2024-02-26T04:52:41.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:41 smithi060 ceph-mon[32382]: Deploying daemon osd.7 on smithi156 2024-02-26T04:52:41.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:41 smithi191 ceph-mon[36352]: Deploying daemon osd.7 on smithi156 2024-02-26T04:52:42.546 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:42 smithi156 ceph-mon[35805]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:42.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:42 smithi060 ceph-mon[32382]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:42.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:42 smithi191 ceph-mon[36352]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:43.614 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:43 smithi156 ceph-mon[35805]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:43.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:43 smithi060 ceph-mon[32382]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:43.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:43 smithi191 ceph-mon[36352]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:45.609 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:45 smithi156 ceph-mon[35805]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:45.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:45 smithi060 ceph-mon[32382]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:45.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:45 smithi191 ceph-mon[36352]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:46.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:46.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:46.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:46.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:46.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:46.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:47.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:47.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:47.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:47.912 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:47 smithi156 ceph-mon[35805]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:47.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:47 smithi060 ceph-mon[32382]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:48.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:47 smithi191 ceph-mon[36352]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:49.155 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 7 on host 'smithi156' 2024-02-26T04:52:49.155 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:49.152+0000 7f38c17fa700 1 -- 172.21.15.156:0/812583786 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f38c402d7d0 con 0x7f38ac0a2150 2024-02-26T04:52:49.157 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:49.155+0000 7f38ca871700 1 -- 172.21.15.156:0/812583786 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f38ac0a2150 msgr2=0x7f38ac0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:52:49.157 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:49.155+0000 7f38ca871700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f38ac0a2150 0x7f38ac0a4610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f38b4005950 tx=0x7f38b400a300 comp rx=0 tx=0).stop 2024-02-26T04:52:49.157 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:49.155+0000 7f38ca871700 1 -- 172.21.15.156:0/812583786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c41079c0 msgr2=0x7f38c419f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:52:49.157 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:49.155+0000 7f38ca871700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c41079c0 0x7f38c419f1c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00ef90 tx=0x7f38bc00c3a0 comp rx=0 tx=0).stop 2024-02-26T04:52:49.157 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:49.155+0000 7f38ca871700 1 -- 172.21.15.156:0/812583786 shutdown_connections 2024-02-26T04:52:49.158 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:49.155+0000 7f38ca871700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f38ac0a2150 0x7f38ac0a4610 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:49.158 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:49.155+0000 7f38ca871700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38c41079c0 0x7f38c419f1c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:49.158 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:49.155+0000 7f38ca871700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38c4102760 0x7f38c4197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:49.158 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:49.155+0000 7f38ca871700 1 --2- 172.21.15.156:0/812583786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38c4105090 0x7f38c4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:49.158 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:49.155+0000 7f38ca871700 1 -- 172.21.15.156:0/812583786 >> 172.21.15.156:0/812583786 conn(0x7f38c40fc190 msgr2=0x7f38c4103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:52:49.158 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:49.155+0000 7f38ca871700 1 -- 172.21.15.156:0/812583786 shutdown_connections 2024-02-26T04:52:49.158 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:52:49.155+0000 7f38ca871700 1 -- 172.21.15.156:0/812583786 wait complete. 2024-02-26T04:52:49.812 DEBUG:teuthology.orchestra.run.smithi156:osd.7> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.7.service 2024-02-26T04:52:49.814 INFO:tasks.cephadm:Deploying osd.8 on smithi191 with /dev/vg_nvme/lv_2... 2024-02-26T04:52:49.814 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-26T04:52:50.063 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T04:52:50.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:50 smithi156 ceph-mon[35805]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:50.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:50 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:50.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:50 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:50.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:50 smithi156 ceph-mon[35805]: from='osd.7 [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T04:52:50.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:50 smithi156 ceph-mon[35805]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T04:52:50.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:50 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:50.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:50 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:50.442 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:50 smithi191 ceph-mon[36352]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:50.442 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:50 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:50.442 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:50 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:50.442 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:50 smithi191 ceph-mon[36352]: from='osd.7 [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T04:52:50.442 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:50 smithi191 ceph-mon[36352]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T04:52:50.442 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:50 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:50.442 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:50 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:50.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:50 smithi060 ceph-mon[32382]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:50.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:50.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:50.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:50 smithi060 ceph-mon[32382]: from='osd.7 [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T04:52:50.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:50 smithi060 ceph-mon[32382]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-26T04:52:50.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:50.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:51.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:51 smithi156 ceph-mon[35805]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-26T04:52:51.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:51 smithi156 ceph-mon[35805]: from='osd.7 [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:52:51.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:51 smithi156 ceph-mon[35805]: osdmap e49: 8 total, 7 up, 8 in 2024-02-26T04:52:51.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:51 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:51.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:51 smithi156 ceph-mon[35805]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:52:51.430 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:51 smithi191 ceph-mon[36352]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-26T04:52:51.430 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:51 smithi191 ceph-mon[36352]: from='osd.7 [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:52:51.430 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:51 smithi191 ceph-mon[36352]: osdmap e49: 8 total, 7 up, 8 in 2024-02-26T04:52:51.430 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:51 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:51.431 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:51 smithi191 ceph-mon[36352]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:52:51.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:51 smithi060 ceph-mon[32382]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-26T04:52:51.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:51 smithi060 ceph-mon[32382]: from='osd.7 [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:52:51.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:51 smithi060 ceph-mon[32382]: osdmap e49: 8 total, 7 up, 8 in 2024-02-26T04:52:51.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:51 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:51.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:51 smithi060 ceph-mon[32382]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:52:51.858 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 04:52:51 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T04:52:51.594+0000 7f4875dfc700 -1 osd.7 0 waiting for initial osdmap 2024-02-26T04:52:51.859 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 04:52:51 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T04:52:51.605+0000 7f486dbe7700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T04:52:52.125 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-26T04:52:52.126 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-26T04:52:52.126 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-02-26T04:52:52.126 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-02-26T04:52:52.126 INFO:teuthology.orchestra.run.smithi191.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112501 s, 932 MB/s 2024-02-26T04:52:52.126 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 2024-02-26T04:52:52.126 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-02-26T04:52:52.426 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:52 smithi191 ceph-mon[36352]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:52.426 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:52 smithi191 ceph-mon[36352]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-26T04:52:52.426 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:52 smithi191 ceph-mon[36352]: osdmap e50: 8 total, 7 up, 8 in 2024-02-26T04:52:52.426 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:52 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:52.426 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:52 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:52.427 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:52 smithi191 ceph-mon[36352]: from='osd.7 ' entity='osd.7' 2024-02-26T04:52:52.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:52 smithi156 ceph-mon[35805]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:52.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:52 smithi156 ceph-mon[35805]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-26T04:52:52.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:52 smithi156 ceph-mon[35805]: osdmap e50: 8 total, 7 up, 8 in 2024-02-26T04:52:52.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:52 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:52.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:52 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:52.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:52 smithi156 ceph-mon[35805]: from='osd.7 ' entity='osd.7' 2024-02-26T04:52:52.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:52 smithi060 ceph-mon[32382]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-02-26T04:52:52.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:52 smithi060 ceph-mon[32382]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-26T04:52:52.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:52 smithi060 ceph-mon[32382]: osdmap e50: 8 total, 7 up, 8 in 2024-02-26T04:52:52.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:52 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:52.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:52 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:52.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:52 smithi060 ceph-mon[32382]: from='osd.7 ' entity='osd.7' 2024-02-26T04:52:52.834 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_2 2024-02-26T04:52:53.073 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T04:52:53.350 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:53 smithi156 ceph-mon[35805]: purged_snaps scrub starts 2024-02-26T04:52:53.350 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:53 smithi156 ceph-mon[35805]: purged_snaps scrub ok 2024-02-26T04:52:53.350 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:53 smithi156 ceph-mon[35805]: osd.7 [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318] boot 2024-02-26T04:52:53.350 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:53 smithi156 ceph-mon[35805]: osdmap e51: 8 total, 8 up, 8 in 2024-02-26T04:52:53.350 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:53 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:53.416 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:53 smithi191 ceph-mon[36352]: purged_snaps scrub starts 2024-02-26T04:52:53.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:53 smithi191 ceph-mon[36352]: purged_snaps scrub ok 2024-02-26T04:52:53.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:53 smithi191 ceph-mon[36352]: osd.7 [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318] boot 2024-02-26T04:52:53.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:53 smithi191 ceph-mon[36352]: osdmap e51: 8 total, 8 up, 8 in 2024-02-26T04:52:53.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:53 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:53.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:53 smithi060 ceph-mon[32382]: purged_snaps scrub starts 2024-02-26T04:52:53.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:53 smithi060 ceph-mon[32382]: purged_snaps scrub ok 2024-02-26T04:52:53.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:53 smithi060 ceph-mon[32382]: osd.7 [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318] boot 2024-02-26T04:52:53.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:53 smithi060 ceph-mon[32382]: osdmap e51: 8 total, 8 up, 8 in 2024-02-26T04:52:53.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:53 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-26T04:52:54.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:52:54.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: osdmap e52: 8 total, 8 up, 8 in 2024-02-26T04:52:54.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:54.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:54.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:54.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:54.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:54.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:54.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:54.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:54.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:52:54.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: osdmap e52: 8 total, 8 up, 8 in 2024-02-26T04:52:54.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:54.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:54.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:54.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:54.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:54.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:54.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:54.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:54.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:52:54.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: osdmap e52: 8 total, 8 up, 8 in 2024-02-26T04:52:54.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:54.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:54.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:52:54.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:54.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:54.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:52:54.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:54.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:52:54.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:52:54.847 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.842+0000 7f3b97f25700 1 -- 172.21.15.191:0/3701816226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90101df0 msgr2=0x7f3b90102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:52:54.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.842+0000 7f3b97f25700 1 --2- 172.21.15.191:0/3701816226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90101df0 0x7f3b90102250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009990 tx=0x7f3b8c009ca0 comp rx=0 tx=0).stop 2024-02-26T04:52:54.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.842+0000 7f3b97f25700 1 -- 172.21.15.191:0/3701816226 shutdown_connections 2024-02-26T04:52:54.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.842+0000 7f3b97f25700 1 --2- 172.21.15.191:0/3701816226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b90102790 0x7f3b90109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:54.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.842+0000 7f3b97f25700 1 --2- 172.21.15.191:0/3701816226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b90100bf0 0x7f3b90101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:54.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.842+0000 7f3b97f25700 1 --2- 172.21.15.191:0/3701816226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90101df0 0x7f3b90102250 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:54.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.842+0000 7f3b97f25700 1 -- 172.21.15.191:0/3701816226 >> 172.21.15.191:0/3701816226 conn(0x7f3b900fc170 msgr2=0x7f3b900fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:52:54.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.842+0000 7f3b97f25700 1 -- 172.21.15.191:0/3701816226 shutdown_connections 2024-02-26T04:52:54.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.842+0000 7f3b97f25700 1 -- 172.21.15.191:0/3701816226 wait complete. 2024-02-26T04:52:54.848 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.843+0000 7f3b97f25700 1 Processor -- start 2024-02-26T04:52:54.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.843+0000 7f3b97f25700 1 -- start start 2024-02-26T04:52:54.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.843+0000 7f3b97f25700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b90100bf0 0x7f3b9006aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:52:54.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.843+0000 7f3b97f25700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b90101df0 0x7f3b9019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:52:54.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.843+0000 7f3b97f25700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90102790 0x7f3b9019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:52:54.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.843+0000 7f3b97f25700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9019f7d0 con 0x7f3b90100bf0 2024-02-26T04:52:54.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.843+0000 7f3b97f25700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9019f940 con 0x7f3b90101df0 2024-02-26T04:52:54.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.843+0000 7f3b97f25700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9019fb00 con 0x7f3b90102790 2024-02-26T04:52:54.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.843+0000 7f3b964c2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90102790 0x7f3b9019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:52:54.849 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.843+0000 7f3b964c2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90102790 0x7f3b9019f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:36368/0 (socket says 172.21.15.191:36368) 2024-02-26T04:52:54.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.843+0000 7f3b964c2700 1 -- 172.21.15.191:0/2429041626 learned_addr learned my addr 172.21.15.191:0/2429041626 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-26T04:52:54.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.843+0000 7f3b95cc1700 1 --2- 172.21.15.191:0/2429041626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b90100bf0 0x7f3b9006aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:52:54.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.843+0000 7f3b954c0700 1 --2- 172.21.15.191:0/2429041626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b90101df0 0x7f3b9019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:52:54.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.844+0000 7f3b964c2700 1 -- 172.21.15.191:0/2429041626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b90101df0 msgr2=0x7f3b9019c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:52:54.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.844+0000 7f3b964c2700 1 --2- 172.21.15.191:0/2429041626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b90101df0 0x7f3b9019c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:54.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.844+0000 7f3b964c2700 1 -- 172.21.15.191:0/2429041626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b90100bf0 msgr2=0x7f3b9006aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:52:54.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.844+0000 7f3b964c2700 1 --2- 172.21.15.191:0/2429041626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b90100bf0 0x7f3b9006aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:52:54.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.844+0000 7f3b964c2700 1 -- 172.21.15.191:0/2429041626 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c0095f0 con 0x7f3b90102790 2024-02-26T04:52:54.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.844+0000 7f3b964c2700 1 --2- 172.21.15.191:0/2429041626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90102790 0x7f3b9019f030 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800b970 tx=0x7f3b8800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:52:54.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.844+0000 7f3b82ffd700 1 -- 172.21.15.191:0/2429041626 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8800d860 con 0x7f3b90102790 2024-02-26T04:52:54.850 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.844+0000 7f3b97f25700 1 -- 172.21.15.191:0/2429041626 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b9019fde0 con 0x7f3b90102790 2024-02-26T04:52:54.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.844+0000 7f3b82ffd700 1 -- 172.21.15.191:0/2429041626 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b8800e430 con 0x7f3b90102790 2024-02-26T04:52:54.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.844+0000 7f3b82ffd700 1 -- 172.21.15.191:0/2429041626 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8800dea0 con 0x7f3b90102790 2024-02-26T04:52:54.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.844+0000 7f3b97f25700 1 -- 172.21.15.191:0/2429041626 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b901a02e0 con 0x7f3b90102790 2024-02-26T04:52:54.851 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.845+0000 7f3b97f25700 1 -- 172.21.15.191:0/2429041626 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b90107fb0 con 0x7f3b90102790 2024-02-26T04:52:54.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.846+0000 7f3b82ffd700 1 -- 172.21.15.191:0/2429041626 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3b8800e910 con 0x7f3b90102790 2024-02-26T04:52:54.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.847+0000 7f3b82ffd700 1 --2- 172.21.15.191:0/2429041626 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3b7c0a2150 0x7f3b7c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:52:54.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.847+0000 7f3b82ffd700 1 -- 172.21.15.191:0/2429041626 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f3b88006e40 con 0x7f3b90102790 2024-02-26T04:52:54.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.847+0000 7f3b95cc1700 1 --2- 172.21.15.191:0/2429041626 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3b7c0a2150 0x7f3b7c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:52:54.852 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.848+0000 7f3b95cc1700 1 --2- 172.21.15.191:0/2429041626 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3b7c0a2150 0x7f3b7c0a4610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3b90101c50 tx=0x7f3b8400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:52:54.854 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:54.851+0000 7f3b82ffd700 1 -- 172.21.15.191:0/2429041626 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3b880d2020 con 0x7f3b90102790 2024-02-26T04:52:55.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:52:55.053+0000 7f3b97f25700 1 -- 172.21.15.191:0/2429041626 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b9002d870 con 0x7f3b7c0a2150 2024-02-26T04:52:55.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:55 smithi156 ceph-mon[35805]: Detected new or changed devices on smithi156 2024-02-26T04:52:55.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:55 smithi156 ceph-mon[35805]: Adjusting osd_memory_target on smithi156 to 5693M 2024-02-26T04:52:55.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:55 smithi156 ceph-mon[35805]: osdmap e53: 8 total, 8 up, 8 in 2024-02-26T04:52:55.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:55 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:52:55.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:55 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:52:55.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:55 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:55.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:55 smithi060 ceph-mon[32382]: Detected new or changed devices on smithi156 2024-02-26T04:52:55.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:55 smithi060 ceph-mon[32382]: Adjusting osd_memory_target on smithi156 to 5693M 2024-02-26T04:52:55.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:55 smithi060 ceph-mon[32382]: osdmap e53: 8 total, 8 up, 8 in 2024-02-26T04:52:55.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:55 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:52:55.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:55 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:52:55.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:55 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:55.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:55 smithi191 ceph-mon[36352]: Detected new or changed devices on smithi156 2024-02-26T04:52:55.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:55 smithi191 ceph-mon[36352]: Adjusting osd_memory_target on smithi156 to 5693M 2024-02-26T04:52:55.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:55 smithi191 ceph-mon[36352]: osdmap e53: 8 total, 8 up, 8 in 2024-02-26T04:52:55.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:55 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:52:55.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:55 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:52:55.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:55 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:52:56.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:56 smithi191 ceph-mon[36352]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:52:56.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:56 smithi191 ceph-mon[36352]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:52:56.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:56 smithi191 ceph-mon[36352]: osdmap e54: 8 total, 8 up, 8 in 2024-02-26T04:52:56.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:56 smithi156 ceph-mon[35805]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:52:56.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:56 smithi156 ceph-mon[35805]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:52:56.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:56 smithi156 ceph-mon[35805]: osdmap e54: 8 total, 8 up, 8 in 2024-02-26T04:52:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:56 smithi060 ceph-mon[32382]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:52:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:56 smithi060 ceph-mon[32382]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:52:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:56 smithi060 ceph-mon[32382]: osdmap e54: 8 total, 8 up, 8 in 2024-02-26T04:52:57.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:57 smithi191 ceph-mon[36352]: osdmap e55: 8 total, 8 up, 8 in 2024-02-26T04:52:57.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:57 smithi156 ceph-mon[35805]: osdmap e55: 8 total, 8 up, 8 in 2024-02-26T04:52:57.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:57 smithi060 ceph-mon[32382]: osdmap e55: 8 total, 8 up, 8 in 2024-02-26T04:52:58.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:58 smithi156 ceph-mon[35805]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:52:58.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:58 smithi156 ceph-mon[35805]: from='client.? 172.21.15.191:0/743755059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5669eb6b-6750-411a-bd9f-61cf8db220fc"}]: dispatch 2024-02-26T04:52:58.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:58 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5669eb6b-6750-411a-bd9f-61cf8db220fc"}]: dispatch 2024-02-26T04:52:58.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:58 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5669eb6b-6750-411a-bd9f-61cf8db220fc"}]': finished 2024-02-26T04:52:58.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:58 smithi156 ceph-mon[35805]: osdmap e56: 9 total, 8 up, 9 in 2024-02-26T04:52:58.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:58 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:52:58.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:58 smithi060 ceph-mon[32382]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:52:58.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:58 smithi060 ceph-mon[32382]: from='client.? 172.21.15.191:0/743755059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5669eb6b-6750-411a-bd9f-61cf8db220fc"}]: dispatch 2024-02-26T04:52:58.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:58 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5669eb6b-6750-411a-bd9f-61cf8db220fc"}]: dispatch 2024-02-26T04:52:58.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:58 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5669eb6b-6750-411a-bd9f-61cf8db220fc"}]': finished 2024-02-26T04:52:58.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:58 smithi060 ceph-mon[32382]: osdmap e56: 9 total, 8 up, 9 in 2024-02-26T04:52:58.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:52:58.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:58 smithi191 ceph-mon[36352]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:52:58.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:58 smithi191 ceph-mon[36352]: from='client.? 172.21.15.191:0/743755059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5669eb6b-6750-411a-bd9f-61cf8db220fc"}]: dispatch 2024-02-26T04:52:58.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:58 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5669eb6b-6750-411a-bd9f-61cf8db220fc"}]: dispatch 2024-02-26T04:52:58.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:58 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5669eb6b-6750-411a-bd9f-61cf8db220fc"}]': finished 2024-02-26T04:52:58.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:58 smithi191 ceph-mon[36352]: osdmap e56: 9 total, 8 up, 9 in 2024-02-26T04:52:58.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:58 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:52:59.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:52:59 smithi060 ceph-mon[32382]: from='client.? 172.21.15.191:0/3702688903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:52:59.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:52:59 smithi191 ceph-mon[36352]: from='client.? 172.21.15.191:0/3702688903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:52:59.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:52:59 smithi156 ceph-mon[35805]: from='client.? 172.21.15.191:0/3702688903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:53:00.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:00 smithi060 ceph-mon[32382]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:00.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:00 smithi191 ceph-mon[36352]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:00.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:00 smithi156 ceph-mon[35805]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:02.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:02 smithi191 ceph-mon[36352]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:02.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:02 smithi060 ceph-mon[32382]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:02.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:02 smithi156 ceph-mon[35805]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:04.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:04 smithi191 ceph-mon[36352]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:04.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:04 smithi060 ceph-mon[32382]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:04.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:04 smithi156 ceph-mon[35805]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:06.293 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:06 smithi191 ceph-mon[36352]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:06.293 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:06 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-26T04:53:06.293 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:06 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:06.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:06 smithi060 ceph-mon[32382]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:06.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:06 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-26T04:53:06.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:06 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:06.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:06 smithi156 ceph-mon[35805]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:06.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:06 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-26T04:53:06.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:06 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:07.406 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:07 smithi191 ceph-mon[36352]: Deploying daemon osd.8 on smithi191 2024-02-26T04:53:07.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:07 smithi060 ceph-mon[32382]: Deploying daemon osd.8 on smithi191 2024-02-26T04:53:07.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:07 smithi156 ceph-mon[35805]: Deploying daemon osd.8 on smithi191 2024-02-26T04:53:08.409 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:08 smithi191 ceph-mon[36352]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:08.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:08 smithi060 ceph-mon[32382]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:08.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:08 smithi156 ceph-mon[35805]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:10.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:10 smithi060 ceph-mon[32382]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:10.480 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:10 smithi156 ceph-mon[35805]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:10.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:10 smithi191 ceph-mon[36352]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:12.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:12 smithi156 ceph-mon[35805]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:12.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:12.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:12.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:53:12.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:12 smithi060 ceph-mon[32382]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:12.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:12.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:12.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:53:12.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:12 smithi191 ceph-mon[36352]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:12.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:12.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:12.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:53:14.240 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:14 smithi191 ceph-mon[36352]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:14.240 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:14 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:14.240 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:14 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:14.240 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:14 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:14.240 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:14 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:53:14.240 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:14 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:14.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:14 smithi156 ceph-mon[35805]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:14.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:14 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:14.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:14 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:14.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:14 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:14.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:14 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:53:14.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:14 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:14.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:14 smithi060 ceph-mon[32382]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:14.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:14 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:14.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:14 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:14.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:14 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:14.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:14 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:53:14.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:14 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:14.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:53:14.977+0000 7f3b82ffd700 1 -- 172.21.15.191:0/2429041626 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3b9002d870 con 0x7f3b7c0a2150 2024-02-26T04:53:14.981 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 8 on host 'smithi191' 2024-02-26T04:53:14.982 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:53:14.980+0000 7f3b97f25700 1 -- 172.21.15.191:0/2429041626 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3b7c0a2150 msgr2=0x7f3b7c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:53:14.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:53:14.980+0000 7f3b97f25700 1 --2- 172.21.15.191:0/2429041626 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3b7c0a2150 0x7f3b7c0a4610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3b90101c50 tx=0x7f3b8400b410 comp rx=0 tx=0).stop 2024-02-26T04:53:14.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:53:14.980+0000 7f3b97f25700 1 -- 172.21.15.191:0/2429041626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90102790 msgr2=0x7f3b9019f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:53:14.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:53:14.980+0000 7f3b97f25700 1 --2- 172.21.15.191:0/2429041626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90102790 0x7f3b9019f030 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800b970 tx=0x7f3b8800bc80 comp rx=0 tx=0).stop 2024-02-26T04:53:14.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:53:14.980+0000 7f3b97f25700 1 -- 172.21.15.191:0/2429041626 shutdown_connections 2024-02-26T04:53:14.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:53:14.980+0000 7f3b97f25700 1 --2- 172.21.15.191:0/2429041626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90102790 0x7f3b9019f030 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:14.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:53:14.980+0000 7f3b97f25700 1 --2- 172.21.15.191:0/2429041626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b90100bf0 0x7f3b9006aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:14.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:53:14.980+0000 7f3b97f25700 1 --2- 172.21.15.191:0/2429041626 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3b7c0a2150 0x7f3b7c0a4610 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:14.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:53:14.980+0000 7f3b97f25700 1 --2- 172.21.15.191:0/2429041626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b90101df0 0x7f3b9019c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:14.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:53:14.980+0000 7f3b97f25700 1 -- 172.21.15.191:0/2429041626 >> 172.21.15.191:0/2429041626 conn(0x7f3b900fc170 msgr2=0x7f3b901050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:53:14.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:53:14.980+0000 7f3b97f25700 1 -- 172.21.15.191:0/2429041626 shutdown_connections 2024-02-26T04:53:14.983 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:53:14.980+0000 7f3b97f25700 1 -- 172.21.15.191:0/2429041626 wait complete. 2024-02-26T04:53:15.742 DEBUG:teuthology.orchestra.run.smithi191:osd.8> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.8.service 2024-02-26T04:53:15.744 INFO:tasks.cephadm:Deploying osd.9 on smithi060 with /dev/vg_nvme/lv_1... 2024-02-26T04:53:15.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-26T04:53:15.999 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:53:16.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:15 smithi191 ceph-mon[36352]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:16.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:15 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:16.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:15 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:16.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:15 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:53:16.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:15 smithi191 ceph-mon[36352]: from='osd.8 [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T04:53:16.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:15 smithi191 ceph-mon[36352]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T04:53:16.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:15 smithi060 ceph-mon[32382]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:16.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:15 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:16.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:15 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:16.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:15 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:53:16.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:15 smithi060 ceph-mon[32382]: from='osd.8 [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T04:53:16.120 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:15 smithi060 ceph-mon[32382]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T04:53:16.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:15 smithi156 ceph-mon[35805]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:16.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:15 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:16.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:15 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:16.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:15 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:53:16.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:15 smithi156 ceph-mon[35805]: from='osd.8 [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T04:53:16.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:15 smithi156 ceph-mon[35805]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-26T04:53:17.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:16 smithi060 ceph-mon[32382]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-26T04:53:17.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:16 smithi060 ceph-mon[32382]: osdmap e57: 9 total, 8 up, 9 in 2024-02-26T04:53:17.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:16 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:53:17.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:16 smithi060 ceph-mon[32382]: from='osd.8 [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:53:17.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:16 smithi060 ceph-mon[32382]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:53:17.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:16 smithi191 ceph-mon[36352]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-26T04:53:17.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:16 smithi191 ceph-mon[36352]: osdmap e57: 9 total, 8 up, 9 in 2024-02-26T04:53:17.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:16 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:53:17.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:16 smithi191 ceph-mon[36352]: from='osd.8 [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:53:17.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:16 smithi191 ceph-mon[36352]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:53:17.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:16 smithi156 ceph-mon[35805]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-26T04:53:17.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:16 smithi156 ceph-mon[35805]: osdmap e57: 9 total, 8 up, 9 in 2024-02-26T04:53:17.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:16 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:53:17.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:16 smithi156 ceph-mon[35805]: from='osd.8 [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:53:17.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:16 smithi156 ceph-mon[35805]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:53:17.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 04:53:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T04:53:17.439+0000 7f6a88cd4700 -1 osd.8 0 waiting for initial osdmap 2024-02-26T04:53:17.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 04:53:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T04:53:17.450+0000 7f6a802be700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T04:53:18.087 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-26T04:53:18.087 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-26T04:53:18.087 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2024-02-26T04:53:18.087 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2024-02-26T04:53:18.087 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00852736 s, 1.2 GB/s 2024-02-26T04:53:18.087 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2024-02-26T04:53:18.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:17 smithi060 ceph-mon[32382]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:18.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:17 smithi060 ceph-mon[32382]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-26T04:53:18.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:17 smithi060 ceph-mon[32382]: osdmap e58: 9 total, 8 up, 9 in 2024-02-26T04:53:18.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:53:18.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:53:18.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:17 smithi060 ceph-mon[32382]: from='osd.8 ' entity='osd.8' 2024-02-26T04:53:18.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:17 smithi191 ceph-mon[36352]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:18.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:17 smithi191 ceph-mon[36352]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-26T04:53:18.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:17 smithi191 ceph-mon[36352]: osdmap e58: 9 total, 8 up, 9 in 2024-02-26T04:53:18.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:53:18.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:53:18.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:17 smithi191 ceph-mon[36352]: from='osd.8 ' entity='osd.8' 2024-02-26T04:53:18.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:17 smithi156 ceph-mon[35805]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:18.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:17 smithi156 ceph-mon[35805]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-26T04:53:18.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:17 smithi156 ceph-mon[35805]: osdmap e58: 9 total, 8 up, 9 in 2024-02-26T04:53:18.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:17 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:53:18.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:17 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:53:18.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:17 smithi156 ceph-mon[35805]: from='osd.8 ' entity='osd.8' 2024-02-26T04:53:18.799 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi060:vg_nvme/lv_1 2024-02-26T04:53:19.048 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:53:19.171 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:18 smithi060 ceph-mon[32382]: purged_snaps scrub starts 2024-02-26T04:53:19.171 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:18 smithi060 ceph-mon[32382]: purged_snaps scrub ok 2024-02-26T04:53:19.171 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:18 smithi060 ceph-mon[32382]: osd.8 [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461] boot 2024-02-26T04:53:19.171 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:18 smithi060 ceph-mon[32382]: osdmap e59: 9 total, 9 up, 9 in 2024-02-26T04:53:19.171 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:18 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:53:19.171 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:18 smithi060 ceph-mon[32382]: osdmap e60: 9 total, 9 up, 9 in 2024-02-26T04:53:19.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:18 smithi191 ceph-mon[36352]: purged_snaps scrub starts 2024-02-26T04:53:19.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:18 smithi191 ceph-mon[36352]: purged_snaps scrub ok 2024-02-26T04:53:19.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:18 smithi191 ceph-mon[36352]: osd.8 [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461] boot 2024-02-26T04:53:19.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:18 smithi191 ceph-mon[36352]: osdmap e59: 9 total, 9 up, 9 in 2024-02-26T04:53:19.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:18 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:53:19.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:18 smithi191 ceph-mon[36352]: osdmap e60: 9 total, 9 up, 9 in 2024-02-26T04:53:19.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:18 smithi156 ceph-mon[35805]: purged_snaps scrub starts 2024-02-26T04:53:19.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:18 smithi156 ceph-mon[35805]: purged_snaps scrub ok 2024-02-26T04:53:19.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:18 smithi156 ceph-mon[35805]: osd.8 [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461] boot 2024-02-26T04:53:19.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:18 smithi156 ceph-mon[35805]: osdmap e59: 9 total, 9 up, 9 in 2024-02-26T04:53:19.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:18 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-26T04:53:19.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:18 smithi156 ceph-mon[35805]: osdmap e60: 9 total, 9 up, 9 in 2024-02-26T04:53:20.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:20 smithi156 ceph-mon[35805]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:20 smithi156 ceph-mon[35805]: Detected new or changed devices on smithi191 2024-02-26T04:53:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:20 smithi156 ceph-mon[35805]: Adjusting osd_memory_target on smithi191 to 5693M 2024-02-26T04:53:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:53:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:20 smithi156 ceph-mon[35805]: osdmap e61: 9 total, 9 up, 9 in 2024-02-26T04:53:20.445 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:20 smithi060 ceph-mon[32382]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:20.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:20 smithi060 ceph-mon[32382]: Detected new or changed devices on smithi191 2024-02-26T04:53:20.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:20.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:20.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:20.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:20.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:20.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:20 smithi060 ceph-mon[32382]: Adjusting osd_memory_target on smithi191 to 5693M 2024-02-26T04:53:20.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:20.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:20.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:53:20.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:20.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:20 smithi060 ceph-mon[32382]: osdmap e61: 9 total, 9 up, 9 in 2024-02-26T04:53:20.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:20 smithi191 ceph-mon[36352]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-02-26T04:53:20.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:20 smithi191 ceph-mon[36352]: Detected new or changed devices on smithi191 2024-02-26T04:53:20.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:20 smithi191 ceph-mon[36352]: Adjusting osd_memory_target on smithi191 to 5693M 2024-02-26T04:53:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:53:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:20 smithi191 ceph-mon[36352]: osdmap e61: 9 total, 9 up, 9 in 2024-02-26T04:53:20.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.859+0000 7fa3934ab700 1 -- 172.21.15.60:0/799784587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa38c101ca0 msgr2=0x7fa38c1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:53:20.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.859+0000 7fa3934ab700 1 --2- 172.21.15.60:0/799784587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa38c101ca0 0x7fa38c1020c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00b0d0 tx=0x7fa37c00b3e0 comp rx=0 tx=0).stop 2024-02-26T04:53:20.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.859+0000 7fa3934ab700 1 -- 172.21.15.60:0/799784587 shutdown_connections 2024-02-26T04:53:20.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.859+0000 7fa3934ab700 1 --2- 172.21.15.60:0/799784587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38c103840 0x7fa38c10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:20.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.859+0000 7fa3934ab700 1 --2- 172.21.15.60:0/799784587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa38c101ca0 0x7fa38c1020c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:20.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.859+0000 7fa3934ab700 1 --2- 172.21.15.60:0/799784587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c102ea0 0x7fa38c103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:20.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.859+0000 7fa3934ab700 1 -- 172.21.15.60:0/799784587 >> 172.21.15.60:0/799784587 conn(0x7fa38c0fd280 msgr2=0x7fa38c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:53:20.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.860+0000 7fa3934ab700 1 -- 172.21.15.60:0/799784587 shutdown_connections 2024-02-26T04:53:20.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.860+0000 7fa3934ab700 1 -- 172.21.15.60:0/799784587 wait complete. 2024-02-26T04:53:20.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.860+0000 7fa3934ab700 1 Processor -- start 2024-02-26T04:53:20.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.860+0000 7fa3934ab700 1 -- start start 2024-02-26T04:53:20.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.860+0000 7fa3934ab700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38c101ca0 0x7fa38c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:53:20.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.860+0000 7fa3934ab700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c102ea0 0x7fa38c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:53:20.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.860+0000 7fa3934ab700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa38c103840 0x7fa38c19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:53:20.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.860+0000 7fa3934ab700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c19f8e0 con 0x7fa38c103840 2024-02-26T04:53:20.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.860+0000 7fa3934ab700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c19fa20 con 0x7fa38c102ea0 2024-02-26T04:53:20.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.860+0000 7fa3934ab700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c19fbb0 con 0x7fa38c101ca0 2024-02-26T04:53:20.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa391a48700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa38c103840 0x7fa38c19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:53:20.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa391a48700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa38c103840 0x7fa38c19f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36274/0 (socket says 172.21.15.60:36274) 2024-02-26T04:53:20.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa391a48700 1 -- 172.21.15.60:0/2320105037 learned_addr learned my addr 172.21.15.60:0/2320105037 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:53:20.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa391247700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38c101ca0 0x7fa38c197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:53:20.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa390a46700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c102ea0 0x7fa38c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:53:20.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa391a48700 1 -- 172.21.15.60:0/2320105037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38c101ca0 msgr2=0x7fa38c197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:53:20.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa391a48700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38c101ca0 0x7fa38c197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:20.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa391a48700 1 -- 172.21.15.60:0/2320105037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c102ea0 msgr2=0x7fa38c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:53:20.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa391a48700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c102ea0 0x7fa38c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:20.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa391a48700 1 -- 172.21.15.60:0/2320105037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c009d00 con 0x7fa38c103840 2024-02-26T04:53:20.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa391247700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38c101ca0 0x7fa38c197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:53:20.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa391a48700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa38c103840 0x7fa38c19f310 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa38400c930 tx=0x7fa38400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:53:20.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa38a7fc700 1 -- 172.21.15.60:0/2320105037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa384007ac0 con 0x7fa38c103840 2024-02-26T04:53:20.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa38a7fc700 1 -- 172.21.15.60:0/2320105037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa384007c20 con 0x7fa38c103840 2024-02-26T04:53:20.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa38a7fc700 1 -- 172.21.15.60:0/2320105037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa384018660 con 0x7fa38c103840 2024-02-26T04:53:20.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa3934ab700 1 -- 172.21.15.60:0/2320105037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38c19fe90 con 0x7fa38c103840 2024-02-26T04:53:20.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.861+0000 7fa3934ab700 1 -- 172.21.15.60:0/2320105037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38c1a02b0 con 0x7fa38c103840 2024-02-26T04:53:20.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.862+0000 7fa3934ab700 1 -- 172.21.15.60:0/2320105037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38c068930 con 0x7fa38c103840 2024-02-26T04:53:20.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.863+0000 7fa38a7fc700 1 -- 172.21.15.60:0/2320105037 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fa38400f450 con 0x7fa38c103840 2024-02-26T04:53:20.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.864+0000 7fa38a7fc700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa3740a2150 0x7fa3740a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:53:20.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.864+0000 7fa391247700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa3740a2150 0x7fa3740a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:53:20.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.864+0000 7fa38a7fc700 1 -- 172.21.15.60:0/2320105037 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fa3840cd2f0 con 0x7fa38c103840 2024-02-26T04:53:20.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.864+0000 7fa391247700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa3740a2150 0x7fa3740a4610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa37c007b60 tx=0x7fa37c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:53:20.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:20.868+0000 7fa38a7fc700 1 -- 172.21.15.60:0/2320105037 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa384096c50 con 0x7fa38c103840 2024-02-26T04:53:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:21.083+0000 7fa3934ab700 1 -- 172.21.15.60:0/2320105037 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa38c02d910 con 0x7fa3740a2150 2024-02-26T04:53:22.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:21 smithi156 ceph-mon[35805]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:21 smithi156 ceph-mon[35805]: osdmap e62: 9 total, 9 up, 9 in 2024-02-26T04:53:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:21 smithi156 ceph-mon[35805]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:53:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:53:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:53:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:22.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:21 smithi191 ceph-mon[36352]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:22.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:21 smithi191 ceph-mon[36352]: osdmap e62: 9 total, 9 up, 9 in 2024-02-26T04:53:22.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:21 smithi191 ceph-mon[36352]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:53:22.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:53:22.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:53:22.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:22.396 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:21 smithi060 ceph-mon[32382]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:22.396 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:21 smithi060 ceph-mon[32382]: osdmap e62: 9 total, 9 up, 9 in 2024-02-26T04:53:22.397 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:21 smithi060 ceph-mon[32382]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:53:22.397 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:53:22.397 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:53:22.397 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:23.109 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:22 smithi060 ceph-mon[32382]: osdmap e63: 9 total, 9 up, 9 in 2024-02-26T04:53:23.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:22 smithi191 ceph-mon[36352]: osdmap e63: 9 total, 9 up, 9 in 2024-02-26T04:53:23.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:22 smithi156 ceph-mon[35805]: osdmap e63: 9 total, 9 up, 9 in 2024-02-26T04:53:24.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:23 smithi191 ceph-mon[36352]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T04:53:24.331 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:23 smithi060 ceph-mon[32382]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T04:53:24.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:23 smithi156 ceph-mon[35805]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-26T04:53:25.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:24 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/89924560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0802892-114a-4875-9914-aa936d399e4f"}]: dispatch 2024-02-26T04:53:25.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:24 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/89924560' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0802892-114a-4875-9914-aa936d399e4f"}]': finished 2024-02-26T04:53:25.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:24 smithi191 ceph-mon[36352]: osdmap e64: 10 total, 9 up, 10 in 2024-02-26T04:53:25.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:25.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:24 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1821418270' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:53:25.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:24 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/89924560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0802892-114a-4875-9914-aa936d399e4f"}]: dispatch 2024-02-26T04:53:25.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:24 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/89924560' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0802892-114a-4875-9914-aa936d399e4f"}]': finished 2024-02-26T04:53:25.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:24 smithi156 ceph-mon[35805]: osdmap e64: 10 total, 9 up, 10 in 2024-02-26T04:53:25.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:25.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:24 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1821418270' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:53:25.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:24 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/89924560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0802892-114a-4875-9914-aa936d399e4f"}]: dispatch 2024-02-26T04:53:25.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:24 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/89924560' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0802892-114a-4875-9914-aa936d399e4f"}]': finished 2024-02-26T04:53:25.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:24 smithi060 ceph-mon[32382]: osdmap e64: 10 total, 9 up, 10 in 2024-02-26T04:53:25.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:25.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:24 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1821418270' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:53:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:26 smithi191 ceph-mon[36352]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-02-26T04:53:26.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:26 smithi156 ceph-mon[35805]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-02-26T04:53:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:26 smithi060 ceph-mon[32382]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-02-26T04:53:28.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:28 smithi191 ceph-mon[36352]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T04:53:28.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:28 smithi060 ceph-mon[32382]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T04:53:28.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:28 smithi156 ceph-mon[35805]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T04:53:30.328 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:30 smithi060 ceph-mon[32382]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-02-26T04:53:30.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:30 smithi156 ceph-mon[35805]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-02-26T04:53:30.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:30 smithi191 ceph-mon[36352]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-02-26T04:53:32.325 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:32 smithi060 ceph-mon[32382]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-02-26T04:53:32.325 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:32 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-26T04:53:32.326 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:32 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:32.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:32 smithi156 ceph-mon[35805]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-02-26T04:53:32.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:32 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-26T04:53:32.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:32 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:32.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:32 smithi191 ceph-mon[36352]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-02-26T04:53:32.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:32 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-26T04:53:32.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:32 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:33.411 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:33 smithi060 ceph-mon[32382]: Deploying daemon osd.9 on smithi060 2024-02-26T04:53:33.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:33 smithi156 ceph-mon[35805]: Deploying daemon osd.9 on smithi060 2024-02-26T04:53:33.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:33 smithi191 ceph-mon[36352]: Deploying daemon osd.9 on smithi060 2024-02-26T04:53:34.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:34 smithi060 ceph-mon[32382]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:34.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:34 smithi156 ceph-mon[35805]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:34.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:34 smithi191 ceph-mon[36352]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:36.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:36 smithi060 ceph-mon[32382]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:36.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:36 smithi156 ceph-mon[35805]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:36.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:36 smithi191 ceph-mon[36352]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:38.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:38 smithi060 ceph-mon[32382]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:38.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:38.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:38.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:53:38.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:38 smithi156 ceph-mon[35805]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:38.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:38.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:38.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:53:38.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:38 smithi191 ceph-mon[36352]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:38.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:38.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:38.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:53:40.197 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:40 smithi060 ceph-mon[32382]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:40.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:40 smithi156 ceph-mon[35805]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:40.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:40 smithi191 ceph-mon[36352]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:41.298 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 9 on host 'smithi060' 2024-02-26T04:53:41.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:41.294+0000 7fa38a7fc700 1 -- 172.21.15.60:0/2320105037 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa38c02d910 con 0x7fa3740a2150 2024-02-26T04:53:41.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:41.296+0000 7fa3934ab700 1 -- 172.21.15.60:0/2320105037 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa3740a2150 msgr2=0x7fa3740a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:53:41.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:41.296+0000 7fa3934ab700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa3740a2150 0x7fa3740a4610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa37c007b60 tx=0x7fa37c00b040 comp rx=0 tx=0).stop 2024-02-26T04:53:41.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:41.297+0000 7fa3934ab700 1 -- 172.21.15.60:0/2320105037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa38c103840 msgr2=0x7fa38c19f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:53:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:41.297+0000 7fa3934ab700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa38c103840 0x7fa38c19f310 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa38400c930 tx=0x7fa38400ccf0 comp rx=0 tx=0).stop 2024-02-26T04:53:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:41.297+0000 7fa3934ab700 1 -- 172.21.15.60:0/2320105037 shutdown_connections 2024-02-26T04:53:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:41.297+0000 7fa3934ab700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa38c103840 0x7fa38c19f310 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:41.297+0000 7fa3934ab700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38c101ca0 0x7fa38c197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:41.297+0000 7fa3934ab700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa3740a2150 0x7fa3740a4610 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:41.297+0000 7fa3934ab700 1 --2- 172.21.15.60:0/2320105037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c102ea0 0x7fa38c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:41.297+0000 7fa3934ab700 1 -- 172.21.15.60:0/2320105037 >> 172.21.15.60:0/2320105037 conn(0x7fa38c0fd280 msgr2=0x7fa38c1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:53:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:41.297+0000 7fa3934ab700 1 -- 172.21.15.60:0/2320105037 shutdown_connections 2024-02-26T04:53:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:53:41.297+0000 7fa3934ab700 1 -- 172.21.15.60:0/2320105037 wait complete. 2024-02-26T04:53:41.969 DEBUG:teuthology.orchestra.run.smithi060:osd.9> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.9.service 2024-02-26T04:53:41.972 INFO:tasks.cephadm:Deploying osd.10 on smithi156 with /dev/vg_nvme/lv_1... 2024-02-26T04:53:41.972 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-26T04:53:42.221 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T04:53:42.342 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:42 smithi156 ceph-mon[35805]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:42.342 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:42 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:42.342 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:42 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:42.342 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:42 smithi156 ceph-mon[35805]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T04:53:42.343 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:42 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:42.343 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:42 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:42.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:42 smithi191 ceph-mon[36352]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:42.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:42 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:42.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:42 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:42.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:42 smithi191 ceph-mon[36352]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T04:53:42.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:42 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:42.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:42 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:42.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:42 smithi060 ceph-mon[32382]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:42.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:42 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:42.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:42 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:42.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:42 smithi060 ceph-mon[32382]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-26T04:53:42.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:42 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:42.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:42 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:43.404 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:43 smithi156 ceph-mon[35805]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-26T04:53:43.404 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:43 smithi156 ceph-mon[35805]: osdmap e65: 10 total, 9 up, 10 in 2024-02-26T04:53:43.404 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:43 smithi156 ceph-mon[35805]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:53:43.405 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:43 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:43.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:43 smithi060 ceph-mon[32382]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-26T04:53:43.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:43 smithi060 ceph-mon[32382]: osdmap e65: 10 total, 9 up, 10 in 2024-02-26T04:53:43.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:43 smithi060 ceph-mon[32382]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:53:43.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:43 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:43.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:43 smithi191 ceph-mon[36352]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-26T04:53:43.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:43 smithi191 ceph-mon[36352]: osdmap e65: 10 total, 9 up, 10 in 2024-02-26T04:53:43.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:43 smithi191 ceph-mon[36352]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-02-26T04:53:43.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:43 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:43.979 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 04:53:43 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T04:53:43.761+0000 7f50e6d09700 -1 osd.9 0 waiting for initial osdmap 2024-02-26T04:53:43.979 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 04:53:43 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T04:53:43.773+0000 7f50deaf4700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T04:53:44.271 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-26T04:53:44.271 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-26T04:53:44.271 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-02-26T04:53:44.271 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-02-26T04:53:44.271 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00842124 s, 1.2 GB/s 2024-02-26T04:53:44.271 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-02-26T04:53:44.662 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:44 smithi156 ceph-mon[35805]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:44.662 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:44 smithi156 ceph-mon[35805]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T04:53:44.663 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:44 smithi156 ceph-mon[35805]: osdmap e66: 10 total, 9 up, 10 in 2024-02-26T04:53:44.663 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:44 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:44.663 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:44 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:44.663 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:44 smithi156 ceph-mon[35805]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' 2024-02-26T04:53:44.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:44 smithi060 ceph-mon[32382]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:44.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:44 smithi060 ceph-mon[32382]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T04:53:44.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:44 smithi060 ceph-mon[32382]: osdmap e66: 10 total, 9 up, 10 in 2024-02-26T04:53:44.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:44 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:44.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:44 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:44.732 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:44 smithi060 ceph-mon[32382]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' 2024-02-26T04:53:44.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:44 smithi191 ceph-mon[36352]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:44.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:44 smithi191 ceph-mon[36352]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-02-26T04:53:44.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:44 smithi191 ceph-mon[36352]: osdmap e66: 10 total, 9 up, 10 in 2024-02-26T04:53:44.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:44 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:44.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:44 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:44.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:44 smithi191 ceph-mon[36352]: from='osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343]' entity='osd.9' 2024-02-26T04:53:45.068 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_1 2024-02-26T04:53:45.305 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T04:53:45.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:45 smithi156 ceph-mon[35805]: purged_snaps scrub starts 2024-02-26T04:53:45.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:45 smithi156 ceph-mon[35805]: purged_snaps scrub ok 2024-02-26T04:53:45.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:45 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:45.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:45 smithi156 ceph-mon[35805]: osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343] boot 2024-02-26T04:53:45.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:45 smithi156 ceph-mon[35805]: osdmap e67: 10 total, 10 up, 10 in 2024-02-26T04:53:45.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:45 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:45.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:45 smithi060 ceph-mon[32382]: purged_snaps scrub starts 2024-02-26T04:53:45.594 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:45 smithi060 ceph-mon[32382]: purged_snaps scrub ok 2024-02-26T04:53:45.594 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:45 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:45.594 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:45 smithi060 ceph-mon[32382]: osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343] boot 2024-02-26T04:53:45.594 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:45 smithi060 ceph-mon[32382]: osdmap e67: 10 total, 10 up, 10 in 2024-02-26T04:53:45.594 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:45 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:45.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:45 smithi191 ceph-mon[36352]: purged_snaps scrub starts 2024-02-26T04:53:45.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:45 smithi191 ceph-mon[36352]: purged_snaps scrub ok 2024-02-26T04:53:45.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:45 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:45.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:45 smithi191 ceph-mon[36352]: osd.9 [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343] boot 2024-02-26T04:53:45.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:45 smithi191 ceph-mon[36352]: osdmap e67: 10 total, 10 up, 10 in 2024-02-26T04:53:45.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:45 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-26T04:53:46.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:46.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:46.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:46.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:46.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:46.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:46.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:53:46.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:53:46.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:46.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:53:46.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:46 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:46.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:46.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:46.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:46.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:46.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:46.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:53:46.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:53:46.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:46.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:53:46.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:46 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-02-26T04:53:46.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:46.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:46.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:46.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:53:46.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:46.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:53:46.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:53:46.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:46.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:53:46.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:46 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:53:46.930 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.926+0000 7f111daa3700 1 -- 172.21.15.156:0/417473700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1118103870 msgr2=0x7f111810a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:53:46.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.926+0000 7f111daa3700 1 --2- 172.21.15.156:0/417473700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1118103870 0x7f111810a120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f111000b0d0 tx=0x7f111000b3e0 comp rx=0 tx=0).stop 2024-02-26T04:53:46.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.926+0000 7f111daa3700 1 -- 172.21.15.156:0/417473700 shutdown_connections 2024-02-26T04:53:46.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.926+0000 7f111daa3700 1 --2- 172.21.15.156:0/417473700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1118103870 0x7f111810a120 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:46.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.926+0000 7f111daa3700 1 --2- 172.21.15.156:0/417473700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1118101cd0 0x7f11181020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:46.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.926+0000 7f111daa3700 1 --2- 172.21.15.156:0/417473700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1118102ed0 0x7f1118103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:46.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.926+0000 7f111daa3700 1 -- 172.21.15.156:0/417473700 >> 172.21.15.156:0/417473700 conn(0x7f11180fd270 msgr2=0x7f11180ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:53:46.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.926+0000 7f111daa3700 1 -- 172.21.15.156:0/417473700 shutdown_connections 2024-02-26T04:53:46.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.926+0000 7f111daa3700 1 -- 172.21.15.156:0/417473700 wait complete. 2024-02-26T04:53:46.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f111daa3700 1 Processor -- start 2024-02-26T04:53:46.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f111daa3700 1 -- start start 2024-02-26T04:53:46.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f111daa3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1118101cd0 0x7f1118197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:53:46.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f111daa3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1118102ed0 0x7f11181981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:53:46.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f111daa3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1118103870 0x7f111819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:53:46.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f111daa3700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111819f7e0 con 0x7f1118102ed0 2024-02-26T04:53:46.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f111daa3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111819f920 con 0x7f1118103870 2024-02-26T04:53:46.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f111daa3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111819fae0 con 0x7f1118101cd0 2024-02-26T04:53:46.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f1117fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1118103870 0x7f111819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:53:46.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f1117fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1118103870 0x7f111819f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49214/0 (socket says 172.21.15.156:49214) 2024-02-26T04:53:46.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f1117fff700 1 -- 172.21.15.156:0/1757556523 learned_addr learned my addr 172.21.15.156:0/1757556523 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-02-26T04:53:46.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f11177fe700 1 --2- 172.21.15.156:0/1757556523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1118101cd0 0x7f1118197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:53:46.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f1116ffd700 1 --2- 172.21.15.156:0/1757556523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1118102ed0 0x7f11181981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:53:46.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f1117fff700 1 -- 172.21.15.156:0/1757556523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1118101cd0 msgr2=0x7f1118197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:53:46.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.927+0000 7f1117fff700 1 --2- 172.21.15.156:0/1757556523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1118101cd0 0x7f1118197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:46.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.928+0000 7f1117fff700 1 -- 172.21.15.156:0/1757556523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1118102ed0 msgr2=0x7f11181981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:53:46.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.928+0000 7f1117fff700 1 --2- 172.21.15.156:0/1757556523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1118102ed0 0x7f11181981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:53:46.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.928+0000 7f1117fff700 1 -- 172.21.15.156:0/1757556523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1110009d00 con 0x7f1118103870 2024-02-26T04:53:46.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.928+0000 7f1117fff700 1 --2- 172.21.15.156:0/1757556523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1118103870 0x7f111819f210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1110007c90 tx=0x7f1110009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:53:46.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.928+0000 7f1114ff9700 1 -- 172.21.15.156:0/1757556523 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111000e040 con 0x7f1118103870 2024-02-26T04:53:46.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.928+0000 7f111daa3700 1 -- 172.21.15.156:0/1757556523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111819fd60 con 0x7f1118103870 2024-02-26T04:53:46.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.928+0000 7f1114ff9700 1 -- 172.21.15.156:0/1757556523 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1110004e60 con 0x7f1118103870 2024-02-26T04:53:46.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.928+0000 7f1114ff9700 1 -- 172.21.15.156:0/1757556523 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111001dab0 con 0x7f1118103870 2024-02-26T04:53:46.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.928+0000 7f111daa3700 1 -- 172.21.15.156:0/1757556523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11181a0180 con 0x7f1118103870 2024-02-26T04:53:46.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.929+0000 7f111daa3700 1 -- 172.21.15.156:0/1757556523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1118108fc0 con 0x7f1118103870 2024-02-26T04:53:46.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.930+0000 7f1114ff9700 1 -- 172.21.15.156:0/1757556523 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f11100057c0 con 0x7f1118103870 2024-02-26T04:53:46.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.931+0000 7f1114ff9700 1 --2- 172.21.15.156:0/1757556523 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f11000a2150 0x7f11000a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:53:46.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.931+0000 7f1114ff9700 1 -- 172.21.15.156:0/1757556523 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f11100cefa0 con 0x7f1118103870 2024-02-26T04:53:46.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.931+0000 7f11177fe700 1 --2- 172.21.15.156:0/1757556523 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f11000a2150 0x7f11000a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:53:46.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.931+0000 7f11177fe700 1 --2- 172.21.15.156:0/1757556523 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f11000a2150 0x7f11000a4610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1118198b40 tx=0x7f110800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:53:46.936 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:46.934+0000 7f1114ff9700 1 -- 172.21.15.156:0/1757556523 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1110098800 con 0x7f1118103870 2024-02-26T04:53:47.135 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:53:47.133+0000 7f111daa3700 1 -- 172.21.15.156:0/1757556523 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f111802d7d0 con 0x7f11000a2150 2024-02-26T04:53:47.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:47 smithi156 ceph-mon[35805]: Detected new or changed devices on smithi060 2024-02-26T04:53:47.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:47 smithi156 ceph-mon[35805]: Adjusting osd_memory_target on smithi060 to 4273M 2024-02-26T04:53:47.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:47 smithi156 ceph-mon[35805]: osdmap e68: 10 total, 10 up, 10 in 2024-02-26T04:53:47.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:47 smithi156 ceph-mon[35805]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:47.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:47 smithi156 ceph-mon[35805]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:53:47.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:53:47.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:53:47.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:47.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:47 smithi060 ceph-mon[32382]: Detected new or changed devices on smithi060 2024-02-26T04:53:47.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:47 smithi060 ceph-mon[32382]: Adjusting osd_memory_target on smithi060 to 4273M 2024-02-26T04:53:47.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:47 smithi060 ceph-mon[32382]: osdmap e68: 10 total, 10 up, 10 in 2024-02-26T04:53:47.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:47 smithi060 ceph-mon[32382]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:47.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:47 smithi060 ceph-mon[32382]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:53:47.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:53:47.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:53:47.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:47.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:47 smithi191 ceph-mon[36352]: Detected new or changed devices on smithi060 2024-02-26T04:53:47.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:47 smithi191 ceph-mon[36352]: Adjusting osd_memory_target on smithi060 to 4273M 2024-02-26T04:53:47.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:47 smithi191 ceph-mon[36352]: osdmap e68: 10 total, 10 up, 10 in 2024-02-26T04:53:47.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:47 smithi191 ceph-mon[36352]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:47.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:47 smithi191 ceph-mon[36352]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:53:47.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:53:47.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:53:47.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:49.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:49 smithi156 ceph-mon[35805]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:49.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:49 smithi060 ceph-mon[32382]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:49.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:49 smithi191 ceph-mon[36352]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:50.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:50 smithi156 ceph-mon[35805]: from='client.? 172.21.15.156:0/3083732720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eda157d9-fe5f-46ad-8a82-0d60cf51c438"}]: dispatch 2024-02-26T04:53:50.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:50 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eda157d9-fe5f-46ad-8a82-0d60cf51c438"}]: dispatch 2024-02-26T04:53:50.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:50 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eda157d9-fe5f-46ad-8a82-0d60cf51c438"}]': finished 2024-02-26T04:53:50.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:50 smithi156 ceph-mon[35805]: osdmap e69: 11 total, 10 up, 11 in 2024-02-26T04:53:50.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:50 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:53:50.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:50 smithi060 ceph-mon[32382]: from='client.? 172.21.15.156:0/3083732720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eda157d9-fe5f-46ad-8a82-0d60cf51c438"}]: dispatch 2024-02-26T04:53:50.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:50 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eda157d9-fe5f-46ad-8a82-0d60cf51c438"}]: dispatch 2024-02-26T04:53:50.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:50 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eda157d9-fe5f-46ad-8a82-0d60cf51c438"}]': finished 2024-02-26T04:53:50.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:50 smithi060 ceph-mon[32382]: osdmap e69: 11 total, 10 up, 11 in 2024-02-26T04:53:50.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:50 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:53:50.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:50 smithi191 ceph-mon[36352]: from='client.? 172.21.15.156:0/3083732720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eda157d9-fe5f-46ad-8a82-0d60cf51c438"}]: dispatch 2024-02-26T04:53:50.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:50 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eda157d9-fe5f-46ad-8a82-0d60cf51c438"}]: dispatch 2024-02-26T04:53:50.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:50 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eda157d9-fe5f-46ad-8a82-0d60cf51c438"}]': finished 2024-02-26T04:53:50.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:50 smithi191 ceph-mon[36352]: osdmap e69: 11 total, 10 up, 11 in 2024-02-26T04:53:50.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:50 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:53:51.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:51 smithi156 ceph-mon[35805]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:51.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:51 smithi156 ceph-mon[35805]: from='client.? 172.21.15.156:0/709105566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:53:51.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:51 smithi060 ceph-mon[32382]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:51.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:51 smithi060 ceph-mon[32382]: from='client.? 172.21.15.156:0/709105566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:53:51.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:51 smithi191 ceph-mon[36352]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:51.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:51 smithi191 ceph-mon[36352]: from='client.? 172.21.15.156:0/709105566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:53:53.615 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:53 smithi156 ceph-mon[35805]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:53.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:53 smithi060 ceph-mon[32382]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:53.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:53 smithi191 ceph-mon[36352]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:55.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:55 smithi156 ceph-mon[35805]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:55.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:55 smithi060 ceph-mon[32382]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:55.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:55 smithi191 ceph-mon[36352]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:57.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:57 smithi156 ceph-mon[35805]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:57.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:57 smithi060 ceph-mon[32382]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:57.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:57 smithi191 ceph-mon[36352]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:58.660 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:58 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-26T04:53:58.660 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:58 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:58.661 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:58 smithi191 ceph-mon[36352]: Deploying daemon osd.10 on smithi156 2024-02-26T04:53:58.680 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:58 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-26T04:53:58.680 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:58 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:58.680 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:58 smithi156 ceph-mon[35805]: Deploying daemon osd.10 on smithi156 2024-02-26T04:53:58.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-26T04:53:58.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:58 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:53:58.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:58 smithi060 ceph-mon[32382]: Deploying daemon osd.10 on smithi156 2024-02-26T04:53:59.596 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:53:59 smithi156 ceph-mon[35805]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:59.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:53:59 smithi060 ceph-mon[32382]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:53:59.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:53:59 smithi191 ceph-mon[36352]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:01.554 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:01 smithi156 ceph-mon[35805]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:01.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:01 smithi060 ceph-mon[32382]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:01.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:01 smithi191 ceph-mon[36352]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:04.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:04 smithi156 ceph-mon[35805]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:04.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:04 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:04.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:04 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:04.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:04 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:04.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:04 smithi060 ceph-mon[32382]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:04.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:04 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:04.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:04 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:04.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:04 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:04.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:04 smithi191 ceph-mon[36352]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:04.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:04 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:04.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:04 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:04.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:04 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:06.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:06 smithi156 ceph-mon[35805]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:06.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:06 smithi060 ceph-mon[32382]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:06.753 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:54:06.751+0000 7f1114ff9700 1 -- 172.21.15.156:0/1757556523 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f111802d7d0 con 0x7f11000a2150 2024-02-26T04:54:06.754 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 10 on host 'smithi156' 2024-02-26T04:54:06.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:54:06.754+0000 7f111daa3700 1 -- 172.21.15.156:0/1757556523 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f11000a2150 msgr2=0x7f11000a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:06.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:54:06.754+0000 7f111daa3700 1 --2- 172.21.15.156:0/1757556523 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f11000a2150 0x7f11000a4610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1118198b40 tx=0x7f110800a3b0 comp rx=0 tx=0).stop 2024-02-26T04:54:06.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:54:06.754+0000 7f111daa3700 1 -- 172.21.15.156:0/1757556523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1118103870 msgr2=0x7f111819f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:06.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:54:06.754+0000 7f111daa3700 1 --2- 172.21.15.156:0/1757556523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1118103870 0x7f111819f210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1110007c90 tx=0x7f1110009380 comp rx=0 tx=0).stop 2024-02-26T04:54:06.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:54:06.754+0000 7f111daa3700 1 -- 172.21.15.156:0/1757556523 shutdown_connections 2024-02-26T04:54:06.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:54:06.754+0000 7f111daa3700 1 --2- 172.21.15.156:0/1757556523 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f11000a2150 0x7f11000a4610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:06.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:54:06.754+0000 7f111daa3700 1 --2- 172.21.15.156:0/1757556523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1118103870 0x7f111819f210 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:06.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:54:06.754+0000 7f111daa3700 1 --2- 172.21.15.156:0/1757556523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1118101cd0 0x7f1118197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:06.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:54:06.754+0000 7f111daa3700 1 --2- 172.21.15.156:0/1757556523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1118102ed0 0x7f11181981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:06.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:54:06.754+0000 7f111daa3700 1 -- 172.21.15.156:0/1757556523 >> 172.21.15.156:0/1757556523 conn(0x7f11180fd270 msgr2=0x7f1118106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:06.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:54:06.754+0000 7f111daa3700 1 -- 172.21.15.156:0/1757556523 shutdown_connections 2024-02-26T04:54:06.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-02-26T04:54:06.754+0000 7f111daa3700 1 -- 172.21.15.156:0/1757556523 wait complete. 2024-02-26T04:54:06.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:06 smithi191 ceph-mon[36352]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:07.416 DEBUG:teuthology.orchestra.run.smithi156:osd.10> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.10.service 2024-02-26T04:54:07.418 INFO:tasks.cephadm:Deploying osd.11 on smithi191 with /dev/vg_nvme/lv_1... 2024-02-26T04:54:07.418 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-26T04:54:07.663 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T04:54:08.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:07 smithi191 ceph-mon[36352]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:08.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:08.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:08.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:07 smithi191 ceph-mon[36352]: from='osd.10 [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T04:54:08.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:07 smithi191 ceph-mon[36352]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T04:54:08.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:08.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:08.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:07 smithi156 ceph-mon[35805]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:08.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:08.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:08.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:07 smithi156 ceph-mon[35805]: from='osd.10 [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T04:54:08.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:07 smithi156 ceph-mon[35805]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T04:54:08.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:08.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:08.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:07 smithi060 ceph-mon[32382]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:08.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:08.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:08.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:07 smithi060 ceph-mon[32382]: from='osd.10 [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T04:54:08.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:07 smithi060 ceph-mon[32382]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-26T04:54:08.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:08.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:08.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:08 smithi156 ceph-mon[35805]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-26T04:54:08.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:08 smithi156 ceph-mon[35805]: osdmap e70: 11 total, 10 up, 11 in 2024-02-26T04:54:08.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:08 smithi156 ceph-mon[35805]: from='osd.10 [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:54:08.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:08 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:54:08.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:08 smithi156 ceph-mon[35805]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:54:08.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 04:54:08 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T04:54:08.756+0000 7f8fe277e700 -1 osd.10 0 waiting for initial osdmap 2024-02-26T04:54:08.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 04:54:08 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T04:54:08.769+0000 7f8fde571700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T04:54:09.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:08 smithi191 ceph-mon[36352]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-26T04:54:09.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:08 smithi191 ceph-mon[36352]: osdmap e70: 11 total, 10 up, 11 in 2024-02-26T04:54:09.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:08 smithi191 ceph-mon[36352]: from='osd.10 [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:54:09.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:54:09.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:08 smithi191 ceph-mon[36352]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:54:09.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:08 smithi060 ceph-mon[32382]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-26T04:54:09.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:08 smithi060 ceph-mon[32382]: osdmap e70: 11 total, 10 up, 11 in 2024-02-26T04:54:09.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:08 smithi060 ceph-mon[32382]: from='osd.10 [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:54:09.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:54:09.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:08 smithi060 ceph-mon[32382]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-02-26T04:54:10.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:09 smithi191 ceph-mon[36352]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:10.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:09 smithi191 ceph-mon[36352]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-26T04:54:10.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:09 smithi191 ceph-mon[36352]: osdmap e71: 11 total, 10 up, 11 in 2024-02-26T04:54:10.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:09 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:54:10.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:09 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:54:10.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:09 smithi191 ceph-mon[36352]: osd.10 [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798] boot 2024-02-26T04:54:10.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:09 smithi191 ceph-mon[36352]: osdmap e72: 11 total, 11 up, 11 in 2024-02-26T04:54:10.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:09 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:54:10.130 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:09 smithi060 ceph-mon[32382]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:10.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:09 smithi060 ceph-mon[32382]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-26T04:54:10.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:09 smithi060 ceph-mon[32382]: osdmap e71: 11 total, 10 up, 11 in 2024-02-26T04:54:10.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:09 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:54:10.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:09 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:54:10.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:09 smithi060 ceph-mon[32382]: osd.10 [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798] boot 2024-02-26T04:54:10.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:09 smithi060 ceph-mon[32382]: osdmap e72: 11 total, 11 up, 11 in 2024-02-26T04:54:10.131 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:09 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:54:10.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:09 smithi156 ceph-mon[35805]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-02-26T04:54:10.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:09 smithi156 ceph-mon[35805]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-02-26T04:54:10.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:09 smithi156 ceph-mon[35805]: osdmap e71: 11 total, 10 up, 11 in 2024-02-26T04:54:10.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:09 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:54:10.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:09 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:54:10.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:09 smithi156 ceph-mon[35805]: osd.10 [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798] boot 2024-02-26T04:54:10.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:09 smithi156 ceph-mon[35805]: osdmap e72: 11 total, 11 up, 11 in 2024-02-26T04:54:10.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:09 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-26T04:54:10.196 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-26T04:54:10.196 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-26T04:54:10.196 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-02-26T04:54:10.197 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-02-26T04:54:10.197 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0164963 s, 636 MB/s 2024-02-26T04:54:10.197 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-02-26T04:54:11.066 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_1 2024-02-26T04:54:11.314 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T04:54:11.418 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:11 smithi060 ceph-mon[32382]: purged_snaps scrub starts 2024-02-26T04:54:11.418 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:11 smithi060 ceph-mon[32382]: purged_snaps scrub ok 2024-02-26T04:54:11.418 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:11 smithi060 ceph-mon[32382]: osdmap e73: 11 total, 11 up, 11 in 2024-02-26T04:54:11.419 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:11 smithi156 ceph-mon[35805]: purged_snaps scrub starts 2024-02-26T04:54:11.419 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:11 smithi156 ceph-mon[35805]: purged_snaps scrub ok 2024-02-26T04:54:11.419 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:11 smithi156 ceph-mon[35805]: osdmap e73: 11 total, 11 up, 11 in 2024-02-26T04:54:11.436 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:11 smithi191 ceph-mon[36352]: purged_snaps scrub starts 2024-02-26T04:54:11.436 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:11 smithi191 ceph-mon[36352]: purged_snaps scrub ok 2024-02-26T04:54:11.436 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:11 smithi191 ceph-mon[36352]: osdmap e73: 11 total, 11 up, 11 in 2024-02-26T04:54:12.390 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:12.390 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: osdmap e74: 11 total, 11 up, 11 in 2024-02-26T04:54:12.390 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:12.390 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:12.390 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:12.390 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:12.390 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:12.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:12.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:12.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:12.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:12.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:12.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:12.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:12.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:12.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:12 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:12.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:12.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: osdmap e74: 11 total, 11 up, 11 in 2024-02-26T04:54:12.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:12.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:12.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:12.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:12.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:12.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:12.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:12.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:12.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:12.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:12.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:12.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:12.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:12.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:12 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:12.463 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:12.463 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: osdmap e74: 11 total, 11 up, 11 in 2024-02-26T04:54:12.463 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:12.463 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:12.464 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:12.464 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:12.464 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:12.464 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:12.464 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:12.464 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:12.464 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:12.464 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:12.464 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:12.464 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:12.464 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:12.464 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:12 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:13.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.202+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/1260268643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c8100e60 msgr2=0x7fc6c81012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:13.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.202+0000 7fc6ce4c3700 1 --2- 172.21.15.191:0/1260268643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c8100e60 0x7fc6c81012e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009990 tx=0x7fc6bc009ca0 comp rx=0 tx=0).stop 2024-02-26T04:54:13.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.202+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/1260268643 shutdown_connections 2024-02-26T04:54:13.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.202+0000 7fc6ce4c3700 1 --2- 172.21.15.191:0/1260268643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6c8101820 0x7fc6c81090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:13.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.202+0000 7fc6ce4c3700 1 --2- 172.21.15.191:0/1260268643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6c8102200 0x7fc6c8102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:13.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.202+0000 7fc6ce4c3700 1 --2- 172.21.15.191:0/1260268643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c8100e60 0x7fc6c81012e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:13.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.202+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/1260268643 >> 172.21.15.191:0/1260268643 conn(0x7fc6c80fb910 msgr2=0x7fc6c80fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:13.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.202+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/1260268643 shutdown_connections 2024-02-26T04:54:13.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.202+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/1260268643 wait complete. 2024-02-26T04:54:13.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.202+0000 7fc6ce4c3700 1 Processor -- start 2024-02-26T04:54:13.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.202+0000 7fc6ce4c3700 1 -- start start 2024-02-26T04:54:13.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6ce4c3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6c8100e60 0x7fc6c8196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:13.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6ce4c3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c8101820 0x7fc6c8197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:13.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6ce4c3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6c8102200 0x7fc6c819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:13.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6ce4c3700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c819e6a0 con 0x7fc6c8100e60 2024-02-26T04:54:13.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6ce4c3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c819e7e0 con 0x7fc6c8102200 2024-02-26T04:54:13.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6ce4c3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c819e970 con 0x7fc6c8101820 2024-02-26T04:54:13.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6c77fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c8101820 0x7fc6c8197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:13.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6c77fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c8101820 0x7fc6c8197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:55174/0 (socket says 172.21.15.191:55174) 2024-02-26T04:54:13.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6c77fe700 1 -- 172.21.15.191:0/3715128732 learned_addr learned my addr 172.21.15.191:0/3715128732 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-26T04:54:13.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6c7fff700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6c8100e60 0x7fc6c8196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:13.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6cca60700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6c8102200 0x7fc6c819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:13.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6c77fe700 1 -- 172.21.15.191:0/3715128732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6c8102200 msgr2=0x7fc6c819e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:13.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6c77fe700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6c8102200 0x7fc6c819e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:13.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6c77fe700 1 -- 172.21.15.191:0/3715128732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6c8100e60 msgr2=0x7fc6c8196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:13.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6c77fe700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6c8100e60 0x7fc6c8196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:13.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6c77fe700 1 -- 172.21.15.191:0/3715128732 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc0095f0 con 0x7fc6c8101820 2024-02-26T04:54:13.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6cca60700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6c8102200 0x7fc6c819e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:54:13.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6c77fe700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c8101820 0x7fc6c8197050 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc000c00 tx=0x7fc6bc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:13.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6c57fa700 1 -- 172.21.15.191:0/3715128732 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc01d070 con 0x7fc6c8101820 2024-02-26T04:54:13.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6c57fa700 1 -- 172.21.15.191:0/3715128732 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6bc00fb40 con 0x7fc6c8101820 2024-02-26T04:54:13.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6c57fa700 1 -- 172.21.15.191:0/3715128732 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc017650 con 0x7fc6c8101820 2024-02-26T04:54:13.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.203+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/3715128732 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c819ebf0 con 0x7fc6c8101820 2024-02-26T04:54:13.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.204+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/3715128732 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c819f060 con 0x7fc6c8101820 2024-02-26T04:54:13.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.205+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/3715128732 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c8107fd0 con 0x7fc6c8101820 2024-02-26T04:54:13.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.206+0000 7fc6c57fa700 1 -- 172.21.15.191:0/3715128732 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc6bc00fcb0 con 0x7fc6c8101820 2024-02-26T04:54:13.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.206+0000 7fc6c57fa700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc6b00a21d0 0x7fc6b00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:13.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.206+0000 7fc6c57fa700 1 -- 172.21.15.191:0/3715128732 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6694+0+0 (secure 0 0 0) 0x7fc6bc0cea50 con 0x7fc6c8101820 2024-02-26T04:54:13.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.206+0000 7fc6c7fff700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc6b00a21d0 0x7fc6b00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:13.211 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.207+0000 7fc6c7fff700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc6b00a21d0 0x7fc6b00a4690 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc6c806a170 tx=0x7fc6b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:13.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.211+0000 7fc6c57fa700 1 -- 172.21.15.191:0/3715128732 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc6bc098220 con 0x7fc6c8101820 2024-02-26T04:54:13.414 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:13.411+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/3715128732 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6c802d8d0 con 0x7fc6b00a21d0 2024-02-26T04:54:13.414 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:13 smithi191 ceph-mon[36352]: Detected new or changed devices on smithi156 2024-02-26T04:54:13.414 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:13 smithi191 ceph-mon[36352]: Adjusting osd_memory_target on smithi156 to 4270M 2024-02-26T04:54:13.414 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:13 smithi191 ceph-mon[36352]: osdmap e75: 11 total, 11 up, 11 in 2024-02-26T04:54:13.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:13 smithi156 ceph-mon[35805]: Detected new or changed devices on smithi156 2024-02-26T04:54:13.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:13 smithi156 ceph-mon[35805]: Adjusting osd_memory_target on smithi156 to 4270M 2024-02-26T04:54:13.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:13 smithi156 ceph-mon[35805]: osdmap e75: 11 total, 11 up, 11 in 2024-02-26T04:54:13.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:13 smithi060 ceph-mon[32382]: Detected new or changed devices on smithi156 2024-02-26T04:54:13.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:13 smithi060 ceph-mon[32382]: Adjusting osd_memory_target on smithi156 to 4270M 2024-02-26T04:54:13.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:13 smithi060 ceph-mon[32382]: osdmap e75: 11 total, 11 up, 11 in 2024-02-26T04:54:14.272 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:14 smithi191 ceph-mon[36352]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 163 KiB/s, 0 objects/s recovering 2024-02-26T04:54:14.272 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:14 smithi191 ceph-mon[36352]: osdmap e76: 11 total, 11 up, 11 in 2024-02-26T04:54:14.272 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:14 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:54:14.273 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:14 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:54:14.273 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:14 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:14.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:14 smithi156 ceph-mon[35805]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 163 KiB/s, 0 objects/s recovering 2024-02-26T04:54:14.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:14 smithi156 ceph-mon[35805]: osdmap e76: 11 total, 11 up, 11 in 2024-02-26T04:54:14.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:14 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:54:14.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:14 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:54:14.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:14 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:14.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:14 smithi060 ceph-mon[32382]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 163 KiB/s, 0 objects/s recovering 2024-02-26T04:54:14.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:14 smithi060 ceph-mon[32382]: osdmap e76: 11 total, 11 up, 11 in 2024-02-26T04:54:14.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:14 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-26T04:54:14.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:14 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-26T04:54:14.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:14 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:15.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:15 smithi156 ceph-mon[35805]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:54:15.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:15 smithi060 ceph-mon[32382]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:54:15.505 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:15 smithi191 ceph-mon[36352]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:54:16.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:16 smithi156 ceph-mon[35805]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-02-26T04:54:16.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:16 smithi060 ceph-mon[32382]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-02-26T04:54:16.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:16 smithi191 ceph-mon[36352]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-02-26T04:54:17.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:17 smithi156 ceph-mon[35805]: from='client.? 172.21.15.191:0/1307188732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ebe4a4-3a19-4074-aa49-bfe2339295b7"}]: dispatch 2024-02-26T04:54:17.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:17 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ebe4a4-3a19-4074-aa49-bfe2339295b7"}]: dispatch 2024-02-26T04:54:17.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:17 smithi156 ceph-mon[35805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80ebe4a4-3a19-4074-aa49-bfe2339295b7"}]': finished 2024-02-26T04:54:17.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:17 smithi156 ceph-mon[35805]: osdmap e77: 12 total, 11 up, 12 in 2024-02-26T04:54:17.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:17 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:17.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:17 smithi156 ceph-mon[35805]: from='client.? 172.21.15.191:0/2002843777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:54:17.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:17 smithi060 ceph-mon[32382]: from='client.? 172.21.15.191:0/1307188732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ebe4a4-3a19-4074-aa49-bfe2339295b7"}]: dispatch 2024-02-26T04:54:17.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:17 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ebe4a4-3a19-4074-aa49-bfe2339295b7"}]: dispatch 2024-02-26T04:54:17.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:17 smithi060 ceph-mon[32382]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80ebe4a4-3a19-4074-aa49-bfe2339295b7"}]': finished 2024-02-26T04:54:17.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:17 smithi060 ceph-mon[32382]: osdmap e77: 12 total, 11 up, 12 in 2024-02-26T04:54:17.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:17.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:17 smithi060 ceph-mon[32382]: from='client.? 172.21.15.191:0/2002843777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:54:17.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:17 smithi191 ceph-mon[36352]: from='client.? 172.21.15.191:0/1307188732' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ebe4a4-3a19-4074-aa49-bfe2339295b7"}]: dispatch 2024-02-26T04:54:17.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:17 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ebe4a4-3a19-4074-aa49-bfe2339295b7"}]: dispatch 2024-02-26T04:54:17.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:17 smithi191 ceph-mon[36352]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80ebe4a4-3a19-4074-aa49-bfe2339295b7"}]': finished 2024-02-26T04:54:17.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:17 smithi191 ceph-mon[36352]: osdmap e77: 12 total, 11 up, 12 in 2024-02-26T04:54:17.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:17.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:17 smithi191 ceph-mon[36352]: from='client.? 172.21.15.191:0/2002843777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-26T04:54:18.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:18 smithi156 ceph-mon[35805]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T04:54:18.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:18 smithi060 ceph-mon[32382]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T04:54:18.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:18 smithi191 ceph-mon[36352]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-26T04:54:20.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:20 smithi191 ceph-mon[36352]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-26T04:54:20.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:20 smithi156 ceph-mon[35805]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-26T04:54:20.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:20 smithi060 ceph-mon[32382]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-26T04:54:22.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:22 smithi156 ceph-mon[35805]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:22.461 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:22 smithi060 ceph-mon[32382]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:22.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:22 smithi191 ceph-mon[36352]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:24.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:24 smithi156 ceph-mon[35805]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:24.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:24 smithi060 ceph-mon[32382]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:24.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:24 smithi191 ceph-mon[36352]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:25.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:25 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-26T04:54:25.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:25 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:25.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:25 smithi156 ceph-mon[35805]: Deploying daemon osd.11 on smithi191 2024-02-26T04:54:25.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:25 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-26T04:54:25.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:25 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:25.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:25 smithi060 ceph-mon[32382]: Deploying daemon osd.11 on smithi191 2024-02-26T04:54:25.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:25 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-26T04:54:25.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:25 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:25.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:25 smithi191 ceph-mon[36352]: Deploying daemon osd.11 on smithi191 2024-02-26T04:54:26.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:26 smithi156 ceph-mon[35805]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:26.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:26 smithi060 ceph-mon[32382]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:26.487 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:26 smithi191 ceph-mon[36352]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:28.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:28 smithi156 ceph-mon[35805]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:28.435 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:28 smithi191 ceph-mon[36352]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:28.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:28 smithi060 ceph-mon[32382]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:30.355 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:30 smithi191 ceph-mon[36352]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:30.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:30 smithi156 ceph-mon[35805]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:30.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:30 smithi060 ceph-mon[32382]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:31.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:31 smithi156 ceph-mon[35805]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:31.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:31 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:31.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:31 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:31.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:31 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:31.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:31 smithi060 ceph-mon[32382]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:31.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:31.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:31.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:31 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:31.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:31 smithi191 ceph-mon[36352]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:31.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:31 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:31.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:31 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:31.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:31 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:33.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:33 smithi191 ceph-mon[36352]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:33.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:33 smithi156 ceph-mon[35805]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:33.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:33 smithi060 ceph-mon[32382]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:34.387 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 11 on host 'smithi191' 2024-02-26T04:54:34.387 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:34.384+0000 7fc6c57fa700 1 -- 172.21.15.191:0/3715128732 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc6c802d8d0 con 0x7fc6b00a21d0 2024-02-26T04:54:34.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:34.386+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/3715128732 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc6b00a21d0 msgr2=0x7fc6b00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:34.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:34.386+0000 7fc6ce4c3700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc6b00a21d0 0x7fc6b00a4690 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc6c806a170 tx=0x7fc6b800b410 comp rx=0 tx=0).stop 2024-02-26T04:54:34.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:34.386+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/3715128732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c8101820 msgr2=0x7fc6c8197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:34.389 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:34.386+0000 7fc6ce4c3700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c8101820 0x7fc6c8197050 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc000c00 tx=0x7fc6bc00f690 comp rx=0 tx=0).stop 2024-02-26T04:54:34.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:34.386+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/3715128732 shutdown_connections 2024-02-26T04:54:34.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:34.386+0000 7fc6ce4c3700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc6b00a21d0 0x7fc6b00a4690 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:34.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:34.386+0000 7fc6ce4c3700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6c8102200 0x7fc6c819e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:34.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:34.386+0000 7fc6ce4c3700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6c8100e60 0x7fc6c8196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:34.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:34.386+0000 7fc6ce4c3700 1 --2- 172.21.15.191:0/3715128732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6c8101820 0x7fc6c8197050 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:34.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:34.386+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/3715128732 >> 172.21.15.191:0/3715128732 conn(0x7fc6c80fb910 msgr2=0x7fc6c8105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:34.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:34.387+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/3715128732 shutdown_connections 2024-02-26T04:54:34.390 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:34.387+0000 7fc6ce4c3700 1 -- 172.21.15.191:0/3715128732 wait complete. 2024-02-26T04:54:34.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:34 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:34.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:34 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:34.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:34 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:34.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:34 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:34.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:34 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:34.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:34 smithi156 ceph-mon[35805]: from='osd.11 [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T04:54:34.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:34 smithi156 ceph-mon[35805]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T04:54:34.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:34 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:34.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:34 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:34.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:34 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:34.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:34 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:34.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:34 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:34.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:34 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:34.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:34 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:34.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:34 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:34.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:34 smithi060 ceph-mon[32382]: from='osd.11 [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T04:54:34.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:34 smithi060 ceph-mon[32382]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T04:54:34.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:34 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:34.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:34 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:34.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:34 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:35.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:34 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:35.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:34 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:35.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:34 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:35.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:34 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:35.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:34 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:35.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:34 smithi191 ceph-mon[36352]: from='osd.11 [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T04:54:35.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:34 smithi191 ceph-mon[36352]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-26T04:54:35.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:34 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:35.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:34 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:35.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:34 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:35.114 DEBUG:teuthology.orchestra.run.smithi191:osd.11> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.11.service 2024-02-26T04:54:35.116 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-02-26T04:54:35.117 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd stat -f json 2024-02-26T04:54:35.370 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:54:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:35 smithi191 ceph-mon[36352]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:35 smithi191 ceph-mon[36352]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-26T04:54:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:35 smithi191 ceph-mon[36352]: osdmap e78: 12 total, 11 up, 12 in 2024-02-26T04:54:35.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:35 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:35.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:35 smithi191 ceph-mon[36352]: from='osd.11 [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:54:35.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:35 smithi191 ceph-mon[36352]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:54:35.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 04:54:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T04:54:35.622+0000 7f8ab57eb700 -1 osd.11 0 waiting for initial osdmap 2024-02-26T04:54:35.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 04:54:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T04:54:35.634+0000 7f8ab15de700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T04:54:35.905 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:35 smithi060 ceph-mon[32382]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:35.905 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:35 smithi060 ceph-mon[32382]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-26T04:54:35.905 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:35 smithi060 ceph-mon[32382]: osdmap e78: 12 total, 11 up, 12 in 2024-02-26T04:54:35.905 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:35 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:35.905 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:35 smithi060 ceph-mon[32382]: from='osd.11 [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:54:35.905 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:35 smithi060 ceph-mon[32382]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:54:35.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:35 smithi156 ceph-mon[35805]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:35.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:35 smithi156 ceph-mon[35805]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-26T04:54:35.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:35 smithi156 ceph-mon[35805]: osdmap e78: 12 total, 11 up, 12 in 2024-02-26T04:54:35.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:35 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:35.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:35 smithi156 ceph-mon[35805]: from='osd.11 [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:54:35.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:35 smithi156 ceph-mon[35805]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-26T04:54:36.624 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:36 smithi191 ceph-mon[36352]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-26T04:54:36.624 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:36 smithi191 ceph-mon[36352]: osdmap e79: 12 total, 11 up, 12 in 2024-02-26T04:54:36.624 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:36 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:36.625 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:36 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:36 smithi060 ceph-mon[32382]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-26T04:54:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:36 smithi060 ceph-mon[32382]: osdmap e79: 12 total, 11 up, 12 in 2024-02-26T04:54:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:36 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:36 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:36.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:36 smithi156 ceph-mon[35805]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-26T04:54:36.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:36 smithi156 ceph-mon[35805]: osdmap e79: 12 total, 11 up, 12 in 2024-02-26T04:54:36.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:36 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:36.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:36 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.314+0000 7fd6b512a700 1 -- 172.21.15.60:0/3924128486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b0101da0 msgr2=0x7fd6b0102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.314+0000 7fd6b512a700 1 --2- 172.21.15.60:0/3924128486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b0101da0 0x7fd6b0102200 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd6980099c0 tx=0x7fd698009cd0 comp rx=0 tx=0).stop 2024-02-26T04:54:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.314+0000 7fd6b512a700 1 -- 172.21.15.60:0/3924128486 shutdown_connections 2024-02-26T04:54:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.314+0000 7fd6b512a700 1 --2- 172.21.15.60:0/3924128486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b0102740 0x7fd6b0108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.314+0000 7fd6b512a700 1 --2- 172.21.15.60:0/3924128486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b0100ba0 0x7fd6b0100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.314+0000 7fd6b512a700 1 --2- 172.21.15.60:0/3924128486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b0101da0 0x7fd6b0102200 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.314+0000 7fd6b512a700 1 -- 172.21.15.60:0/3924128486 >> 172.21.15.60:0/3924128486 conn(0x7fd6b00fc140 msgr2=0x7fd6b00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.314+0000 7fd6b512a700 1 -- 172.21.15.60:0/3924128486 shutdown_connections 2024-02-26T04:54:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.314+0000 7fd6b512a700 1 -- 172.21.15.60:0/3924128486 wait complete. 2024-02-26T04:54:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.314+0000 7fd6b512a700 1 Processor -- start 2024-02-26T04:54:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6b512a700 1 -- start start 2024-02-26T04:54:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6b512a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b0100ba0 0x7fd6b0199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6b512a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b0101da0 0x7fd6b019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6b512a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b0102740 0x7fd6b01a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6b512a700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b01a19c0 con 0x7fd6b0101da0 2024-02-26T04:54:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6b512a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b01a1b00 con 0x7fd6b0102740 2024-02-26T04:54:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6b512a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b01a1c90 con 0x7fd6b0100ba0 2024-02-26T04:54:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6ae59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b0101da0 0x7fd6b019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6aed9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b0100ba0 0x7fd6b0199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6ae59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b0101da0 0x7fd6b019a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59770/0 (socket says 172.21.15.60:59770) 2024-02-26T04:54:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6ae59c700 1 -- 172.21.15.60:0/76594520 learned_addr learned my addr 172.21.15.60:0/76594520 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:54:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6af59e700 1 --2- 172.21.15.60:0/76594520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b0102740 0x7fd6b01a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6ae59c700 1 -- 172.21.15.60:0/76594520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b0100ba0 msgr2=0x7fd6b0199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6ae59c700 1 --2- 172.21.15.60:0/76594520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b0100ba0 0x7fd6b0199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6ae59c700 1 -- 172.21.15.60:0/76594520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b0102740 msgr2=0x7fd6b01a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6ae59c700 1 --2- 172.21.15.60:0/76594520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b0102740 0x7fd6b01a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6ae59c700 1 -- 172.21.15.60:0/76594520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6980095f0 con 0x7fd6b0101da0 2024-02-26T04:54:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.315+0000 7fd6af59e700 1 --2- 172.21.15.60:0/76594520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b0102740 0x7fd6b01a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:54:37.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.316+0000 7fd6ae59c700 1 --2- 172.21.15.60:0/76594520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b0101da0 0x7fd6b019a3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd69800b580 tx=0x7fd698009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:37.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.316+0000 7fd6a7fff700 1 -- 172.21.15.60:0/76594520 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69801d070 con 0x7fd6b0101da0 2024-02-26T04:54:37.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.316+0000 7fd6a7fff700 1 -- 172.21.15.60:0/76594520 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd698022470 con 0x7fd6b0101da0 2024-02-26T04:54:37.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.316+0000 7fd6b512a700 1 -- 172.21.15.60:0/76594520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b01a1f10 con 0x7fd6b0101da0 2024-02-26T04:54:37.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.316+0000 7fd6a7fff700 1 -- 172.21.15.60:0/76594520 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698017650 con 0x7fd6b0101da0 2024-02-26T04:54:37.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.316+0000 7fd6b512a700 1 -- 172.21.15.60:0/76594520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b01a22c0 con 0x7fd6b0101da0 2024-02-26T04:54:37.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.317+0000 7fd6b512a700 1 -- 172.21.15.60:0/76594520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b0068930 con 0x7fd6b0101da0 2024-02-26T04:54:37.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.318+0000 7fd6a7fff700 1 -- 172.21.15.60:0/76594520 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd6980225e0 con 0x7fd6b0101da0 2024-02-26T04:54:37.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.318+0000 7fd6a7fff700 1 --2- 172.21.15.60:0/76594520 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd69c0a2150 0x7fd69c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:37.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.319+0000 7fd6aed9d700 1 --2- 172.21.15.60:0/76594520 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd69c0a2150 0x7fd69c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:37.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.319+0000 7fd6a7fff700 1 -- 172.21.15.60:0/76594520 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd6980ce8c0 con 0x7fd6b0101da0 2024-02-26T04:54:37.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.319+0000 7fd6aed9d700 1 --2- 172.21.15.60:0/76594520 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd69c0a2150 0x7fd69c0a4610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0101c00 tx=0x7fd6a0007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:37.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.322+0000 7fd6a7fff700 1 -- 172.21.15.60:0/76594520 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd69805dd20 con 0x7fd6b0101da0 2024-02-26T04:54:37.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.525+0000 7fd6b512a700 1 -- 172.21.15.60:0/76594520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fd6b010c730 con 0x7fd6b0101da0 2024-02-26T04:54:37.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.525+0000 7fd6a7fff700 1 -- 172.21.15.60:0/76594520 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7fd698097dc0 con 0x7fd6b0101da0 2024-02-26T04:54:37.528 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:54:37.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.527+0000 7fd6b512a700 1 -- 172.21.15.60:0/76594520 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd69c0a2150 msgr2=0x7fd69c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:37.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.527+0000 7fd6b512a700 1 --2- 172.21.15.60:0/76594520 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd69c0a2150 0x7fd69c0a4610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0101c00 tx=0x7fd6a0007400 comp rx=0 tx=0).stop 2024-02-26T04:54:37.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.527+0000 7fd6b512a700 1 -- 172.21.15.60:0/76594520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b0101da0 msgr2=0x7fd6b019a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:37.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.527+0000 7fd6b512a700 1 --2- 172.21.15.60:0/76594520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b0101da0 0x7fd6b019a3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd69800b580 tx=0x7fd698009580 comp rx=0 tx=0).stop 2024-02-26T04:54:37.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.527+0000 7fd6b512a700 1 -- 172.21.15.60:0/76594520 shutdown_connections 2024-02-26T04:54:37.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.528+0000 7fd6b512a700 1 --2- 172.21.15.60:0/76594520 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd69c0a2150 0x7fd69c0a4610 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:37.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.528+0000 7fd6b512a700 1 --2- 172.21.15.60:0/76594520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6b0102740 0x7fd6b01a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:37.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.528+0000 7fd6b512a700 1 --2- 172.21.15.60:0/76594520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b0100ba0 0x7fd6b0199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:37.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.528+0000 7fd6b512a700 1 --2- 172.21.15.60:0/76594520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b0101da0 0x7fd6b019a3c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:37.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.528+0000 7fd6b512a700 1 -- 172.21.15.60:0/76594520 >> 172.21.15.60:0/76594520 conn(0x7fd6b00fc140 msgr2=0x7fd6b00fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:37.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.528+0000 7fd6b512a700 1 -- 172.21.15.60:0/76594520 shutdown_connections 2024-02-26T04:54:37.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:37.528+0000 7fd6b512a700 1 -- 172.21.15.60:0/76594520 wait complete. 2024-02-26T04:54:37.754 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:37 smithi060 ceph-mon[32382]: purged_snaps scrub starts 2024-02-26T04:54:37.754 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:37 smithi060 ceph-mon[32382]: purged_snaps scrub ok 2024-02-26T04:54:37.754 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:37 smithi060 ceph-mon[32382]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:37.754 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:37 smithi060 ceph-mon[32382]: osd.11 [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354] boot 2024-02-26T04:54:37.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:37 smithi060 ceph-mon[32382]: osdmap e80: 12 total, 12 up, 12 in 2024-02-26T04:54:37.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:37 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:37.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:37 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/76594520' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T04:54:37.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:37 smithi191 ceph-mon[36352]: purged_snaps scrub starts 2024-02-26T04:54:37.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:37 smithi191 ceph-mon[36352]: purged_snaps scrub ok 2024-02-26T04:54:37.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:37 smithi191 ceph-mon[36352]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:37.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:37 smithi191 ceph-mon[36352]: osd.11 [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354] boot 2024-02-26T04:54:37.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:37 smithi191 ceph-mon[36352]: osdmap e80: 12 total, 12 up, 12 in 2024-02-26T04:54:37.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:37 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:37.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:37 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/76594520' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T04:54:37.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:37 smithi156 ceph-mon[35805]: purged_snaps scrub starts 2024-02-26T04:54:37.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:37 smithi156 ceph-mon[35805]: purged_snaps scrub ok 2024-02-26T04:54:37.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:37 smithi156 ceph-mon[35805]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-02-26T04:54:37.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:37 smithi156 ceph-mon[35805]: osd.11 [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354] boot 2024-02-26T04:54:37.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:37 smithi156 ceph-mon[35805]: osdmap e80: 12 total, 12 up, 12 in 2024-02-26T04:54:37.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:37 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-26T04:54:37.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:37 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/76594520' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-26T04:54:38.302 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1708923276,"num_in_osds":12,"osd_in_since":1708923256,"num_remapped_pgs":0} 2024-02-26T04:54:38.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-26T04:54:38.548 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:54:38.926 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:38 smithi060 ceph-mon[32382]: osdmap e81: 12 total, 12 up, 12 in 2024-02-26T04:54:38.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:38 smithi060 ceph-mon[32382]: Detected new or changed devices on smithi191 2024-02-26T04:54:38.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:38.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:38.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:38.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:38.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:38.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:38.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:38 smithi060 ceph-mon[32382]: Adjusting osd_memory_target on smithi191 to 4270M 2024-02-26T04:54:38.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:38.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:38.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:38.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:38 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:38.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:38 smithi156 ceph-mon[35805]: osdmap e81: 12 total, 12 up, 12 in 2024-02-26T04:54:38.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:38 smithi156 ceph-mon[35805]: Detected new or changed devices on smithi191 2024-02-26T04:54:38.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:38.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:38.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:38.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:38.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:38.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:38.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:38 smithi156 ceph-mon[35805]: Adjusting osd_memory_target on smithi191 to 4270M 2024-02-26T04:54:38.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:38.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:38.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:38.929 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:38 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:39.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:38 smithi191 ceph-mon[36352]: osdmap e81: 12 total, 12 up, 12 in 2024-02-26T04:54:39.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:38 smithi191 ceph-mon[36352]: Detected new or changed devices on smithi191 2024-02-26T04:54:39.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:39.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:39.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:39.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:39.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:39.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-26T04:54:39.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:38 smithi191 ceph-mon[36352]: Adjusting osd_memory_target on smithi191 to 4270M 2024-02-26T04:54:39.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:39.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:39.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:39.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:38 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:40.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:40 smithi060 ceph-mon[32382]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.371+0000 7f1232a5b700 1 -- 172.21.15.60:0/483546907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f122c101cc0 msgr2=0x7f122c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.371+0000 7f1232a5b700 1 --2- 172.21.15.60:0/483546907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f122c101cc0 0x7f122c102140 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f12200099c0 tx=0x7f1220009cd0 comp rx=0 tx=0).stop 2024-02-26T04:54:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.371+0000 7f1232a5b700 1 -- 172.21.15.60:0/483546907 shutdown_connections 2024-02-26T04:54:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.371+0000 7f1232a5b700 1 --2- 172.21.15.60:0/483546907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f122c102680 0x7f122c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.371+0000 7f1232a5b700 1 --2- 172.21.15.60:0/483546907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c107ca0 0x7f122c108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.371+0000 7f1232a5b700 1 --2- 172.21.15.60:0/483546907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f122c101cc0 0x7f122c102140 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.371+0000 7f1232a5b700 1 -- 172.21.15.60:0/483546907 >> 172.21.15.60:0/483546907 conn(0x7f122c078da0 msgr2=0x7f122c0791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.371+0000 7f1232a5b700 1 -- 172.21.15.60:0/483546907 shutdown_connections 2024-02-26T04:54:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.371+0000 7f1232a5b700 1 -- 172.21.15.60:0/483546907 wait complete. 2024-02-26T04:54:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.372+0000 7f1232a5b700 1 Processor -- start 2024-02-26T04:54:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.372+0000 7f1232a5b700 1 -- start start 2024-02-26T04:54:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.372+0000 7f1232a5b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f122c101cc0 0x7f122c073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.372+0000 7f1232a5b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f122c102680 0x7f122c0744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.372+0000 7f1232a5b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c107ca0 0x7f122c0749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.372+0000 7f1232a5b700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c1a65c0 con 0x7f122c101cc0 2024-02-26T04:54:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.372+0000 7f1232a5b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c1a6730 con 0x7f122c107ca0 2024-02-26T04:54:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.372+0000 7f1232a5b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c1a68f0 con 0x7f122c102680 2024-02-26T04:54:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.372+0000 7f122bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f122c101cc0 0x7f122c073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.372+0000 7f122bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f122c101cc0 0x7f122c073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55676/0 (socket says 172.21.15.60:55676) 2024-02-26T04:54:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.372+0000 7f122bfff700 1 -- 172.21.15.60:0/4027852600 learned_addr learned my addr 172.21.15.60:0/4027852600 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:54:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.372+0000 7f122b7fe700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f122c102680 0x7f122c0744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.373+0000 7f1230ff8700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c107ca0 0x7f122c0749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.373+0000 7f122bfff700 1 -- 172.21.15.60:0/4027852600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f122c102680 msgr2=0x7f122c0744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.373+0000 7f122bfff700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f122c102680 0x7f122c0744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.373+0000 7f122bfff700 1 -- 172.21.15.60:0/4027852600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c107ca0 msgr2=0x7f122c0749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.373+0000 7f122bfff700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c107ca0 0x7f122c0749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.373+0000 7f122bfff700 1 -- 172.21.15.60:0/4027852600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12200095f0 con 0x7f122c101cc0 2024-02-26T04:54:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.373+0000 7f1230ff8700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c107ca0 0x7f122c0749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:54:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.373+0000 7f122bfff700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f122c101cc0 0x7f122c073f70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f122400ef10 tx=0x7f122400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.373+0000 7f12297fa700 1 -- 172.21.15.60:0/4027852600 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122400ca70 con 0x7f122c101cc0 2024-02-26T04:54:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.373+0000 7f12297fa700 1 -- 172.21.15.60:0/4027852600 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f122400f040 con 0x7f122c101cc0 2024-02-26T04:54:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.373+0000 7f1232a5b700 1 -- 172.21.15.60:0/4027852600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122c1a6b70 con 0x7f122c101cc0 2024-02-26T04:54:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.373+0000 7f12297fa700 1 -- 172.21.15.60:0/4027852600 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12240146e0 con 0x7f122c101cc0 2024-02-26T04:54:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.373+0000 7f1232a5b700 1 -- 172.21.15.60:0/4027852600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c1a7090 con 0x7f122c101cc0 2024-02-26T04:54:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.374+0000 7f1232a5b700 1 -- 172.21.15.60:0/4027852600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122c10a180 con 0x7f122c101cc0 2024-02-26T04:54:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.375+0000 7f12297fa700 1 -- 172.21.15.60:0/4027852600 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1224007500 con 0x7f122c101cc0 2024-02-26T04:54:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.376+0000 7f12297fa700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f12140a21d0 0x7f12140a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.376+0000 7f122b7fe700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f12140a21d0 0x7f12140a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.376+0000 7f12297fa700 1 -- 172.21.15.60:0/4027852600 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f12240cdb00 con 0x7f122c101cc0 2024-02-26T04:54:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.376+0000 7f122b7fe700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f12140a21d0 0x7f12140a4690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1220000c00 tx=0x7f1220005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:40.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.380+0000 7f12297fa700 1 -- 172.21.15.60:0/4027852600 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f12240cfe60 con 0x7f122c101cc0 2024-02-26T04:54:40.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:40 smithi191 ceph-mon[36352]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.583+0000 7f1232a5b700 1 -- 172.21.15.60:0/4027852600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f122c0754d0 con 0x7f122c101cc0 2024-02-26T04:54:40.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.585+0000 7f12297fa700 1 -- 172.21.15.60:0/4027852600 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18454 (secure 0 0 0) 0x7f1224096ed0 con 0x7f122c101cc0 2024-02-26T04:54:40.588 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:54:40.588 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":81,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:54:37.620492+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:54:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.587+0000 7f1232a5b700 1 -- 172.21.15.60:0/4027852600 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f12140a21d0 msgr2=0x7f12140a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.587+0000 7f1232a5b700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f12140a21d0 0x7f12140a4690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1220000c00 tx=0x7f1220005fb0 comp rx=0 tx=0).stop 2024-02-26T04:54:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.587+0000 7f1232a5b700 1 -- 172.21.15.60:0/4027852600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f122c101cc0 msgr2=0x7f122c073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.587+0000 7f1232a5b700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f122c101cc0 0x7f122c073f70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f122400ef10 tx=0x7f122400c3a0 comp rx=0 tx=0).stop 2024-02-26T04:54:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.588+0000 7f1232a5b700 1 -- 172.21.15.60:0/4027852600 shutdown_connections 2024-02-26T04:54:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.588+0000 7f1232a5b700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f12140a21d0 0x7f12140a4690 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.588+0000 7f1232a5b700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c107ca0 0x7f122c0749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.588+0000 7f1232a5b700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f122c101cc0 0x7f122c073f70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.588+0000 7f1232a5b700 1 --2- 172.21.15.60:0/4027852600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f122c102680 0x7f122c0744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:40.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.588+0000 7f1232a5b700 1 -- 172.21.15.60:0/4027852600 >> 172.21.15.60:0/4027852600 conn(0x7f122c078da0 msgr2=0x7f122c100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:40.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.588+0000 7f1232a5b700 1 -- 172.21.15.60:0/4027852600 shutdown_connections 2024-02-26T04:54:40.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:40.588+0000 7f1232a5b700 1 -- 172.21.15.60:0/4027852600 wait complete. 2024-02-26T04:54:40.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:40 smithi156 ceph-mon[35805]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:41.353 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T04:50:44.572921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T04:54:41.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-26T04:54:41.601 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:54:41.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:41 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4027852600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:54:41.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:41 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4027852600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:54:41.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:41 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4027852600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:54:42.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:42 smithi060 ceph-mon[32382]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:42.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:42 smithi156 ceph-mon[35805]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:42.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:42 smithi191 ceph-mon[36352]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:43.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.243+0000 7fddb8f28700 1 -- 172.21.15.60:0/4081831622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb4101cc0 msgr2=0x7fddb4102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:43.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.243+0000 7fddb8f28700 1 --2- 172.21.15.60:0/4081831622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb4101cc0 0x7fddb4102140 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdda80099c0 tx=0x7fdda8009cd0 comp rx=0 tx=0).stop 2024-02-26T04:54:43.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.243+0000 7fddb8f28700 1 -- 172.21.15.60:0/4081831622 shutdown_connections 2024-02-26T04:54:43.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.243+0000 7fddb8f28700 1 --2- 172.21.15.60:0/4081831622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb4102680 0x7fddb410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:43.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.243+0000 7fddb8f28700 1 --2- 172.21.15.60:0/4081831622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddb4107ca0 0x7fddb4108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:43.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.243+0000 7fddb8f28700 1 --2- 172.21.15.60:0/4081831622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb4101cc0 0x7fddb4102140 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:43.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.243+0000 7fddb8f28700 1 -- 172.21.15.60:0/4081831622 >> 172.21.15.60:0/4081831622 conn(0x7fddb4078da0 msgr2=0x7fddb40791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:43.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.243+0000 7fddb8f28700 1 -- 172.21.15.60:0/4081831622 shutdown_connections 2024-02-26T04:54:43.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.243+0000 7fddb8f28700 1 -- 172.21.15.60:0/4081831622 wait complete. 2024-02-26T04:54:43.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.244+0000 7fddb8f28700 1 Processor -- start 2024-02-26T04:54:43.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.244+0000 7fddb8f28700 1 -- start start 2024-02-26T04:54:43.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.244+0000 7fddb8f28700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb4101cc0 0x7fddb41980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:43.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.244+0000 7fddb8f28700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddb4102680 0x7fddb41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:43.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.244+0000 7fddb8f28700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb4107ca0 0x7fddb419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:43.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.244+0000 7fddb8f28700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb419cf90 con 0x7fddb4107ca0 2024-02-26T04:54:43.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.244+0000 7fddb8f28700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb419d100 con 0x7fddb4102680 2024-02-26T04:54:43.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.244+0000 7fddb8f28700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb419d2c0 con 0x7fddb4101cc0 2024-02-26T04:54:43.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.244+0000 7fddb2d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb4107ca0 0x7fddb419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:43.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.244+0000 7fddb2d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb4107ca0 0x7fddb419c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55688/0 (socket says 172.21.15.60:55688) 2024-02-26T04:54:43.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.244+0000 7fddb2d9d700 1 -- 172.21.15.60:0/4004549750 learned_addr learned my addr 172.21.15.60:0/4004549750 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:54:43.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.245+0000 7fddb259c700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb4101cc0 0x7fddb41980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:43.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.245+0000 7fddb1d9b700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddb4102680 0x7fddb41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:43.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.245+0000 7fddb2d9d700 1 -- 172.21.15.60:0/4004549750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb4101cc0 msgr2=0x7fddb41980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:43.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.245+0000 7fddb2d9d700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb4101cc0 0x7fddb41980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:43.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.245+0000 7fddb2d9d700 1 -- 172.21.15.60:0/4004549750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddb4102680 msgr2=0x7fddb41985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:43.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.245+0000 7fddb2d9d700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddb4102680 0x7fddb41985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:43.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.245+0000 7fddb2d9d700 1 -- 172.21.15.60:0/4004549750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda80095f0 con 0x7fddb4107ca0 2024-02-26T04:54:43.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.245+0000 7fddb1d9b700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddb4102680 0x7fddb41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:54:43.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.245+0000 7fddb2d9d700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb4107ca0 0x7fddb419c980 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fddac00b970 tx=0x7fddac00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:43.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.245+0000 7fdda37fe700 1 -- 172.21.15.60:0/4004549750 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac00c930 con 0x7fddb4107ca0 2024-02-26T04:54:43.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.245+0000 7fddb8f28700 1 -- 172.21.15.60:0/4004549750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb419d5a0 con 0x7fddb4107ca0 2024-02-26T04:54:43.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.245+0000 7fdda37fe700 1 -- 172.21.15.60:0/4004549750 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fddac014440 con 0x7fddb4107ca0 2024-02-26T04:54:43.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.245+0000 7fdda37fe700 1 -- 172.21.15.60:0/4004549750 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac0075d0 con 0x7fddb4107ca0 2024-02-26T04:54:43.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.245+0000 7fddb8f28700 1 -- 172.21.15.60:0/4004549750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb41a4e00 con 0x7fddb4107ca0 2024-02-26T04:54:43.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.246+0000 7fddb8f28700 1 -- 172.21.15.60:0/4004549750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb410a180 con 0x7fddb4107ca0 2024-02-26T04:54:43.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.247+0000 7fdda37fe700 1 -- 172.21.15.60:0/4004549750 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fddac0145b0 con 0x7fddb4107ca0 2024-02-26T04:54:43.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.248+0000 7fdda37fe700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdd9c0a2150 0x7fdd9c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:43.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.248+0000 7fddb259c700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdd9c0a2150 0x7fdd9c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:43.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.248+0000 7fdda37fe700 1 -- 172.21.15.60:0/4004549750 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fddac006e40 con 0x7fddb4107ca0 2024-02-26T04:54:43.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.248+0000 7fddb259c700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdd9c0a2150 0x7fdd9c0a4610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdda400ac50 tx=0x7fdda400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:43.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.253+0000 7fdda37fe700 1 -- 172.21.15.60:0/4004549750 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fddac096320 con 0x7fddb4107ca0 2024-02-26T04:54:43.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.453+0000 7fddb8f28700 1 -- 172.21.15.60:0/4004549750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fddb4199410 con 0x7fddb4107ca0 2024-02-26T04:54:43.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.453+0000 7fdda37fe700 1 -- 172.21.15.60:0/4004549750 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fddac0999e0 con 0x7fddb4107ca0 2024-02-26T04:54:43.456 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T04:54:43.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.455+0000 7fddb8f28700 1 -- 172.21.15.60:0/4004549750 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdd9c0a2150 msgr2=0x7fdd9c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:43.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.455+0000 7fddb8f28700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdd9c0a2150 0x7fdd9c0a4610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdda400ac50 tx=0x7fdda400a380 comp rx=0 tx=0).stop 2024-02-26T04:54:43.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.455+0000 7fddb8f28700 1 -- 172.21.15.60:0/4004549750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb4107ca0 msgr2=0x7fddb419c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:43.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.455+0000 7fddb8f28700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb4107ca0 0x7fddb419c980 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fddac00b970 tx=0x7fddac00bc80 comp rx=0 tx=0).stop 2024-02-26T04:54:43.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.455+0000 7fddb8f28700 1 -- 172.21.15.60:0/4004549750 shutdown_connections 2024-02-26T04:54:43.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.455+0000 7fddb8f28700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddb4107ca0 0x7fddb419c980 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:43.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.455+0000 7fddb8f28700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb4101cc0 0x7fddb41980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:43.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.455+0000 7fddb8f28700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdd9c0a2150 0x7fdd9c0a4610 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:43.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.455+0000 7fddb8f28700 1 --2- 172.21.15.60:0/4004549750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fddb4102680 0x7fddb41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:43.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.455+0000 7fddb8f28700 1 -- 172.21.15.60:0/4004549750 >> 172.21.15.60:0/4004549750 conn(0x7fddb4078da0 msgr2=0x7fddb41007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:43.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.455+0000 7fddb8f28700 1 -- 172.21.15.60:0/4004549750 shutdown_connections 2024-02-26T04:54:43.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:43.456+0000 7fddb8f28700 1 -- 172.21.15.60:0/4004549750 wait complete. 2024-02-26T04:54:44.281 INFO:tasks.cephadm:Adding mds.a on smithi060 2024-02-26T04:54:44.282 INFO:tasks.cephadm:Adding mds.d on smithi060 2024-02-26T04:54:44.282 INFO:tasks.cephadm:Adding mds.g on smithi060 2024-02-26T04:54:44.282 INFO:tasks.cephadm:Adding mds.j on smithi060 2024-02-26T04:54:44.282 INFO:tasks.cephadm:Adding mds.b on smithi156 2024-02-26T04:54:44.282 INFO:tasks.cephadm:Adding mds.e on smithi156 2024-02-26T04:54:44.282 INFO:tasks.cephadm:Adding mds.h on smithi156 2024-02-26T04:54:44.282 INFO:tasks.cephadm:Adding mds.k on smithi156 2024-02-26T04:54:44.282 INFO:tasks.cephadm:Adding mds.c on smithi191 2024-02-26T04:54:44.282 INFO:tasks.cephadm:Adding mds.f on smithi191 2024-02-26T04:54:44.282 INFO:tasks.cephadm:Adding mds.i on smithi191 2024-02-26T04:54:44.282 INFO:tasks.cephadm:Adding mds.l on smithi191 2024-02-26T04:54:44.283 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph orch apply mds all '12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l' 2024-02-26T04:54:44.530 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T04:54:44.652 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:44 smithi191 ceph-mon[36352]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:44.652 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:44 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4004549750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:54:44.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:44 smithi156 ceph-mon[35805]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:44.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:44 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4004549750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:54:44.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:44 smithi060 ceph-mon[32382]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:44.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:44 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4004549750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:54:46.293 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:46 smithi191 ceph-mon[36352]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:46.299 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.290+0000 7f204ea0b700 1 -- 172.21.15.191:0/2427220493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2048100ba0 msgr2=0x7f2048100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.290+0000 7f204ea0b700 1 --2- 172.21.15.191:0/2427220493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2048100ba0 0x7f2048100fc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f203c009990 tx=0x7f203c009ca0 comp rx=0 tx=0).stop 2024-02-26T04:54:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.290+0000 7f204ea0b700 1 -- 172.21.15.191:0/2427220493 shutdown_connections 2024-02-26T04:54:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.290+0000 7f204ea0b700 1 --2- 172.21.15.191:0/2427220493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2048102740 0x7f2048108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.290+0000 7f204ea0b700 1 --2- 172.21.15.191:0/2427220493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2048100ba0 0x7f2048100fc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.290+0000 7f204ea0b700 1 --2- 172.21.15.191:0/2427220493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2048101da0 0x7f2048102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.290+0000 7f204ea0b700 1 -- 172.21.15.191:0/2427220493 >> 172.21.15.191:0/2427220493 conn(0x7f20480fc140 msgr2=0x7f20480fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.290+0000 7f204ea0b700 1 -- 172.21.15.191:0/2427220493 shutdown_connections 2024-02-26T04:54:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.290+0000 7f204ea0b700 1 -- 172.21.15.191:0/2427220493 wait complete. 2024-02-26T04:54:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.291+0000 7f204ea0b700 1 Processor -- start 2024-02-26T04:54:46.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.291+0000 7f204ea0b700 1 -- start start 2024-02-26T04:54:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.291+0000 7f204ea0b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2048100ba0 0x7f2048199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.291+0000 7f204ea0b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2048101da0 0x7f204819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.291+0000 7f204ea0b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2048102740 0x7f20481a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.291+0000 7f204ea0b700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20481a1a30 con 0x7f2048102740 2024-02-26T04:54:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.291+0000 7f204ea0b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20481a1b70 con 0x7f2048100ba0 2024-02-26T04:54:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.291+0000 7f204ea0b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20481a1d00 con 0x7f2048101da0 2024-02-26T04:54:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.291+0000 7f20477fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2048101da0 0x7f204819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.291+0000 7f20477fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2048101da0 0x7f204819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:43550/0 (socket says 172.21.15.191:43550) 2024-02-26T04:54:46.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.291+0000 7f20477fe700 1 -- 172.21.15.191:0/2241178429 learned_addr learned my addr 172.21.15.191:0/2241178429 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-26T04:54:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.291+0000 7f2047fff700 1 --2- 172.21.15.191:0/2241178429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2048100ba0 0x7f2048199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.291+0000 7f204cfa8700 1 --2- 172.21.15.191:0/2241178429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2048102740 0x7f20481a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.292+0000 7f20477fe700 1 -- 172.21.15.191:0/2241178429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2048100ba0 msgr2=0x7f2048199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.292+0000 7f20477fe700 1 --2- 172.21.15.191:0/2241178429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2048100ba0 0x7f2048199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.292+0000 7f20477fe700 1 -- 172.21.15.191:0/2241178429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2048102740 msgr2=0x7f20481a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.292+0000 7f20477fe700 1 --2- 172.21.15.191:0/2241178429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2048102740 0x7f20481a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.292+0000 7f20477fe700 1 -- 172.21.15.191:0/2241178429 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0095f0 con 0x7f2048101da0 2024-02-26T04:54:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.292+0000 7f20477fe700 1 --2- 172.21.15.191:0/2241178429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2048101da0 0x7f204819a3e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f203800c9e0 tx=0x7f203800ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.292+0000 7f20457fa700 1 -- 172.21.15.191:0/2241178429 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038007ac0 con 0x7f2048101da0 2024-02-26T04:54:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.292+0000 7f20457fa700 1 -- 172.21.15.191:0/2241178429 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f203800ce80 con 0x7f2048101da0 2024-02-26T04:54:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.292+0000 7f20457fa700 1 -- 172.21.15.191:0/2241178429 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038018660 con 0x7f2048101da0 2024-02-26T04:54:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.292+0000 7f204ea0b700 1 -- 172.21.15.191:0/2241178429 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20481a1fe0 con 0x7f2048101da0 2024-02-26T04:54:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.292+0000 7f204ea0b700 1 -- 172.21.15.191:0/2241178429 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20481a2480 con 0x7f2048101da0 2024-02-26T04:54:46.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.293+0000 7f204ea0b700 1 -- 172.21.15.191:0/2241178429 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2048068970 con 0x7f2048101da0 2024-02-26T04:54:46.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.294+0000 7f20457fa700 1 -- 172.21.15.191:0/2241178429 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f203800f450 con 0x7f2048101da0 2024-02-26T04:54:46.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.295+0000 7f20457fa700 1 --2- 172.21.15.191:0/2241178429 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f20300a21d0 0x7f20300a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:46.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.295+0000 7f20457fa700 1 -- 172.21.15.191:0/2241178429 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f20380cd8c0 con 0x7f2048101da0 2024-02-26T04:54:46.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.295+0000 7f2047fff700 1 --2- 172.21.15.191:0/2241178429 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f20300a21d0 0x7f20300a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:46.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.296+0000 7f2047fff700 1 --2- 172.21.15.191:0/2241178429 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f20300a21d0 0x7f20300a4690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f203c00b5c0 tx=0x7f203c0038d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:46.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.299+0000 7f20457fa700 1 -- 172.21.15.191:0/2241178429 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2038096e40 con 0x7f2048101da0 2024-02-26T04:54:46.513 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.510+0000 7f204ea0b700 1 -- 172.21.15.191:0/2241178429 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f204806a130 con 0x7f20300a21d0 2024-02-26T04:54:46.520 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.518+0000 7f20457fa700 1 -- 172.21.15.191:0/2241178429 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f204806a130 con 0x7f20300a21d0 2024-02-26T04:54:46.520 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mds.all update... 2024-02-26T04:54:46.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.520+0000 7f204ea0b700 1 -- 172.21.15.191:0/2241178429 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f20300a21d0 msgr2=0x7f20300a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:46.522 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.520+0000 7f204ea0b700 1 --2- 172.21.15.191:0/2241178429 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f20300a21d0 0x7f20300a4690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f203c00b5c0 tx=0x7f203c0038d0 comp rx=0 tx=0).stop 2024-02-26T04:54:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.520+0000 7f204ea0b700 1 -- 172.21.15.191:0/2241178429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2048101da0 msgr2=0x7f204819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.520+0000 7f204ea0b700 1 --2- 172.21.15.191:0/2241178429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2048101da0 0x7f204819a3e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f203800c9e0 tx=0x7f203800ccf0 comp rx=0 tx=0).stop 2024-02-26T04:54:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.520+0000 7f204ea0b700 1 -- 172.21.15.191:0/2241178429 shutdown_connections 2024-02-26T04:54:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.520+0000 7f204ea0b700 1 --2- 172.21.15.191:0/2241178429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2048102740 0x7f20481a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.520+0000 7f204ea0b700 1 --2- 172.21.15.191:0/2241178429 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f20300a21d0 0x7f20300a4690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.520+0000 7f204ea0b700 1 --2- 172.21.15.191:0/2241178429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2048100ba0 0x7f2048199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.520+0000 7f204ea0b700 1 --2- 172.21.15.191:0/2241178429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2048101da0 0x7f204819a3e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.520+0000 7f204ea0b700 1 -- 172.21.15.191:0/2241178429 >> 172.21.15.191:0/2241178429 conn(0x7f20480fc140 msgr2=0x7f2048069920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.520+0000 7f204ea0b700 1 -- 172.21.15.191:0/2241178429 shutdown_connections 2024-02-26T04:54:46.523 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-26T04:54:46.520+0000 7f204ea0b700 1 -- 172.21.15.191:0/2241178429 wait complete. 2024-02-26T04:54:46.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:46 smithi156 ceph-mon[35805]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:46.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:46 smithi060 ceph-mon[32382]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:47.232 DEBUG:teuthology.orchestra.run.smithi060:mds.a> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.a.service 2024-02-26T04:54:47.234 DEBUG:teuthology.orchestra.run.smithi060:mds.d> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.d.service 2024-02-26T04:54:47.277 DEBUG:teuthology.orchestra.run.smithi060:mds.g> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.g.service 2024-02-26T04:54:47.320 DEBUG:teuthology.orchestra.run.smithi060:mds.j> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.j.service 2024-02-26T04:54:47.363 DEBUG:teuthology.orchestra.run.smithi156:mds.b> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.b.service 2024-02-26T04:54:47.366 DEBUG:teuthology.orchestra.run.smithi156:mds.e> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.e.service 2024-02-26T04:54:47.410 DEBUG:teuthology.orchestra.run.smithi156:mds.h> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.h.service 2024-02-26T04:54:47.453 DEBUG:teuthology.orchestra.run.smithi156:mds.k> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.k.service 2024-02-26T04:54:47.496 DEBUG:teuthology.orchestra.run.smithi191:mds.c> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.c.service 2024-02-26T04:54:47.498 DEBUG:teuthology.orchestra.run.smithi191:mds.f> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.f.service 2024-02-26T04:54:47.542 DEBUG:teuthology.orchestra.run.smithi191:mds.i> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.i.service 2024-02-26T04:54:47.585 DEBUG:teuthology.orchestra.run.smithi191:mds.l> sudo journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.l.service 2024-02-26T04:54:47.629 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-02-26T04:54:47.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:54:47.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:47 smithi156 ceph-mon[35805]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:47.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:47 smithi156 ceph-mon[35805]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:54:47.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:47 smithi156 ceph-mon[35805]: Saving service mds.all spec with placement smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-02-26T04:54:47.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:47.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:47.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:47.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:47.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:47.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:54:47.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:54:47.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:47 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:47.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.738+0000 7f42c678c640 1 -- 172.21.15.60:0/3665778660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c0101c40 msgr2=0x7f42c0104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:47.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.738+0000 7f42c678c640 1 --2- 172.21.15.60:0/3665778660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c0101c40 0x7f42c0104030 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009900 tx=0x7f42b402bc30 comp rx=0 tx=0).stop 2024-02-26T04:54:47.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.738+0000 7f42c678c640 1 -- 172.21.15.60:0/3665778660 shutdown_connections 2024-02-26T04:54:47.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.738+0000 7f42c678c640 1 --2- 172.21.15.60:0/3665778660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c0106ea0 0x7f42c01092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:47.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.738+0000 7f42c678c640 1 --2- 172.21.15.60:0/3665778660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c0104570 0x7f42c0106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:47.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.738+0000 7f42c678c640 1 --2- 172.21.15.60:0/3665778660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c0101c40 0x7f42c0104030 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:47.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.738+0000 7f42c678c640 1 -- 172.21.15.60:0/3665778660 >> 172.21.15.60:0/3665778660 conn(0x7f42c00fb630 msgr2=0x7f42c00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:47.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.738+0000 7f42c678c640 1 -- 172.21.15.60:0/3665778660 shutdown_connections 2024-02-26T04:54:47.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.738+0000 7f42c678c640 1 -- 172.21.15.60:0/3665778660 wait complete. 2024-02-26T04:54:47.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.738+0000 7f42c678c640 1 Processor -- start 2024-02-26T04:54:47.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.738+0000 7f42c678c640 1 -- start start 2024-02-26T04:54:47.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.738+0000 7f42c678c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c0101c40 0x7f42c0197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:47.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.738+0000 7f42c678c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c0104570 0x7f42c01977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:47.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.738+0000 7f42c678c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c0106ea0 0x7f42c019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:47.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42c678c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c019ed50 con 0x7f42c0106ea0 2024-02-26T04:54:47.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42c678c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c019eec0 con 0x7f42c0104570 2024-02-26T04:54:47.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42c678c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c019f080 con 0x7f42c0101c40 2024-02-26T04:54:47.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42c4d29640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c0106ea0 0x7f42c019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:47.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42bffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c0101c40 0x7f42c0197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:47.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42c4d29640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c0106ea0 0x7f42c019e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55710/0 (socket says 172.21.15.60:55710) 2024-02-26T04:54:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42bffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c0101c40 0x7f42c0197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.60:40036/0 (socket says 172.21.15.60:40036) 2024-02-26T04:54:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42c4d29640 1 -- 172.21.15.60:0/2996366782 learned_addr learned my addr 172.21.15.60:0/2996366782 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:54:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42bf7fe640 1 --2- 172.21.15.60:0/2996366782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c0104570 0x7f42c01977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42c4d29640 1 -- 172.21.15.60:0/2996366782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c0101c40 msgr2=0x7f42c0197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42c4d29640 1 --2- 172.21.15.60:0/2996366782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c0101c40 0x7f42c0197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42c4d29640 1 -- 172.21.15.60:0/2996366782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c0104570 msgr2=0x7f42c01977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42c4d29640 1 --2- 172.21.15.60:0/2996366782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c0104570 0x7f42c01977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42c4d29640 1 -- 172.21.15.60:0/2996366782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b4009560 con 0x7f42c0106ea0 2024-02-26T04:54:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42bf7fe640 1 --2- 172.21.15.60:0/2996366782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c0104570 0x7f42c01977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:54:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42c4d29640 1 --2- 172.21.15.60:0/2996366782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c0106ea0 0x7f42c019e810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f42b800eb10 tx=0x7f42b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42bd7fa640 1 -- 172.21.15.60:0/2996366782 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b8004b20 con 0x7f42c0106ea0 2024-02-26T04:54:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42c678c640 1 -- 172.21.15.60:0/2996366782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c019f360 con 0x7f42c0106ea0 2024-02-26T04:54:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42bd7fa640 1 -- 172.21.15.60:0/2996366782 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42b8004c80 con 0x7f42c0106ea0 2024-02-26T04:54:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42bd7fa640 1 -- 172.21.15.60:0/2996366782 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b8010650 con 0x7f42c0106ea0 2024-02-26T04:54:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.739+0000 7f42c678c640 1 -- 172.21.15.60:0/2996366782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c019f870 con 0x7f42c0106ea0 2024-02-26T04:54:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.740+0000 7f42c678c640 1 -- 172.21.15.60:0/2996366782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c00fd120 con 0x7f42c0106ea0 2024-02-26T04:54:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.741+0000 7f42bd7fa640 1 -- 172.21.15.60:0/2996366782 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f42b80107f0 con 0x7f42c0106ea0 2024-02-26T04:54:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.742+0000 7f42bd7fa640 1 --2- 172.21.15.60:0/2996366782 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f42a80a20c0 0x7f42a80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.742+0000 7f42bd7fa640 1 -- 172.21.15.60:0/2996366782 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f42b8014070 con 0x7f42c0106ea0 2024-02-26T04:54:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.742+0000 7f42bffff640 1 --2- 172.21.15.60:0/2996366782 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f42a80a20c0 0x7f42a80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.742+0000 7f42bffff640 1 --2- 172.21.15.60:0/2996366782 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f42a80a20c0 0x7f42a80a4580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009230 tx=0x7f42b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:47.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.746+0000 7f42bd7fa640 1 -- 172.21.15.60:0/2996366782 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42b8097480 con 0x7f42c0106ea0 2024-02-26T04:54:47.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.928+0000 7f42c678c640 1 -- 172.21.15.60:0/2996366782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42c002cdd0 con 0x7f42c0106ea0 2024-02-26T04:54:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.929+0000 7f42bd7fa640 1 -- 172.21.15.60:0/2996366782 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18454 (secure 0 0 0) 0x7f42b809ab40 con 0x7f42c0106ea0 2024-02-26T04:54:47.936 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:54:47.936 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":81,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:54:37.620492+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:54:47.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.931+0000 7f42c678c640 1 -- 172.21.15.60:0/2996366782 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f42a80a20c0 msgr2=0x7f42a80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:47.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.931+0000 7f42c678c640 1 --2- 172.21.15.60:0/2996366782 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f42a80a20c0 0x7f42a80a4580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009230 tx=0x7f42b4005fd0 comp rx=0 tx=0).stop 2024-02-26T04:54:47.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.931+0000 7f42c678c640 1 -- 172.21.15.60:0/2996366782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c0106ea0 msgr2=0x7f42c019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:47.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.931+0000 7f42c678c640 1 --2- 172.21.15.60:0/2996366782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c0106ea0 0x7f42c019e810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f42b800eb10 tx=0x7f42b800c2d0 comp rx=0 tx=0).stop 2024-02-26T04:54:47.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.931+0000 7f42c678c640 1 -- 172.21.15.60:0/2996366782 shutdown_connections 2024-02-26T04:54:47.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.931+0000 7f42c678c640 1 --2- 172.21.15.60:0/2996366782 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f42a80a20c0 0x7f42a80a4580 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:47.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.931+0000 7f42c678c640 1 --2- 172.21.15.60:0/2996366782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c0106ea0 0x7f42c019e810 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:47.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.931+0000 7f42c678c640 1 --2- 172.21.15.60:0/2996366782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c0104570 0x7f42c01977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.931+0000 7f42c678c640 1 --2- 172.21.15.60:0/2996366782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42c0101c40 0x7f42c0197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.931+0000 7f42c678c640 1 -- 172.21.15.60:0/2996366782 >> 172.21.15.60:0/2996366782 conn(0x7f42c00fb630 msgr2=0x7f42c0102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.931+0000 7f42c678c640 1 -- 172.21.15.60:0/2996366782 shutdown_connections 2024-02-26T04:54:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:47.931+0000 7f42c678c640 1 -- 172.21.15.60:0/2996366782 wait complete. 2024-02-26T04:54:47.945 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:47 smithi060 ceph-mon[32382]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:47.945 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:47 smithi060 ceph-mon[32382]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:54:47.945 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:47 smithi060 ceph-mon[32382]: Saving service mds.all spec with placement smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-02-26T04:54:47.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:47.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:47.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:47.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:47.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:47.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:54:47.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:54:47.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:47 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:47.949 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T04:50:44.572921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T04:54:47.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T04:54:48.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:47 smithi191 ceph-mon[36352]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:48.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:47 smithi191 ceph-mon[36352]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:54:48.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:47 smithi191 ceph-mon[36352]: Saving service mds.all spec with placement smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-02-26T04:54:48.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:48.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:54:48.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:48.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:54:48.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:48.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:54:48.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:54:48.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:47 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:48.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.053+0000 7f001d91e640 1 -- 172.21.15.60:0/2668187536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0018102d40 msgr2=0x7f00181095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:48.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.053+0000 7f001d91e640 1 --2- 172.21.15.60:0/2668187536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0018102d40 0x7f00181095d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0008009ca0 tx=0x7f000802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 -- 172.21.15.60:0/2668187536 shutdown_connections 2024-02-26T04:54:48.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 --2- 172.21.15.60:0/2668187536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0018102d40 0x7f00181095d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 --2- 172.21.15.60:0/2668187536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0018102380 0x7f0018102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 --2- 172.21.15.60:0/2668187536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0018101190 0x7f00181015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 -- 172.21.15.60:0/2668187536 >> 172.21.15.60:0/2668187536 conn(0x7f00180fc720 msgr2=0x7f00180feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:48.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 -- 172.21.15.60:0/2668187536 shutdown_connections 2024-02-26T04:54:48.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 -- 172.21.15.60:0/2668187536 wait complete. 2024-02-26T04:54:48.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 Processor -- start 2024-02-26T04:54:48.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 -- start start 2024-02-26T04:54:48.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0018101190 0x7f00181971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:48.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0018102380 0x7f0018197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:48.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0018102d40 0x7f001819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:48.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001819ece0 con 0x7f0018102380 2024-02-26T04:54:48.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001819ee50 con 0x7f0018102d40 2024-02-26T04:54:48.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.054+0000 7f001d91e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001819f010 con 0x7f0018101190 2024-02-26T04:54:48.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f00167fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0018102380 0x7f0018197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:48.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f00167fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0018102380 0x7f0018197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55738/0 (socket says 172.21.15.60:55738) 2024-02-26T04:54:48.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f00167fc640 1 -- 172.21.15.60:0/2705826386 learned_addr learned my addr 172.21.15.60:0/2705826386 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:54:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f00177fe640 1 --2- 172.21.15.60:0/2705826386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0018102d40 0x7f001819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f0016ffd640 1 --2- 172.21.15.60:0/2705826386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0018101190 0x7f00181971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f00167fc640 1 -- 172.21.15.60:0/2705826386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0018101190 msgr2=0x7f00181971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f00167fc640 1 --2- 172.21.15.60:0/2705826386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0018101190 0x7f00181971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f00167fc640 1 -- 172.21.15.60:0/2705826386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0018102d40 msgr2=0x7f001819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f00167fc640 1 --2- 172.21.15.60:0/2705826386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0018102d40 0x7f001819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f00167fc640 1 -- 172.21.15.60:0/2705826386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0008009900 con 0x7f0018102380 2024-02-26T04:54:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f00167fc640 1 --2- 172.21.15.60:0/2705826386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0018102380 0x7f0018197730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f000000e9f0 tx=0x7f000000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f001c91c640 1 -- 172.21.15.60:0/2705826386 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0000009720 con 0x7f0018102380 2024-02-26T04:54:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f001d91e640 1 -- 172.21.15.60:0/2705826386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001819f2f0 con 0x7f0018102380 2024-02-26T04:54:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f001c91c640 1 -- 172.21.15.60:0/2705826386 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f000000cd80 con 0x7f0018102380 2024-02-26T04:54:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f001c91c640 1 -- 172.21.15.60:0/2705826386 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0000010690 con 0x7f0018102380 2024-02-26T04:54:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.055+0000 7f001d91e640 1 -- 172.21.15.60:0/2705826386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001819f7b0 con 0x7f0018102380 2024-02-26T04:54:48.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.056+0000 7f001d91e640 1 -- 172.21.15.60:0/2705826386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00181084b0 con 0x7f0018102380 2024-02-26T04:54:48.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.057+0000 7f001c91c640 1 -- 172.21.15.60:0/2705826386 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f00000046c0 con 0x7f0018102380 2024-02-26T04:54:48.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.058+0000 7f001c91c640 1 --2- 172.21.15.60:0/2705826386 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f00040a20c0 0x7f00040a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:48.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.058+0000 7f0016ffd640 1 --2- 172.21.15.60:0/2705826386 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f00040a20c0 0x7f00040a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:48.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.058+0000 7f001c91c640 1 -- 172.21.15.60:0/2705826386 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0000014070 con 0x7f0018102380 2024-02-26T04:54:48.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.058+0000 7f0016ffd640 1 --2- 172.21.15.60:0/2705826386 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f00040a20c0 0x7f00040a4580 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0010005fd0 tx=0x7f001000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:48.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.062+0000 7f001c91c640 1 -- 172.21.15.60:0/2705826386 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0000096f50 con 0x7f0018102380 2024-02-26T04:54:48.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.244+0000 7f001d91e640 1 -- 172.21.15.60:0/2705826386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f001802ce40 con 0x7f0018102380 2024-02-26T04:54:48.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.244+0000 7f001c91c640 1 -- 172.21.15.60:0/2705826386 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f000009a610 con 0x7f0018102380 2024-02-26T04:54:48.247 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T04:54:48.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.246+0000 7f001d91e640 1 -- 172.21.15.60:0/2705826386 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f00040a20c0 msgr2=0x7f00040a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:48.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.246+0000 7f001d91e640 1 --2- 172.21.15.60:0/2705826386 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f00040a20c0 0x7f00040a4580 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0010005fd0 tx=0x7f001000b410 comp rx=0 tx=0).stop 2024-02-26T04:54:48.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.246+0000 7f001d91e640 1 -- 172.21.15.60:0/2705826386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0018102380 msgr2=0x7f0018197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:48.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.246+0000 7f001d91e640 1 --2- 172.21.15.60:0/2705826386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0018102380 0x7f0018197730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f000000e9f0 tx=0x7f000000edf0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.246+0000 7f001d91e640 1 -- 172.21.15.60:0/2705826386 shutdown_connections 2024-02-26T04:54:48.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.246+0000 7f001d91e640 1 --2- 172.21.15.60:0/2705826386 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f00040a20c0 0x7f00040a4580 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.246+0000 7f001d91e640 1 --2- 172.21.15.60:0/2705826386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0018102d40 0x7f001819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.246+0000 7f001d91e640 1 --2- 172.21.15.60:0/2705826386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0018102380 0x7f0018197730 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.246+0000 7f001d91e640 1 --2- 172.21.15.60:0/2705826386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0018101190 0x7f00181971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.246+0000 7f001d91e640 1 -- 172.21.15.60:0/2705826386 >> 172.21.15.60:0/2705826386 conn(0x7f00180fc720 msgr2=0x7f0018105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:48.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.246+0000 7f001d91e640 1 -- 172.21.15.60:0/2705826386 shutdown_connections 2024-02-26T04:54:48.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.246+0000 7f001d91e640 1 -- 172.21.15.60:0/2705826386 wait complete. 2024-02-26T04:54:48.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:54:48.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.360+0000 7fd2a18d5640 1 -- 172.21.15.60:0/3080274953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29c101cd0 msgr2=0x7fd29c108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:48.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.360+0000 7fd2a18d5640 1 --2- 172.21.15.60:0/3080274953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29c101cd0 0x7fd29c108560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd294009ca0 tx=0x7fd29402c0f0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 -- 172.21.15.60:0/3080274953 shutdown_connections 2024-02-26T04:54:48.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 --2- 172.21.15.60:0/3080274953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29c101cd0 0x7fd29c108560 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 --2- 172.21.15.60:0/3080274953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd29c101310 0x7fd29c101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 --2- 172.21.15.60:0/3080274953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd29c100120 0x7fd29c100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 -- 172.21.15.60:0/3080274953 >> 172.21.15.60:0/3080274953 conn(0x7fd29c0fb6b0 msgr2=0x7fd29c0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 -- 172.21.15.60:0/3080274953 shutdown_connections 2024-02-26T04:54:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 -- 172.21.15.60:0/3080274953 wait complete. 2024-02-26T04:54:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 Processor -- start 2024-02-26T04:54:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 -- start start 2024-02-26T04:54:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd29c100120 0x7fd29c06a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29c101310 0x7fd29c19bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd29c101cd0 0x7fd29c19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c19ecc0 con 0x7fd29c101310 2024-02-26T04:54:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c19ee30 con 0x7fd29c101cd0 2024-02-26T04:54:48.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.361+0000 7fd2a18d5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c19eff0 con 0x7fd29c100120 2024-02-26T04:54:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd29a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29c101310 0x7fd29c19bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd29a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29c101310 0x7fd29c19bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55754/0 (socket says 172.21.15.60:55754) 2024-02-26T04:54:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd29a7fc640 1 -- 172.21.15.60:0/2766892120 learned_addr learned my addr 172.21.15.60:0/2766892120 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:54:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd29affd640 1 --2- 172.21.15.60:0/2766892120 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd29c100120 0x7fd29c06a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd29b7fe640 1 --2- 172.21.15.60:0/2766892120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd29c101cd0 0x7fd29c19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd29a7fc640 1 -- 172.21.15.60:0/2766892120 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd29c100120 msgr2=0x7fd29c06a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd29a7fc640 1 --2- 172.21.15.60:0/2766892120 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd29c100120 0x7fd29c06a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd29a7fc640 1 -- 172.21.15.60:0/2766892120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd29c101cd0 msgr2=0x7fd29c19e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd29a7fc640 1 --2- 172.21.15.60:0/2766892120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd29c101cd0 0x7fd29c19e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd29a7fc640 1 -- 172.21.15.60:0/2766892120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd294009900 con 0x7fd29c101310 2024-02-26T04:54:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd29a7fc640 1 --2- 172.21.15.60:0/2766892120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29c101310 0x7fd29c19bef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd29000d950 tx=0x7fd29000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd2a08d3640 1 -- 172.21.15.60:0/2766892120 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290009c30 con 0x7fd29c101310 2024-02-26T04:54:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd2a18d5640 1 -- 172.21.15.60:0/2766892120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29c19f2d0 con 0x7fd29c101310 2024-02-26T04:54:48.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd2a08d3640 1 -- 172.21.15.60:0/2766892120 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd290009d90 con 0x7fd29c101310 2024-02-26T04:54:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd2a08d3640 1 -- 172.21.15.60:0/2766892120 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29000f7e0 con 0x7fd29c101310 2024-02-26T04:54:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.362+0000 7fd2a18d5640 1 -- 172.21.15.60:0/2766892120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29c19f830 con 0x7fd29c101310 2024-02-26T04:54:48.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.363+0000 7fd2a18d5640 1 -- 172.21.15.60:0/2766892120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29c1073e0 con 0x7fd29c101310 2024-02-26T04:54:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.364+0000 7fd2a08d3640 1 -- 172.21.15.60:0/2766892120 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd290010ad0 con 0x7fd29c101310 2024-02-26T04:54:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.365+0000 7fd2a08d3640 1 --2- 172.21.15.60:0/2766892120 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd2840a20c0 0x7fd2840a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.365+0000 7fd29affd640 1 --2- 172.21.15.60:0/2766892120 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd2840a20c0 0x7fd2840a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.365+0000 7fd2a08d3640 1 -- 172.21.15.60:0/2766892120 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd2900cd6a0 con 0x7fd29c101310 2024-02-26T04:54:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.365+0000 7fd29affd640 1 --2- 172.21.15.60:0/2766892120 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd2840a20c0 0x7fd2840a4580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd29c101170 tx=0x7fd28c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.368+0000 7fd2a08d3640 1 -- 172.21.15.60:0/2766892120 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd290096bc0 con 0x7fd29c101310 2024-02-26T04:54:48.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.587+0000 7fd2a18d5640 1 -- 172.21.15.60:0/2766892120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd29c10bac0 con 0x7fd29c101310 2024-02-26T04:54:48.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.587+0000 7fd2a08d3640 1 -- 172.21.15.60:0/2766892120 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fd29009a280 con 0x7fd29c101310 2024-02-26T04:54:48.590 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:54:48.590 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-02-26T04:54:48.591 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 1 2024-02-26T04:54:48.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.589+0000 7fd2a18d5640 1 -- 172.21.15.60:0/2766892120 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd2840a20c0 msgr2=0x7fd2840a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:48.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.589+0000 7fd2a18d5640 1 --2- 172.21.15.60:0/2766892120 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd2840a20c0 0x7fd2840a4580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd29c101170 tx=0x7fd28c005f90 comp rx=0 tx=0).stop 2024-02-26T04:54:48.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.589+0000 7fd2a18d5640 1 -- 172.21.15.60:0/2766892120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29c101310 msgr2=0x7fd29c19bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:48.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.589+0000 7fd2a18d5640 1 --2- 172.21.15.60:0/2766892120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29c101310 0x7fd29c19bef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd29000d950 tx=0x7fd29000de10 comp rx=0 tx=0).stop 2024-02-26T04:54:48.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.589+0000 7fd2a18d5640 1 -- 172.21.15.60:0/2766892120 shutdown_connections 2024-02-26T04:54:48.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.589+0000 7fd2a18d5640 1 --2- 172.21.15.60:0/2766892120 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd2840a20c0 0x7fd2840a4580 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.589+0000 7fd2a18d5640 1 --2- 172.21.15.60:0/2766892120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd29c101cd0 0x7fd29c19e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.589+0000 7fd2a18d5640 1 --2- 172.21.15.60:0/2766892120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29c101310 0x7fd29c19bef0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.589+0000 7fd2a18d5640 1 --2- 172.21.15.60:0/2766892120 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd29c100120 0x7fd29c06a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:48.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.589+0000 7fd2a18d5640 1 -- 172.21.15.60:0/2766892120 >> 172.21.15.60:0/2766892120 conn(0x7fd29c0fb6b0 msgr2=0x7fd29c1045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:48.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.590+0000 7fd2a18d5640 1 -- 172.21.15.60:0/2766892120 shutdown_connections 2024-02-26T04:54:48.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:48.590+0000 7fd2a18d5640 1 -- 172.21.15.60:0/2766892120 wait complete. 2024-02-26T04:54:48.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:48 smithi191 ceph-mon[36352]: Deploying daemon mds.l on smithi191 2024-02-26T04:54:48.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:48 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2996366782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:54:48.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:48 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2705826386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:54:48.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:48 smithi156 ceph-mon[35805]: Deploying daemon mds.l on smithi191 2024-02-26T04:54:48.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:48 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2996366782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:54:48.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:48 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2705826386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:54:48.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:48 smithi060 ceph-mon[32382]: Deploying daemon mds.l on smithi191 2024-02-26T04:54:48.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:48 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2996366782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:54:48.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:48 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2705826386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:54:49.921 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:49 smithi191 ceph-mon[36352]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:49.921 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:49 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2766892120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:49.921 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 04:54:49 smithi191 systemd[1]: Starting Ceph mds.l for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T04:54:49.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:49 smithi156 ceph-mon[35805]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:49.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:49 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2766892120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:49.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:49 smithi060 ceph-mon[32382]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:49.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:49 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2766892120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:50.244 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 04:54:49 smithi191 podman[65935]: 2024-02-26 04:54:49.821609056 +0000 UTC m=+0.017792062 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T04:54:50.244 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 04:54:49 smithi191 podman[65935]: 2024-02-26 04:54:49.95408901 +0000 UTC m=+0.150272042 container create 1639a982b18c1104303635d396f823ea29082e87307e0c30028912405182f25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-l, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS) 2024-02-26T04:54:50.244 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 04:54:50 smithi191 podman[65935]: 2024-02-26 04:54:50.242196719 +0000 UTC m=+0.438379738 container init 1639a982b18c1104303635d396f823ea29082e87307e0c30028912405182f25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-l, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T04:54:50.509 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 04:54:50 smithi191 podman[65935]: 2024-02-26 04:54:50.245090566 +0000 UTC m=+0.441273559 container start 1639a982b18c1104303635d396f823ea29082e87307e0c30028912405182f25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-l, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-26T04:54:50.509 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 04:54:50 smithi191 ceph-mds[65955]: -- 172.21.15.191:0/2415632694 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c30671a1a0 con 0x55c3066f1000 2024-02-26T04:54:50.509 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 04:54:50 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-l[65951]: starting mds.l at 2024-02-26T04:54:50.510 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 04:54:50 smithi191 bash[65935]: 1639a982b18c1104303635d396f823ea29082e87307e0c30028912405182f25c 2024-02-26T04:54:50.510 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 04:54:50 smithi191 systemd[1]: Started Ceph mds.l for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:54:50.604 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:54:50.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.709+0000 7f7c40488640 1 -- 172.21.15.60:0/2333060219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c38101310 msgr2=0x7f7c38101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.709+0000 7f7c40488640 1 --2- 172.21.15.60:0/2333060219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c38101310 0x7f7c38101790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009900 tx=0x7f7c2802bc30 comp rx=0 tx=0).stop 2024-02-26T04:54:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.709+0000 7f7c40488640 1 -- 172.21.15.60:0/2333060219 shutdown_connections 2024-02-26T04:54:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.709+0000 7f7c40488640 1 --2- 172.21.15.60:0/2333060219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c38101cd0 0x7f7c38108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.709+0000 7f7c40488640 1 --2- 172.21.15.60:0/2333060219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c38101310 0x7f7c38101790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.709+0000 7f7c40488640 1 --2- 172.21.15.60:0/2333060219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c38100120 0x7f7c38100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.709+0000 7f7c40488640 1 -- 172.21.15.60:0/2333060219 >> 172.21.15.60:0/2333060219 conn(0x7f7c380fb6b0 msgr2=0x7f7c380fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.709+0000 7f7c40488640 1 -- 172.21.15.60:0/2333060219 shutdown_connections 2024-02-26T04:54:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.709+0000 7f7c40488640 1 -- 172.21.15.60:0/2333060219 wait complete. 2024-02-26T04:54:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.709+0000 7f7c40488640 1 Processor -- start 2024-02-26T04:54:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.709+0000 7f7c40488640 1 -- start start 2024-02-26T04:54:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c40488640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c38100120 0x7f7c38197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c40488640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c38101310 0x7f7c38197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c40488640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c38101cd0 0x7f7c3819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c40488640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3819ed30 con 0x7f7c38100120 2024-02-26T04:54:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c40488640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3819eea0 con 0x7f7c38101cd0 2024-02-26T04:54:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c40488640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3819f060 con 0x7f7c38101310 2024-02-26T04:54:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c3e224640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c38100120 0x7f7c38197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c3e224640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c38100120 0x7f7c38197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51040/0 (socket says 172.21.15.60:51040) 2024-02-26T04:54:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c3e224640 1 -- 172.21.15.60:0/308872474 learned_addr learned my addr 172.21.15.60:0/308872474 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:54:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c3ea25640 1 --2- 172.21.15.60:0/308872474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c38101cd0 0x7f7c3819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c3da23640 1 --2- 172.21.15.60:0/308872474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c38101310 0x7f7c38197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c3e224640 1 -- 172.21.15.60:0/308872474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c38101310 msgr2=0x7f7c38197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c3e224640 1 --2- 172.21.15.60:0/308872474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c38101310 0x7f7c38197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c3e224640 1 -- 172.21.15.60:0/308872474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c38101cd0 msgr2=0x7f7c3819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c3e224640 1 --2- 172.21.15.60:0/308872474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c38101cd0 0x7f7c3819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c3e224640 1 -- 172.21.15.60:0/308872474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c28009560 con 0x7f7c38100120 2024-02-26T04:54:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c3e224640 1 --2- 172.21.15.60:0/308872474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c38100120 0x7f7c38197240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400e9c0 tx=0x7f7c3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c2f7fe640 1 -- 172.21.15.60:0/308872474 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34009700 con 0x7f7c38100120 2024-02-26T04:54:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c2f7fe640 1 -- 172.21.15.60:0/308872474 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c3400cd80 con 0x7f7c38100120 2024-02-26T04:54:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c2f7fe640 1 -- 172.21.15.60:0/308872474 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34010430 con 0x7f7c38100120 2024-02-26T04:54:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c40488640 1 -- 172.21.15.60:0/308872474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3819f340 con 0x7f7c38100120 2024-02-26T04:54:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.710+0000 7f7c40488640 1 -- 172.21.15.60:0/308872474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3819f8a0 con 0x7f7c38100120 2024-02-26T04:54:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.711+0000 7f7c40488640 1 -- 172.21.15.60:0/308872474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c38107490 con 0x7f7c38100120 2024-02-26T04:54:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.712+0000 7f7c2f7fe640 1 -- 172.21.15.60:0/308872474 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f7c34003680 con 0x7f7c38100120 2024-02-26T04:54:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.713+0000 7f7c2f7fe640 1 --2- 172.21.15.60:0/308872474 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7c240a2140 0x7f7c240a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.713+0000 7f7c3da23640 1 --2- 172.21.15.60:0/308872474 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7c240a2140 0x7f7c240a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.713+0000 7f7c2f7fe640 1 -- 172.21.15.60:0/308872474 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7c34014070 con 0x7f7c38100120 2024-02-26T04:54:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.713+0000 7f7c3da23640 1 --2- 172.21.15.60:0/308872474 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7c240a2140 0x7f7c240a4600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7c2802c6a0 tx=0x7f7c28005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:50.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.718+0000 7f7c2f7fe640 1 -- 172.21.15.60:0/308872474 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7c34096fe0 con 0x7f7c38100120 2024-02-26T04:54:50.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.937+0000 7f7c40488640 1 -- 172.21.15.60:0/308872474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c3810cbc0 con 0x7f7c38100120 2024-02-26T04:54:50.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.937+0000 7f7c2f7fe640 1 -- 172.21.15.60:0/308872474 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f7c3409a6a0 con 0x7f7c38100120 2024-02-26T04:54:50.940 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:54:50.940 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-26T04:54:50.940 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 2 2024-02-26T04:54:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.939+0000 7f7c40488640 1 -- 172.21.15.60:0/308872474 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7c240a2140 msgr2=0x7f7c240a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.939+0000 7f7c40488640 1 --2- 172.21.15.60:0/308872474 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7c240a2140 0x7f7c240a4600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7c2802c6a0 tx=0x7f7c28005fd0 comp rx=0 tx=0).stop 2024-02-26T04:54:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.939+0000 7f7c40488640 1 -- 172.21.15.60:0/308872474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c38100120 msgr2=0x7f7c38197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.939+0000 7f7c40488640 1 --2- 172.21.15.60:0/308872474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c38100120 0x7f7c38197240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400e9c0 tx=0x7f7c3400ee80 comp rx=0 tx=0).stop 2024-02-26T04:54:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.939+0000 7f7c40488640 1 -- 172.21.15.60:0/308872474 shutdown_connections 2024-02-26T04:54:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.939+0000 7f7c40488640 1 --2- 172.21.15.60:0/308872474 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7c240a2140 0x7f7c240a4600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.939+0000 7f7c40488640 1 --2- 172.21.15.60:0/308872474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c38101cd0 0x7f7c3819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.939+0000 7f7c40488640 1 --2- 172.21.15.60:0/308872474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c38101310 0x7f7c38197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.939+0000 7f7c40488640 1 --2- 172.21.15.60:0/308872474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c38100120 0x7f7c38197240 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.939+0000 7f7c40488640 1 -- 172.21.15.60:0/308872474 >> 172.21.15.60:0/308872474 conn(0x7f7c380fb6b0 msgr2=0x7f7c38104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.939+0000 7f7c40488640 1 -- 172.21.15.60:0/308872474 shutdown_connections 2024-02-26T04:54:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:50.939+0000 7f7c40488640 1 -- 172.21.15.60:0/308872474 wait complete. 2024-02-26T04:54:51.310 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:51 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.191:6832/1119041220,v1:172.21.15.191:6833/1119041220] up:boot 2024-02-26T04:54:51.310 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:51 smithi156 ceph-mon[35805]: fsmap 1 up:standby 2024-02-26T04:54:51.310 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:51 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T04:54:51.310 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:51 smithi156 ceph-mon[35805]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:51.310 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:51 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:51.310 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:51 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:51.310 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:51 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:51.310 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:51 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:54:51.310 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:51 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:54:51.311 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:51 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:51.311 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:51 smithi156 ceph-mon[35805]: Deploying daemon mds.k on smithi156 2024-02-26T04:54:51.311 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:51 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/308872474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:51.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:51 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.191:6832/1119041220,v1:172.21.15.191:6833/1119041220] up:boot 2024-02-26T04:54:51.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:51 smithi060 ceph-mon[32382]: fsmap 1 up:standby 2024-02-26T04:54:51.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:51 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T04:54:51.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:51 smithi060 ceph-mon[32382]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:51.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:51 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:51.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:51 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:51.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:51 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:51.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:51 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:54:51.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:51 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:54:51.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:51 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:51.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:51 smithi060 ceph-mon[32382]: Deploying daemon mds.k on smithi156 2024-02-26T04:54:51.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:51 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/308872474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:51.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:51 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.191:6832/1119041220,v1:172.21.15.191:6833/1119041220] up:boot 2024-02-26T04:54:51.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:51 smithi191 ceph-mon[36352]: fsmap 1 up:standby 2024-02-26T04:54:51.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:51 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-26T04:54:51.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:51 smithi191 ceph-mon[36352]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:51.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:51 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:51.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:51 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:51.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:51 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:51.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:51 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:54:51.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:51 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:54:51.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:51 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:51.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:51 smithi191 ceph-mon[36352]: Deploying daemon mds.k on smithi156 2024-02-26T04:54:51.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:51 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/308872474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:52.927 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 26 04:54:52 smithi156 podman[64906]: 2024-02-26 04:54:52.899519471 +0000 UTC m=+0.141789447 container create a5348b5648366ae3853859154570e9b6c164764190f5e4be1fb1fcd264bbc293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-k, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-26T04:54:52.953 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:54:53.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.066+0000 7fbd85605640 1 -- 172.21.15.60:0/4173611805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd801011b0 msgr2=0x7fbd801015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:53.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.066+0000 7fbd85605640 1 --2- 172.21.15.60:0/4173611805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd801011b0 0x7fbd801015d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009900 tx=0x7fbd7002bc30 comp rx=0 tx=0).stop 2024-02-26T04:54:53.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.066+0000 7fbd85605640 1 -- 172.21.15.60:0/4173611805 shutdown_connections 2024-02-26T04:54:53.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.066+0000 7fbd85605640 1 --2- 172.21.15.60:0/4173611805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd80102d60 0x7fbd801095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.066+0000 7fbd85605640 1 --2- 172.21.15.60:0/4173611805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd801023a0 0x7fbd80102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.066+0000 7fbd85605640 1 --2- 172.21.15.60:0/4173611805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd801011b0 0x7fbd801015d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.066+0000 7fbd85605640 1 -- 172.21.15.60:0/4173611805 >> 172.21.15.60:0/4173611805 conn(0x7fbd800fc760 msgr2=0x7fbd800feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.066+0000 7fbd85605640 1 -- 172.21.15.60:0/4173611805 shutdown_connections 2024-02-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.066+0000 7fbd85605640 1 -- 172.21.15.60:0/4173611805 wait complete. 2024-02-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd85605640 1 Processor -- start 2024-02-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd85605640 1 -- start start 2024-02-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd85605640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd801011b0 0x7fbd80074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd85605640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd801023a0 0x7fbd80073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd85605640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd80102d60 0x7fbd80073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd85605640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd80074210 con 0x7fbd801023a0 2024-02-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd85605640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd80074380 con 0x7fbd80102d60 2024-02-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd85605640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd801a12e0 con 0x7fbd801011b0 2024-02-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd77fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd801023a0 0x7fbd80073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd77fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd801023a0 0x7fbd80073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51048/0 (socket says 172.21.15.60:51048) 2024-02-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd77fff640 1 -- 172.21.15.60:0/3364581449 learned_addr learned my addr 172.21.15.60:0/3364581449 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd7f7fe640 1 --2- 172.21.15.60:0/3364581449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd80102d60 0x7fbd80073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd7effd640 1 --2- 172.21.15.60:0/3364581449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd801011b0 0x7fbd80074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd77fff640 1 -- 172.21.15.60:0/3364581449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd801011b0 msgr2=0x7fbd80074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd77fff640 1 --2- 172.21.15.60:0/3364581449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd801011b0 0x7fbd80074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd77fff640 1 -- 172.21.15.60:0/3364581449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd80102d60 msgr2=0x7fbd80073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd77fff640 1 --2- 172.21.15.60:0/3364581449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd80102d60 0x7fbd80073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.067+0000 7fbd77fff640 1 -- 172.21.15.60:0/3364581449 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd70009560 con 0x7fbd801023a0 2024-02-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.068+0000 7fbd77fff640 1 --2- 172.21.15.60:0/3364581449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd801023a0 0x7fbd80073570 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800d7a0 tx=0x7fbd6800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.068+0000 7fbd7cff9640 1 -- 172.21.15.60:0/3364581449 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68009c30 con 0x7fbd801023a0 2024-02-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.068+0000 7fbd7cff9640 1 -- 172.21.15.60:0/3364581449 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd6800deb0 con 0x7fbd801023a0 2024-02-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.068+0000 7fbd85605640 1 -- 172.21.15.60:0/3364581449 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd801a1590 con 0x7fbd801023a0 2024-02-26T04:54:53.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.068+0000 7fbd7cff9640 1 -- 172.21.15.60:0/3364581449 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68004ea0 con 0x7fbd801023a0 2024-02-26T04:54:53.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.068+0000 7fbd85605640 1 -- 172.21.15.60:0/3364581449 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd801a1a50 con 0x7fbd801023a0 2024-02-26T04:54:53.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.069+0000 7fbd85605640 1 -- 172.21.15.60:0/3364581449 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd801085a0 con 0x7fbd801023a0 2024-02-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.070+0000 7fbd7cff9640 1 -- 172.21.15.60:0/3364581449 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fbd68005020 con 0x7fbd801023a0 2024-02-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.071+0000 7fbd7cff9640 1 --2- 172.21.15.60:0/3364581449 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbd6c0a20c0 0x7fbd6c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.071+0000 7fbd7effd640 1 --2- 172.21.15.60:0/3364581449 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbd6c0a20c0 0x7fbd6c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.071+0000 7fbd7cff9640 1 -- 172.21.15.60:0/3364581449 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbd680cd2e0 con 0x7fbd801023a0 2024-02-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.071+0000 7fbd7effd640 1 --2- 172.21.15.60:0/3364581449 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbd6c0a20c0 0x7fbd6c0a4580 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbd7002c6a0 tx=0x7fbd70005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:53.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.075+0000 7fbd7cff9640 1 -- 172.21.15.60:0/3364581449 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbd680966b0 con 0x7fbd801023a0 2024-02-26T04:54:53.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.294+0000 7fbd85605640 1 -- 172.21.15.60:0/3364581449 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd8002ce40 con 0x7fbd801023a0 2024-02-26T04:54:53.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.294+0000 7fbd7cff9640 1 -- 172.21.15.60:0/3364581449 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fbd68099d70 con 0x7fbd801023a0 2024-02-26T04:54:53.297 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:54:53.297 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-26T04:54:53.297 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 2 2024-02-26T04:54:53.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.296+0000 7fbd85605640 1 -- 172.21.15.60:0/3364581449 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbd6c0a20c0 msgr2=0x7fbd6c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:53.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.296+0000 7fbd85605640 1 --2- 172.21.15.60:0/3364581449 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbd6c0a20c0 0x7fbd6c0a4580 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbd7002c6a0 tx=0x7fbd70005fd0 comp rx=0 tx=0).stop 2024-02-26T04:54:53.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.296+0000 7fbd85605640 1 -- 172.21.15.60:0/3364581449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd801023a0 msgr2=0x7fbd80073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:53.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.296+0000 7fbd85605640 1 --2- 172.21.15.60:0/3364581449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd801023a0 0x7fbd80073570 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800d7a0 tx=0x7fbd6800dc60 comp rx=0 tx=0).stop 2024-02-26T04:54:53.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.296+0000 7fbd85605640 1 -- 172.21.15.60:0/3364581449 shutdown_connections 2024-02-26T04:54:53.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.296+0000 7fbd85605640 1 --2- 172.21.15.60:0/3364581449 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbd6c0a20c0 0x7fbd6c0a4580 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:53.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.296+0000 7fbd85605640 1 --2- 172.21.15.60:0/3364581449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd80102d60 0x7fbd80073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:53.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.296+0000 7fbd85605640 1 --2- 172.21.15.60:0/3364581449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd801023a0 0x7fbd80073570 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:53.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.296+0000 7fbd85605640 1 --2- 172.21.15.60:0/3364581449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd801011b0 0x7fbd80074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:53.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.296+0000 7fbd85605640 1 -- 172.21.15.60:0/3364581449 >> 172.21.15.60:0/3364581449 conn(0x7fbd800fc760 msgr2=0x7fbd80105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:53.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.296+0000 7fbd85605640 1 -- 172.21.15.60:0/3364581449 shutdown_connections 2024-02-26T04:54:53.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:53.296+0000 7fbd85605640 1 -- 172.21.15.60:0/3364581449 wait complete. 2024-02-26T04:54:53.354 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 26 04:54:53 smithi156 podman[64906]: 2024-02-26 04:54:53.161218413 +0000 UTC m=+0.403488406 container init a5348b5648366ae3853859154570e9b6c164764190f5e4be1fb1fcd264bbc293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-k, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux ) 2024-02-26T04:54:53.354 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 26 04:54:53 smithi156 podman[64906]: 2024-02-26 04:54:53.163953607 +0000 UTC m=+0.406223600 container start a5348b5648366ae3853859154570e9b6c164764190f5e4be1fb1fcd264bbc293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-k, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T04:54:53.354 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 26 04:54:53 smithi156 ceph-mds[64925]: -- 172.21.15.156:0/3228860528 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563d1ff1c1a0 con 0x563d1fef2800 2024-02-26T04:54:53.354 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 26 04:54:53 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-k[64921]: starting mds.k at 2024-02-26T04:54:53.354 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 26 04:54:53 smithi156 bash[64906]: a5348b5648366ae3853859154570e9b6c164764190f5e4be1fb1fcd264bbc293 2024-02-26T04:54:53.354 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 26 04:54:53 smithi156 systemd[1]: Started Ceph mds.k for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:54:53.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:53 smithi156 ceph-mon[35805]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:53.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:53 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3364581449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:53.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:53 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:53.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:53 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:53.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:53 smithi060 ceph-mon[32382]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:53.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:53 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3364581449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:53.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:53 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:53.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:53 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:53.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:53 smithi191 ceph-mon[36352]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:53.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:53 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3364581449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:53.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:53 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:53.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:53 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:54.625 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:54 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.156:6832/629272517,v1:172.21.15.156:6833/629272517] up:boot 2024-02-26T04:54:54.625 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:54 smithi156 ceph-mon[35805]: fsmap 2 up:standby 2024-02-26T04:54:54.625 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T04:54:54.625 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:54.625 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:54:54.625 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:54:54.625 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:54 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:54.625 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:54 smithi156 ceph-mon[35805]: Deploying daemon mds.j on smithi060 2024-02-26T04:54:54.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:54 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.156:6832/629272517,v1:172.21.15.156:6833/629272517] up:boot 2024-02-26T04:54:54.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:54 smithi060 ceph-mon[32382]: fsmap 2 up:standby 2024-02-26T04:54:54.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T04:54:54.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:54.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:54:54.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:54:54.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:54 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:54.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:54 smithi060 ceph-mon[32382]: Deploying daemon mds.j on smithi060 2024-02-26T04:54:54.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:54 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.156:6832/629272517,v1:172.21.15.156:6833/629272517] up:boot 2024-02-26T04:54:54.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:54 smithi191 ceph-mon[36352]: fsmap 2 up:standby 2024-02-26T04:54:54.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-26T04:54:54.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:54.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:54:54.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:54:54.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:54 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:54.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:54 smithi191 ceph-mon[36352]: Deploying daemon mds.j on smithi060 2024-02-26T04:54:55.312 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:54:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 -- 172.21.15.60:0/3775796141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe880101290 msgr2=0x7fe880101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 --2- 172.21.15.60:0/3775796141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe880101290 0x7fe880101710 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009900 tx=0x7fe86c02bc30 comp rx=0 tx=0).stop 2024-02-26T04:54:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 -- 172.21.15.60:0/3775796141 shutdown_connections 2024-02-26T04:54:55.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 --2- 172.21.15.60:0/3775796141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe880101c50 0x7fe8801084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 --2- 172.21.15.60:0/3775796141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe880101290 0x7fe880101710 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 --2- 172.21.15.60:0/3775796141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8801000a0 0x7fe8801004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 -- 172.21.15.60:0/3775796141 >> 172.21.15.60:0/3775796141 conn(0x7fe8800fb610 msgr2=0x7fe8800fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 -- 172.21.15.60:0/3775796141 shutdown_connections 2024-02-26T04:54:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 -- 172.21.15.60:0/3775796141 wait complete. 2024-02-26T04:54:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 Processor -- start 2024-02-26T04:54:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 -- start start 2024-02-26T04:54:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8801000a0 0x7fe880197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe880101290 0x7fe880197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe880101c50 0x7fe88019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88019ed40 con 0x7fe880101290 2024-02-26T04:54:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88019eeb0 con 0x7fe880101c50 2024-02-26T04:54:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe887111640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88019f070 con 0x7fe8801000a0 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe87ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe880101290 0x7fe880197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.431+0000 7fe87ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe880101290 0x7fe880197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51062/0 (socket says 172.21.15.60:51062) 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.432+0000 7fe87ffff640 1 -- 172.21.15.60:0/2853980600 learned_addr learned my addr 172.21.15.60:0/2853980600 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.432+0000 7fe8856ae640 1 --2- 172.21.15.60:0/2853980600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe880101c50 0x7fe88019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.432+0000 7fe884ead640 1 --2- 172.21.15.60:0/2853980600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8801000a0 0x7fe880197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.432+0000 7fe87ffff640 1 -- 172.21.15.60:0/2853980600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8801000a0 msgr2=0x7fe880197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.432+0000 7fe87ffff640 1 --2- 172.21.15.60:0/2853980600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8801000a0 0x7fe880197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.432+0000 7fe87ffff640 1 -- 172.21.15.60:0/2853980600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe880101c50 msgr2=0x7fe88019e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.432+0000 7fe87ffff640 1 --2- 172.21.15.60:0/2853980600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe880101c50 0x7fe88019e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.432+0000 7fe87ffff640 1 -- 172.21.15.60:0/2853980600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe878009840 con 0x7fe880101290 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.432+0000 7fe87ffff640 1 --2- 172.21.15.60:0/2853980600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe880101290 0x7fe880197790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe86c006010 tx=0x7fe86c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.432+0000 7fe87dffb640 1 -- 172.21.15.60:0/2853980600 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0044c0 con 0x7fe880101290 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.432+0000 7fe87dffb640 1 -- 172.21.15.60:0/2853980600 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe86c0052d0 con 0x7fe880101290 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.432+0000 7fe87dffb640 1 -- 172.21.15.60:0/2853980600 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c003d10 con 0x7fe880101290 2024-02-26T04:54:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.432+0000 7fe887111640 1 -- 172.21.15.60:0/2853980600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86c009560 con 0x7fe880101290 2024-02-26T04:54:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.432+0000 7fe887111640 1 -- 172.21.15.60:0/2853980600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88019f670 con 0x7fe880101290 2024-02-26T04:54:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.433+0000 7fe887111640 1 -- 172.21.15.60:0/2853980600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe880107410 con 0x7fe880101290 2024-02-26T04:54:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.435+0000 7fe87dffb640 1 -- 172.21.15.60:0/2853980600 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe86c005440 con 0x7fe880101290 2024-02-26T04:54:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.436+0000 7fe87dffb640 1 --2- 172.21.15.60:0/2853980600 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe8700a20c0 0x7fe8700a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.436+0000 7fe884ead640 1 --2- 172.21.15.60:0/2853980600 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe8700a20c0 0x7fe8700a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.436+0000 7fe87dffb640 1 -- 172.21.15.60:0/2853980600 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe86c0f01e0 con 0x7fe880101290 2024-02-26T04:54:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.436+0000 7fe884ead640 1 --2- 172.21.15.60:0/2853980600 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe8700a20c0 0x7fe8700a4580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe8740058c0 tx=0x7fe874005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.439+0000 7fe87dffb640 1 -- 172.21.15.60:0/2853980600 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe86c0b95b0 con 0x7fe880101290 2024-02-26T04:54:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.671+0000 7fe887111640 1 -- 172.21.15.60:0/2853980600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe88010cbb0 con 0x7fe880101290 2024-02-26T04:54:55.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.671+0000 7fe87dffb640 1 -- 172.21.15.60:0/2853980600 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7fe86c0bcc70 con 0x7fe880101290 2024-02-26T04:54:55.674 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:54:55.674 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-26T04:54:55.675 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 3 2024-02-26T04:54:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.673+0000 7fe887111640 1 -- 172.21.15.60:0/2853980600 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe8700a20c0 msgr2=0x7fe8700a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.673+0000 7fe887111640 1 --2- 172.21.15.60:0/2853980600 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe8700a20c0 0x7fe8700a4580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe8740058c0 tx=0x7fe874005c80 comp rx=0 tx=0).stop 2024-02-26T04:54:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.673+0000 7fe887111640 1 -- 172.21.15.60:0/2853980600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe880101290 msgr2=0x7fe880197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.673+0000 7fe887111640 1 --2- 172.21.15.60:0/2853980600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe880101290 0x7fe880197790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe86c006010 tx=0x7fe86c0040c0 comp rx=0 tx=0).stop 2024-02-26T04:54:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.673+0000 7fe887111640 1 -- 172.21.15.60:0/2853980600 shutdown_connections 2024-02-26T04:54:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.673+0000 7fe887111640 1 --2- 172.21.15.60:0/2853980600 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe8700a20c0 0x7fe8700a4580 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:55.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.673+0000 7fe887111640 1 --2- 172.21.15.60:0/2853980600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe880101c50 0x7fe88019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:55.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.673+0000 7fe887111640 1 --2- 172.21.15.60:0/2853980600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe880101290 0x7fe880197790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:55.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.673+0000 7fe887111640 1 --2- 172.21.15.60:0/2853980600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8801000a0 0x7fe880197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:55.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.673+0000 7fe887111640 1 -- 172.21.15.60:0/2853980600 >> 172.21.15.60:0/2853980600 conn(0x7fe8800fb610 msgr2=0x7fe8801045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:55.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.673+0000 7fe887111640 1 -- 172.21.15.60:0/2853980600 shutdown_connections 2024-02-26T04:54:55.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:55.673+0000 7fe887111640 1 -- 172.21.15.60:0/2853980600 wait complete. 2024-02-26T04:54:55.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:55 smithi156 ceph-mon[35805]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:55.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:55 smithi060 ceph-mon[32382]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:55.686 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 04:54:55 smithi060 systemd[1]: Starting Ceph mds.j for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T04:54:55.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:55 smithi191 ceph-mon[36352]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:55.953 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 04:54:55 smithi060 podman[70255]: 2024-02-26 04:54:55.667576931 +0000 UTC m=+0.018747064 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T04:54:55.953 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 04:54:55 smithi060 podman[70255]: 2024-02-26 04:54:55.797578105 +0000 UTC m=+0.148748217 container create e1af7e57f0a7702f94765e4b7455711e0f69eebe2cdd7d51e522feaf80292dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-j, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True) 2024-02-26T04:54:56.230 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 04:54:56 smithi060 podman[70255]: 2024-02-26 04:54:56.092364987 +0000 UTC m=+0.443535101 container init e1af7e57f0a7702f94765e4b7455711e0f69eebe2cdd7d51e522feaf80292dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-j, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T04:54:56.230 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 04:54:56 smithi060 podman[70255]: 2024-02-26 04:54:56.094947317 +0000 UTC m=+0.446117437 container start e1af7e57f0a7702f94765e4b7455711e0f69eebe2cdd7d51e522feaf80292dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-j, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T04:54:56.230 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 04:54:56 smithi060 ceph-mds[70277]: -- 172.21.15.60:0/3679830434 <== mon.0 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560e2625a1a0 con 0x560e26230800 2024-02-26T04:54:56.230 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 04:54:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-j[70273]: starting mds.j at 2024-02-26T04:54:56.230 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 04:54:56 smithi060 bash[70255]: e1af7e57f0a7702f94765e4b7455711e0f69eebe2cdd7d51e522feaf80292dcc 2024-02-26T04:54:56.230 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 04:54:56 smithi060 systemd[1]: Started Ceph mds.j for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:54:56.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:56 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2853980600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:56.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:56.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:56.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:56.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:54:56.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:54:56.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:56 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:56.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:56 smithi156 ceph-mon[35805]: Deploying daemon mds.d on smithi060 2024-02-26T04:54:56.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:56 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2853980600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:56.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:56.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:56.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:56.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:54:56.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:54:56.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:56 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:56.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:56 smithi060 ceph-mon[32382]: Deploying daemon mds.d on smithi060 2024-02-26T04:54:56.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:56 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2853980600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:56.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:56.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:56.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:54:56.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:54:56.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:54:56.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:56 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:54:56.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:56 smithi191 ceph-mon[36352]: Deploying daemon mds.d on smithi060 2024-02-26T04:54:57.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:57 smithi060 ceph-mon[32382]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:57.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:57 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.60:6834/894258792,v1:172.21.15.60:6835/894258792] up:boot 2024-02-26T04:54:57.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:57 smithi060 ceph-mon[32382]: fsmap 3 up:standby 2024-02-26T04:54:57.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:57 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T04:54:57.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:57 smithi156 ceph-mon[35805]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:57.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:57 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.60:6834/894258792,v1:172.21.15.60:6835/894258792] up:boot 2024-02-26T04:54:57.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:57 smithi156 ceph-mon[35805]: fsmap 3 up:standby 2024-02-26T04:54:57.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:57 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T04:54:57.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:54:57.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:57 smithi191 ceph-mon[36352]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:54:57.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:57 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.60:6834/894258792,v1:172.21.15.60:6835/894258792] up:boot 2024-02-26T04:54:57.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:57 smithi191 ceph-mon[36352]: fsmap 3 up:standby 2024-02-26T04:54:57.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:57 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-26T04:54:57.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.799+0000 7f31bc1aa640 1 -- 172.21.15.60:0/692032217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31b4102d60 msgr2=0x7f31b41095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:57.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.799+0000 7f31bc1aa640 1 --2- 172.21.15.60:0/692032217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31b4102d60 0x7f31b41095f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009ca0 tx=0x7f31b002c0f0 comp rx=0 tx=0).stop 2024-02-26T04:54:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.799+0000 7f31bc1aa640 1 -- 172.21.15.60:0/692032217 shutdown_connections 2024-02-26T04:54:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.799+0000 7f31bc1aa640 1 --2- 172.21.15.60:0/692032217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31b4102d60 0x7f31b41095f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.799+0000 7f31bc1aa640 1 --2- 172.21.15.60:0/692032217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31b41023a0 0x7f31b4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.799+0000 7f31bc1aa640 1 --2- 172.21.15.60:0/692032217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31b41011b0 0x7f31b41015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.799+0000 7f31bc1aa640 1 -- 172.21.15.60:0/692032217 >> 172.21.15.60:0/692032217 conn(0x7f31b40fc760 msgr2=0x7f31b40feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.800+0000 7f31bc1aa640 1 -- 172.21.15.60:0/692032217 shutdown_connections 2024-02-26T04:54:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.800+0000 7f31bc1aa640 1 -- 172.21.15.60:0/692032217 wait complete. 2024-02-26T04:54:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.800+0000 7f31bc1aa640 1 Processor -- start 2024-02-26T04:54:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.800+0000 7f31bc1aa640 1 -- start start 2024-02-26T04:54:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.800+0000 7f31bc1aa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31b41011b0 0x7f31b41971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.800+0000 7f31bc1aa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31b41023a0 0x7f31b41976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.800+0000 7f31bc1aa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31b4102d60 0x7f31b419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.800+0000 7f31bc1aa640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b419eca0 con 0x7f31b41023a0 2024-02-26T04:54:57.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.800+0000 7f31bc1aa640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b419ee10 con 0x7f31b41011b0 2024-02-26T04:54:57.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.800+0000 7f31bc1aa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b419efd0 con 0x7f31b4102d60 2024-02-26T04:54:57.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.800+0000 7f31b9745640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31b41023a0 0x7f31b41976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:57.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31b9745640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31b41023a0 0x7f31b41976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51114/0 (socket says 172.21.15.60:51114) 2024-02-26T04:54:57.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31b9745640 1 -- 172.21.15.60:0/1636374423 learned_addr learned my addr 172.21.15.60:0/1636374423 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:54:57.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31b9f46640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31b41011b0 0x7f31b41971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:57.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31ba747640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31b4102d60 0x7f31b419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:57.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31b9745640 1 -- 172.21.15.60:0/1636374423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31b4102d60 msgr2=0x7f31b419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:57.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31b9745640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31b4102d60 0x7f31b419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:57.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31b9745640 1 -- 172.21.15.60:0/1636374423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31b41011b0 msgr2=0x7f31b41971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:57.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31b9745640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31b41011b0 0x7f31b41971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:57.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31b9745640 1 -- 172.21.15.60:0/1636374423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b0009900 con 0x7f31b41023a0 2024-02-26T04:54:57.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31ba747640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31b4102d60 0x7f31b419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:54:57.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31b9745640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31b41023a0 0x7f31b41976f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f31a400d950 tx=0x7f31a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:57.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31aaffd640 1 -- 172.21.15.60:0/1636374423 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a4009c30 con 0x7f31b41023a0 2024-02-26T04:54:57.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31bc1aa640 1 -- 172.21.15.60:0/1636374423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b419f2b0 con 0x7f31b41023a0 2024-02-26T04:54:57.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31aaffd640 1 -- 172.21.15.60:0/1636374423 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31a4009d90 con 0x7f31b41023a0 2024-02-26T04:54:57.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31aaffd640 1 -- 172.21.15.60:0/1636374423 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a400f7e0 con 0x7f31b41023a0 2024-02-26T04:54:57.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.801+0000 7f31bc1aa640 1 -- 172.21.15.60:0/1636374423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b419f810 con 0x7f31b41023a0 2024-02-26T04:54:57.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.802+0000 7f31bc1aa640 1 -- 172.21.15.60:0/1636374423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b41084d0 con 0x7f31b41023a0 2024-02-26T04:54:57.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.803+0000 7f31aaffd640 1 -- 172.21.15.60:0/1636374423 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f31a4010ad0 con 0x7f31b41023a0 2024-02-26T04:54:57.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.804+0000 7f31aaffd640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f31a00a20c0 0x7f31a00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:54:57.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.804+0000 7f31b9f46640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f31a00a20c0 0x7f31a00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:54:57.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.804+0000 7f31aaffd640 1 -- 172.21.15.60:0/1636374423 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f31a40cd430 con 0x7f31b41023a0 2024-02-26T04:54:57.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.804+0000 7f31b9f46640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f31a00a20c0 0x7f31a00a4580 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f31ac005fd0 tx=0x7f31ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:54:57.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:57.807+0000 7f31aaffd640 1 -- 172.21.15.60:0/1636374423 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f31a4096800 con 0x7f31b41023a0 2024-02-26T04:54:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:58.031+0000 7f31bc1aa640 1 -- 172.21.15.60:0/1636374423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31b410cb80 con 0x7f31b41023a0 2024-02-26T04:54:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:58.032+0000 7f31aaffd640 1 -- 172.21.15.60:0/1636374423 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2907 (secure 0 0 0) 0x7f31a4020b70 con 0x7f31b41023a0 2024-02-26T04:54:58.035 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:54:58.035 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-26T04:54:58.035 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 4 2024-02-26T04:54:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:58.034+0000 7f31bc1aa640 1 -- 172.21.15.60:0/1636374423 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f31a00a20c0 msgr2=0x7f31a00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:58.034+0000 7f31bc1aa640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f31a00a20c0 0x7f31a00a4580 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f31ac005fd0 tx=0x7f31ac00b410 comp rx=0 tx=0).stop 2024-02-26T04:54:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:58.034+0000 7f31bc1aa640 1 -- 172.21.15.60:0/1636374423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31b41023a0 msgr2=0x7f31b41976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:54:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:58.034+0000 7f31bc1aa640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31b41023a0 0x7f31b41976f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f31a400d950 tx=0x7f31a400de10 comp rx=0 tx=0).stop 2024-02-26T04:54:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:58.034+0000 7f31bc1aa640 1 -- 172.21.15.60:0/1636374423 shutdown_connections 2024-02-26T04:54:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:58.034+0000 7f31bc1aa640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f31a00a20c0 0x7f31a00a4580 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:58.034+0000 7f31bc1aa640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31b4102d60 0x7f31b419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:58.034+0000 7f31bc1aa640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31b41023a0 0x7f31b41976f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:58.034+0000 7f31bc1aa640 1 --2- 172.21.15.60:0/1636374423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31b41011b0 0x7f31b41971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:54:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:58.034+0000 7f31bc1aa640 1 -- 172.21.15.60:0/1636374423 >> 172.21.15.60:0/1636374423 conn(0x7f31b40fc760 msgr2=0x7f31b41056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:54:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:58.034+0000 7f31bc1aa640 1 -- 172.21.15.60:0/1636374423 shutdown_connections 2024-02-26T04:54:58.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:54:58.034+0000 7f31bc1aa640 1 -- 172.21.15.60:0/1636374423 wait complete. 2024-02-26T04:54:58.661 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:54:58 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1636374423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:58.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:54:58 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1636374423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:58.690 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:54:58 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1636374423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:54:58.691 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 04:54:58 smithi060 systemd[1]: Starting Ceph mds.d for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T04:54:58.952 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 04:54:58 smithi060 podman[70611]: 2024-02-26 04:54:58.591412552 +0000 UTC m=+0.017249753 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T04:54:58.952 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 04:54:58 smithi060 podman[70611]: 2024-02-26 04:54:58.714180507 +0000 UTC m=+0.140017744 container create 5caf4150ff031438d340a21ba5078462b7904da8875038e3573341bb4bd383db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-d, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-02-26T04:54:59.230 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 04:54:58 smithi060 podman[70611]: 2024-02-26 04:54:58.950071311 +0000 UTC m=+0.375908507 container init 5caf4150ff031438d340a21ba5078462b7904da8875038e3573341bb4bd383db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=) 2024-02-26T04:54:59.230 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 04:54:58 smithi060 podman[70611]: 2024-02-26 04:54:58.952867463 +0000 UTC m=+0.378704661 container start 5caf4150ff031438d340a21ba5078462b7904da8875038e3573341bb4bd383db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-26T04:54:59.230 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 04:54:58 smithi060 ceph-mds[70629]: -- 172.21.15.60:0/2449721574 <== mon.0 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e884de41a0 con 0x55e884dbac00 2024-02-26T04:54:59.230 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 04:54:58 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-d[70625]: starting mds.d at 2024-02-26T04:54:59.230 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 04:54:59 smithi060 bash[70611]: 5caf4150ff031438d340a21ba5078462b7904da8875038e3573341bb4bd383db 2024-02-26T04:54:59.230 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 04:54:59 smithi060 systemd[1]: Started Ceph mds.d for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:55:00.047 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:55:00.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.164+0000 7fe16deb0640 1 -- 172.21.15.60:0/439944512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1681012b0 msgr2=0x7fe168101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:00.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.164+0000 7fe16deb0640 1 --2- 172.21.15.60:0/439944512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1681012b0 0x7fe168101730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009900 tx=0x7fe15c02bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:00.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.164+0000 7fe16deb0640 1 -- 172.21.15.60:0/439944512 shutdown_connections 2024-02-26T04:55:00.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.164+0000 7fe16deb0640 1 --2- 172.21.15.60:0/439944512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe168101c70 0x7fe168108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:00.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.164+0000 7fe16deb0640 1 --2- 172.21.15.60:0/439944512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1681012b0 0x7fe168101730 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:00.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.164+0000 7fe16deb0640 1 --2- 172.21.15.60:0/439944512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1681000c0 0x7fe1681004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:00.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.164+0000 7fe16deb0640 1 -- 172.21.15.60:0/439944512 >> 172.21.15.60:0/439944512 conn(0x7fe1680fb630 msgr2=0x7fe1680fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:00.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.164+0000 7fe16deb0640 1 -- 172.21.15.60:0/439944512 shutdown_connections 2024-02-26T04:55:00.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.164+0000 7fe16deb0640 1 -- 172.21.15.60:0/439944512 wait complete. 2024-02-26T04:55:00.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.165+0000 7fe16deb0640 1 Processor -- start 2024-02-26T04:55:00.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.165+0000 7fe16deb0640 1 -- start start 2024-02-26T04:55:00.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.165+0000 7fe16deb0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1681000c0 0x7fe168197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:00.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.165+0000 7fe16deb0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1681012b0 0x7fe168197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:00.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.165+0000 7fe16deb0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe168101c70 0x7fe16819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:00.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.165+0000 7fe16deb0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16819ed40 con 0x7fe1681012b0 2024-02-26T04:55:00.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.165+0000 7fe16deb0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16819eeb0 con 0x7fe168101c70 2024-02-26T04:55:00.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.165+0000 7fe16deb0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16819f070 con 0x7fe1681000c0 2024-02-26T04:55:00.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.165+0000 7fe166ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1681012b0 0x7fe168197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.165+0000 7fe166ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1681012b0 0x7fe168197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35838/0 (socket says 172.21.15.60:35838) 2024-02-26T04:55:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.165+0000 7fe166ffd640 1 -- 172.21.15.60:0/3895843741 learned_addr learned my addr 172.21.15.60:0/3895843741 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.165+0000 7fe1677fe640 1 --2- 172.21.15.60:0/3895843741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1681000c0 0x7fe168197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.166+0000 7fe167fff640 1 --2- 172.21.15.60:0/3895843741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe168101c70 0x7fe16819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.166+0000 7fe166ffd640 1 -- 172.21.15.60:0/3895843741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1681000c0 msgr2=0x7fe168197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.166+0000 7fe166ffd640 1 --2- 172.21.15.60:0/3895843741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1681000c0 0x7fe168197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.166+0000 7fe166ffd640 1 -- 172.21.15.60:0/3895843741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe168101c70 msgr2=0x7fe16819e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.166+0000 7fe166ffd640 1 --2- 172.21.15.60:0/3895843741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe168101c70 0x7fe16819e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.166+0000 7fe166ffd640 1 -- 172.21.15.60:0/3895843741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c009560 con 0x7fe1681012b0 2024-02-26T04:55:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.166+0000 7fe166ffd640 1 --2- 172.21.15.60:0/3895843741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1681012b0 0x7fe168197790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe15c006010 tx=0x7fe15c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.166+0000 7fe164ff9640 1 -- 172.21.15.60:0/3895843741 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c0044c0 con 0x7fe1681012b0 2024-02-26T04:55:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.166+0000 7fe164ff9640 1 -- 172.21.15.60:0/3895843741 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe15c0052d0 con 0x7fe1681012b0 2024-02-26T04:55:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.166+0000 7fe16deb0640 1 -- 172.21.15.60:0/3895843741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16819f2f0 con 0x7fe1681012b0 2024-02-26T04:55:00.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.166+0000 7fe164ff9640 1 -- 172.21.15.60:0/3895843741 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c003e10 con 0x7fe1681012b0 2024-02-26T04:55:00.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.166+0000 7fe16deb0640 1 -- 172.21.15.60:0/3895843741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16819f800 con 0x7fe1681012b0 2024-02-26T04:55:00.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.167+0000 7fe16deb0640 1 -- 172.21.15.60:0/3895843741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16810bad0 con 0x7fe1681012b0 2024-02-26T04:55:00.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.168+0000 7fe164ff9640 1 -- 172.21.15.60:0/3895843741 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe15c005440 con 0x7fe1681012b0 2024-02-26T04:55:00.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.169+0000 7fe164ff9640 1 --2- 172.21.15.60:0/3895843741 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe1500a20c0 0x7fe1500a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:00.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.169+0000 7fe1677fe640 1 --2- 172.21.15.60:0/3895843741 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe1500a20c0 0x7fe1500a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:00.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.169+0000 7fe164ff9640 1 -- 172.21.15.60:0/3895843741 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe15c0efc30 con 0x7fe1681012b0 2024-02-26T04:55:00.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.169+0000 7fe1677fe640 1 --2- 172.21.15.60:0/3895843741 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe1500a20c0 0x7fe1500a4580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe158005fd0 tx=0x7fe15800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:00.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.173+0000 7fe164ff9640 1 -- 172.21.15.60:0/3895843741 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe15c0b9000 con 0x7fe1681012b0 2024-02-26T04:55:00.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:00 smithi191 ceph-mon[36352]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:00.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:00 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:00.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:00 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:00.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:00 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:00.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:00 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:00.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:00 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:00.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:00 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:00.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:00 smithi191 ceph-mon[36352]: Deploying daemon mds.e on smithi156 2024-02-26T04:55:00.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:00 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.60:6836/1357036637,v1:172.21.15.60:6837/1357036637] up:boot 2024-02-26T04:55:00.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:00 smithi191 ceph-mon[36352]: fsmap 4 up:standby 2024-02-26T04:55:00.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:00 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T04:55:00.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.391+0000 7fe16deb0640 1 -- 172.21.15.60:0/3895843741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe16810bc60 con 0x7fe1681012b0 2024-02-26T04:55:00.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.392+0000 7fe164ff9640 1 -- 172.21.15.60:0/3895843741 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3698 (secure 0 0 0) 0x7fe15c043e90 con 0x7fe1681012b0 2024-02-26T04:55:00.394 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:00.394 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-26T04:55:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 5 2024-02-26T04:55:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.394+0000 7fe16deb0640 1 -- 172.21.15.60:0/3895843741 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe1500a20c0 msgr2=0x7fe1500a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.394+0000 7fe16deb0640 1 --2- 172.21.15.60:0/3895843741 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe1500a20c0 0x7fe1500a4580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe158005fd0 tx=0x7fe15800b390 comp rx=0 tx=0).stop 2024-02-26T04:55:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.394+0000 7fe16deb0640 1 -- 172.21.15.60:0/3895843741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1681012b0 msgr2=0x7fe168197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.394+0000 7fe16deb0640 1 --2- 172.21.15.60:0/3895843741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1681012b0 0x7fe168197790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe15c006010 tx=0x7fe15c0040c0 comp rx=0 tx=0).stop 2024-02-26T04:55:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.394+0000 7fe16deb0640 1 -- 172.21.15.60:0/3895843741 shutdown_connections 2024-02-26T04:55:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.394+0000 7fe16deb0640 1 --2- 172.21.15.60:0/3895843741 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe1500a20c0 0x7fe1500a4580 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.394+0000 7fe16deb0640 1 --2- 172.21.15.60:0/3895843741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe168101c70 0x7fe16819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.394+0000 7fe16deb0640 1 --2- 172.21.15.60:0/3895843741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1681012b0 0x7fe168197790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.394+0000 7fe16deb0640 1 --2- 172.21.15.60:0/3895843741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1681000c0 0x7fe168197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.394+0000 7fe16deb0640 1 -- 172.21.15.60:0/3895843741 >> 172.21.15.60:0/3895843741 conn(0x7fe1680fb630 msgr2=0x7fe1680697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.394+0000 7fe16deb0640 1 -- 172.21.15.60:0/3895843741 shutdown_connections 2024-02-26T04:55:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:00.394+0000 7fe16deb0640 1 -- 172.21.15.60:0/3895843741 wait complete. 2024-02-26T04:55:00.406 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:00 smithi060 ceph-mon[32382]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:00.406 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:00 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:00.406 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:00 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:00.406 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:00 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:00.406 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:00 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:00.407 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:00 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:00.407 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:00 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:00.407 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:00 smithi060 ceph-mon[32382]: Deploying daemon mds.e on smithi156 2024-02-26T04:55:00.407 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:00 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.60:6836/1357036637,v1:172.21.15.60:6837/1357036637] up:boot 2024-02-26T04:55:00.407 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:00 smithi060 ceph-mon[32382]: fsmap 4 up:standby 2024-02-26T04:55:00.407 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:00 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T04:55:00.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:00 smithi156 ceph-mon[35805]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:00.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:00 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:00.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:00 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:00.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:00 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:00.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:00 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:00.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:00 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:00.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:00 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:00.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:00 smithi156 ceph-mon[35805]: Deploying daemon mds.e on smithi156 2024-02-26T04:55:00.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:00 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.60:6836/1357036637,v1:172.21.15.60:6837/1357036637] up:boot 2024-02-26T04:55:00.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:00 smithi156 ceph-mon[35805]: fsmap 4 up:standby 2024-02-26T04:55:00.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:00 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-26T04:55:01.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:01 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3895843741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:01.359 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:01 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3895843741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:01.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:01 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3895843741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:01.677 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 04:55:01 smithi156 podman[65232]: 2024-02-26 04:55:01.597989351 +0000 UTC m=+0.139733237 container create c6fb7835692b26ab6c9611511439b49176d7b4463232740dff9813068e1cecb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True) 2024-02-26T04:55:02.112 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 04:55:01 smithi156 podman[65232]: 2024-02-26 04:55:01.825735636 +0000 UTC m=+0.367479528 container init c6fb7835692b26ab6c9611511439b49176d7b4463232740dff9813068e1cecb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2) 2024-02-26T04:55:02.112 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 04:55:01 smithi156 podman[65232]: 2024-02-26 04:55:01.82840566 +0000 UTC m=+0.370149543 container start c6fb7835692b26ab6c9611511439b49176d7b4463232740dff9813068e1cecb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-e, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-26T04:55:02.112 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 04:55:01 smithi156 ceph-mds[65251]: -- 172.21.15.156:0/2894882179 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55abec9041a0 con 0x55abec8dac00 2024-02-26T04:55:02.112 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 04:55:01 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-e[65247]: starting mds.e at 2024-02-26T04:55:02.113 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 04:55:01 smithi156 bash[65232]: c6fb7835692b26ab6c9611511439b49176d7b4463232740dff9813068e1cecb0 2024-02-26T04:55:02.113 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 04:55:01 smithi156 systemd[1]: Started Ceph mds.e for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:55:02.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:55:02.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:02 smithi156 ceph-mon[35805]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:02.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:02.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:02.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:02.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:02.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:02.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:02 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:02.437 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:02 smithi060 ceph-mon[32382]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:02.437 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:02.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:02.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:02.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:02.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:02.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:02 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:02.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:02 smithi191 ceph-mon[36352]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:02.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:02.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:02.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:02.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:02.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:02.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:02 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:02.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.522+0000 7fc7f0cde640 1 -- 172.21.15.60:0/400594389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ec102d70 msgr2=0x7fc7ec109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:02.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.522+0000 7fc7f0cde640 1 --2- 172.21.15.60:0/400594389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ec102d70 0x7fc7ec109600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009ca0 tx=0x7fc7dc02c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:02.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.522+0000 7fc7f0cde640 1 -- 172.21.15.60:0/400594389 shutdown_connections 2024-02-26T04:55:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.522+0000 7fc7f0cde640 1 --2- 172.21.15.60:0/400594389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ec102d70 0x7fc7ec109600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.522+0000 7fc7f0cde640 1 --2- 172.21.15.60:0/400594389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7ec1023b0 0x7fc7ec102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.522+0000 7fc7f0cde640 1 --2- 172.21.15.60:0/400594389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7ec1011c0 0x7fc7ec1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.522+0000 7fc7f0cde640 1 -- 172.21.15.60:0/400594389 >> 172.21.15.60:0/400594389 conn(0x7fc7ec0fc730 msgr2=0x7fc7ec0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.522+0000 7fc7f0cde640 1 -- 172.21.15.60:0/400594389 shutdown_connections 2024-02-26T04:55:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.522+0000 7fc7f0cde640 1 -- 172.21.15.60:0/400594389 wait complete. 2024-02-26T04:55:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7f0cde640 1 Processor -- start 2024-02-26T04:55:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7f0cde640 1 -- start start 2024-02-26T04:55:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7f0cde640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7ec1011c0 0x7fc7ec1971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7f0cde640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7ec1023b0 0x7fc7ec197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7f0cde640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ec102d70 0x7fc7ec19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7f0cde640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec19ece0 con 0x7fc7ec102d70 2024-02-26T04:55:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7f0cde640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec19ee50 con 0x7fc7ec1023b0 2024-02-26T04:55:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7f0cde640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec19f010 con 0x7fc7ec1011c0 2024-02-26T04:55:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7ead9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ec102d70 0x7fc7ec19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7ea59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7ec1011c0 0x7fc7ec1971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7ead9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ec102d70 0x7fc7ec19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35860/0 (socket says 172.21.15.60:35860) 2024-02-26T04:55:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7ead9d640 1 -- 172.21.15.60:0/1222819239 learned_addr learned my addr 172.21.15.60:0/1222819239 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7e9d9b640 1 --2- 172.21.15.60:0/1222819239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7ec1023b0 0x7fc7ec197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7ead9d640 1 -- 172.21.15.60:0/1222819239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7ec1011c0 msgr2=0x7fc7ec1971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7ead9d640 1 --2- 172.21.15.60:0/1222819239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7ec1011c0 0x7fc7ec1971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7ead9d640 1 -- 172.21.15.60:0/1222819239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7ec1023b0 msgr2=0x7fc7ec197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7ead9d640 1 --2- 172.21.15.60:0/1222819239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7ec1023b0 0x7fc7ec197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7ead9d640 1 -- 172.21.15.60:0/1222819239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc009900 con 0x7fc7ec102d70 2024-02-26T04:55:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7ead9d640 1 --2- 172.21.15.60:0/1222819239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ec102d70 0x7fc7ec19e7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc037040 tx=0x7fc7dc034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.523+0000 7fc7e37fe640 1 -- 172.21.15.60:0/1222819239 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc0329b0 con 0x7fc7ec102d70 2024-02-26T04:55:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.524+0000 7fc7e37fe640 1 -- 172.21.15.60:0/1222819239 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7dc003e50 con 0x7fc7ec102d70 2024-02-26T04:55:02.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.524+0000 7fc7f0cde640 1 -- 172.21.15.60:0/1222819239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ec19f290 con 0x7fc7ec102d70 2024-02-26T04:55:02.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.524+0000 7fc7e37fe640 1 -- 172.21.15.60:0/1222819239 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc040450 con 0x7fc7ec102d70 2024-02-26T04:55:02.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.524+0000 7fc7f0cde640 1 -- 172.21.15.60:0/1222819239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ec19f740 con 0x7fc7ec102d70 2024-02-26T04:55:02.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.525+0000 7fc7f0cde640 1 -- 172.21.15.60:0/1222819239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ec108480 con 0x7fc7ec102d70 2024-02-26T04:55:02.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.526+0000 7fc7e37fe640 1 -- 172.21.15.60:0/1222819239 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc7dc00a040 con 0x7fc7ec102d70 2024-02-26T04:55:02.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.526+0000 7fc7e37fe640 1 --2- 172.21.15.60:0/1222819239 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc7d80a20c0 0x7fc7d80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:02.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.526+0000 7fc7ea59c640 1 --2- 172.21.15.60:0/1222819239 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc7d80a20c0 0x7fc7d80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:02.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.527+0000 7fc7e37fe640 1 -- 172.21.15.60:0/1222819239 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc7dc0f06a0 con 0x7fc7ec102d70 2024-02-26T04:55:02.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.527+0000 7fc7ea59c640 1 --2- 172.21.15.60:0/1222819239 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc7d80a20c0 0x7fc7d80a4580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4005fd0 tx=0x7fc7e400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:02.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.531+0000 7fc7e37fe640 1 -- 172.21.15.60:0/1222819239 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc7dc0b9b10 con 0x7fc7ec102d70 2024-02-26T04:55:02.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.751+0000 7fc7f0cde640 1 -- 172.21.15.60:0/1222819239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7ec02ce40 con 0x7fc7ec102d70 2024-02-26T04:55:02.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.751+0000 7fc7e37fe640 1 -- 172.21.15.60:0/1222819239 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7fc7dc0040d0 con 0x7fc7ec102d70 2024-02-26T04:55:02.754 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:02.754 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-02-26T04:55:02.754 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 6 2024-02-26T04:55:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.753+0000 7fc7f0cde640 1 -- 172.21.15.60:0/1222819239 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc7d80a20c0 msgr2=0x7fc7d80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.753+0000 7fc7f0cde640 1 --2- 172.21.15.60:0/1222819239 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc7d80a20c0 0x7fc7d80a4580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4005fd0 tx=0x7fc7e400b410 comp rx=0 tx=0).stop 2024-02-26T04:55:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.753+0000 7fc7f0cde640 1 -- 172.21.15.60:0/1222819239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ec102d70 msgr2=0x7fc7ec19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.753+0000 7fc7f0cde640 1 --2- 172.21.15.60:0/1222819239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ec102d70 0x7fc7ec19e7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc037040 tx=0x7fc7dc034a90 comp rx=0 tx=0).stop 2024-02-26T04:55:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.753+0000 7fc7f0cde640 1 -- 172.21.15.60:0/1222819239 shutdown_connections 2024-02-26T04:55:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.753+0000 7fc7f0cde640 1 --2- 172.21.15.60:0/1222819239 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc7d80a20c0 0x7fc7d80a4580 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.753+0000 7fc7f0cde640 1 --2- 172.21.15.60:0/1222819239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ec102d70 0x7fc7ec19e7a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.753+0000 7fc7f0cde640 1 --2- 172.21.15.60:0/1222819239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7ec1023b0 0x7fc7ec197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:02.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.753+0000 7fc7f0cde640 1 --2- 172.21.15.60:0/1222819239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7ec1011c0 0x7fc7ec1971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:02.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.753+0000 7fc7f0cde640 1 -- 172.21.15.60:0/1222819239 >> 172.21.15.60:0/1222819239 conn(0x7fc7ec0fc730 msgr2=0x7fc7ec105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:02.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.753+0000 7fc7f0cde640 1 -- 172.21.15.60:0/1222819239 shutdown_connections 2024-02-26T04:55:02.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:02.753+0000 7fc7f0cde640 1 -- 172.21.15.60:0/1222819239 wait complete. 2024-02-26T04:55:03.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:03 smithi191 ceph-mon[36352]: Deploying daemon mds.f on smithi191 2024-02-26T04:55:03.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:03 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.156:6834/2819944290,v1:172.21.15.156:6835/2819944290] up:boot 2024-02-26T04:55:03.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:03 smithi191 ceph-mon[36352]: fsmap 5 up:standby 2024-02-26T04:55:03.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:03 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T04:55:03.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:03 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1222819239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:03.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:03 smithi156 ceph-mon[35805]: Deploying daemon mds.f on smithi191 2024-02-26T04:55:03.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:03 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.156:6834/2819944290,v1:172.21.15.156:6835/2819944290] up:boot 2024-02-26T04:55:03.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:03 smithi156 ceph-mon[35805]: fsmap 5 up:standby 2024-02-26T04:55:03.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:03 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T04:55:03.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:03 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1222819239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:03.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:03 smithi060 ceph-mon[32382]: Deploying daemon mds.f on smithi191 2024-02-26T04:55:03.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:03 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.156:6834/2819944290,v1:172.21.15.156:6835/2819944290] up:boot 2024-02-26T04:55:03.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:03 smithi060 ceph-mon[32382]: fsmap 5 up:standby 2024-02-26T04:55:03.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:03 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T04:55:03.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:03 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1222819239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:04.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:04 smithi191 ceph-mon[36352]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:04.259 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 04:55:04 smithi191 systemd[1]: Starting Ceph mds.f for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T04:55:04.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:04 smithi156 ceph-mon[35805]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:04.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:04 smithi060 ceph-mon[32382]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:04.696 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 04:55:04 smithi191 podman[66256]: 2024-02-26 04:55:04.296400159 +0000 UTC m=+0.017638378 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T04:55:04.696 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 04:55:04 smithi191 podman[66256]: 2024-02-26 04:55:04.43252565 +0000 UTC m=+0.153763864 container create 6fac6472ec87cc4a0b5b0923132f520557e1f6c2da816ae7a648bb6143dffdcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-f, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-26T04:55:04.696 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 04:55:04 smithi191 podman[66256]: 2024-02-26 04:55:04.694553791 +0000 UTC m=+0.415792022 container init 6fac6472ec87cc4a0b5b0923132f520557e1f6c2da816ae7a648bb6143dffdcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-26T04:55:04.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:55:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.883+0000 7fd20bb99640 1 -- 172.21.15.60:0/3824981681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd204101ca0 msgr2=0x7fd204104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.883+0000 7fd20bb99640 1 --2- 172.21.15.60:0/3824981681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd204101ca0 0x7fd204104090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd200009900 tx=0x7fd20002bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.883+0000 7fd20bb99640 1 -- 172.21.15.60:0/3824981681 shutdown_connections 2024-02-26T04:55:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.883+0000 7fd20bb99640 1 --2- 172.21.15.60:0/3824981681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204106f00 0x7fd2041092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.883+0000 7fd20bb99640 1 --2- 172.21.15.60:0/3824981681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2041045d0 0x7fd2041069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.883+0000 7fd20bb99640 1 --2- 172.21.15.60:0/3824981681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd204101ca0 0x7fd204104090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.883+0000 7fd20bb99640 1 -- 172.21.15.60:0/3824981681 >> 172.21.15.60:0/3824981681 conn(0x7fd2040fb630 msgr2=0x7fd2040fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.883+0000 7fd20bb99640 1 -- 172.21.15.60:0/3824981681 shutdown_connections 2024-02-26T04:55:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.883+0000 7fd20bb99640 1 -- 172.21.15.60:0/3824981681 wait complete. 2024-02-26T04:55:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.884+0000 7fd20bb99640 1 Processor -- start 2024-02-26T04:55:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.884+0000 7fd20bb99640 1 -- start start 2024-02-26T04:55:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.884+0000 7fd20bb99640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204101ca0 0x7fd204101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.884+0000 7fd20bb99640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2041045d0 0x7fd2040ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.884+0000 7fd20bb99640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd204106f00 0x7fd204100190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.884+0000 7fd20bb99640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2041008f0 con 0x7fd2041045d0 2024-02-26T04:55:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.884+0000 7fd20bb99640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204100a60 con 0x7fd204101ca0 2024-02-26T04:55:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.884+0000 7fd20bb99640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2041a12f0 con 0x7fd204106f00 2024-02-26T04:55:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.884+0000 7fd209134640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2041045d0 0x7fd2040ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.884+0000 7fd209935640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204101ca0 0x7fd204101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:04.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.884+0000 7fd209134640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2041045d0 0x7fd2040ffc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35882/0 (socket says 172.21.15.60:35882) 2024-02-26T04:55:04.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.884+0000 7fd209134640 1 -- 172.21.15.60:0/2518713882 learned_addr learned my addr 172.21.15.60:0/2518713882 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:04.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.884+0000 7fd20a136640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd204106f00 0x7fd204100190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:04.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.885+0000 7fd209134640 1 -- 172.21.15.60:0/2518713882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd204106f00 msgr2=0x7fd204100190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:04.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.885+0000 7fd209134640 1 --2- 172.21.15.60:0/2518713882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd204106f00 0x7fd204100190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:04.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.885+0000 7fd209134640 1 -- 172.21.15.60:0/2518713882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204101ca0 msgr2=0x7fd204101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:04.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.885+0000 7fd209134640 1 --2- 172.21.15.60:0/2518713882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204101ca0 0x7fd204101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:04.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.885+0000 7fd209134640 1 -- 172.21.15.60:0/2518713882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd200009560 con 0x7fd2041045d0 2024-02-26T04:55:04.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.885+0000 7fd209134640 1 --2- 172.21.15.60:0/2518713882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2041045d0 0x7fd2040ffc50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800e9f0 tx=0x7fd1f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.887+0000 7fd1f6ffd640 1 -- 172.21.15.60:0/2518713882 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8009700 con 0x7fd2041045d0 2024-02-26T04:55:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.887+0000 7fd1f6ffd640 1 -- 172.21.15.60:0/2518713882 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1f800cd80 con 0x7fd2041045d0 2024-02-26T04:55:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.887+0000 7fd20bb99640 1 -- 172.21.15.60:0/2518713882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2041a15a0 con 0x7fd2041045d0 2024-02-26T04:55:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.887+0000 7fd1f6ffd640 1 -- 172.21.15.60:0/2518713882 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8010690 con 0x7fd2041045d0 2024-02-26T04:55:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.887+0000 7fd20bb99640 1 -- 172.21.15.60:0/2518713882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2041a1b00 con 0x7fd2041045d0 2024-02-26T04:55:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.888+0000 7fd20bb99640 1 -- 172.21.15.60:0/2518713882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2040fd120 con 0x7fd2041045d0 2024-02-26T04:55:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.889+0000 7fd1f6ffd640 1 -- 172.21.15.60:0/2518713882 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd1f80046c0 con 0x7fd2041045d0 2024-02-26T04:55:04.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.890+0000 7fd1f6ffd640 1 --2- 172.21.15.60:0/2518713882 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd1f00a2140 0x7fd1f00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:04.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.890+0000 7fd209935640 1 --2- 172.21.15.60:0/2518713882 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd1f00a2140 0x7fd1f00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:04.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.890+0000 7fd1f6ffd640 1 -- 172.21.15.60:0/2518713882 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd1f8014070 con 0x7fd2041045d0 2024-02-26T04:55:04.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.890+0000 7fd209935640 1 --2- 172.21.15.60:0/2518713882 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd1f00a2140 0x7fd1f00a4600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd200009230 tx=0x7fd200005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:04.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:04.895+0000 7fd1f6ffd640 1 -- 172.21.15.60:0/2518713882 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd1f8096ef0 con 0x7fd2041045d0 2024-02-26T04:55:05.009 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 04:55:04 smithi191 podman[66256]: 2024-02-26 04:55:04.6970999 +0000 UTC m=+0.418338115 container start 6fac6472ec87cc4a0b5b0923132f520557e1f6c2da816ae7a648bb6143dffdcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-f, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-26T04:55:05.009 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 04:55:04 smithi191 ceph-mds[66276]: -- 172.21.15.191:0/953910067 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557475dc81a0 con 0x557475d9e800 2024-02-26T04:55:05.009 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 04:55:04 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-f[66272]: starting mds.f at 2024-02-26T04:55:05.009 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 04:55:04 smithi191 bash[66256]: 6fac6472ec87cc4a0b5b0923132f520557e1f6c2da816ae7a648bb6143dffdcb 2024-02-26T04:55:05.009 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 04:55:04 smithi191 systemd[1]: Started Ceph mds.f for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:55:05.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:05.116+0000 7fd20bb99640 1 -- 172.21.15.60:0/2518713882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd20410cbd0 con 0x7fd2041045d0 2024-02-26T04:55:05.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:05.117+0000 7fd1f6ffd640 1 -- 172.21.15.60:0/2518713882 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7fd1f809a5b0 con 0x7fd2041045d0 2024-02-26T04:55:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:05.119 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-02-26T04:55:05.119 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 6 2024-02-26T04:55:05.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:05.119+0000 7fd20bb99640 1 -- 172.21.15.60:0/2518713882 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd1f00a2140 msgr2=0x7fd1f00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:05.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:05.119+0000 7fd20bb99640 1 --2- 172.21.15.60:0/2518713882 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd1f00a2140 0x7fd1f00a4600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd200009230 tx=0x7fd200005fd0 comp rx=0 tx=0).stop 2024-02-26T04:55:05.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:05.119+0000 7fd20bb99640 1 -- 172.21.15.60:0/2518713882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2041045d0 msgr2=0x7fd2040ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:05.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:05.119+0000 7fd20bb99640 1 --2- 172.21.15.60:0/2518713882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2041045d0 0x7fd2040ffc50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800e9f0 tx=0x7fd1f800eeb0 comp rx=0 tx=0).stop 2024-02-26T04:55:05.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:05.119+0000 7fd20bb99640 1 -- 172.21.15.60:0/2518713882 shutdown_connections 2024-02-26T04:55:05.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:05.119+0000 7fd20bb99640 1 --2- 172.21.15.60:0/2518713882 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd1f00a2140 0x7fd1f00a4600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:05.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:05.119+0000 7fd20bb99640 1 --2- 172.21.15.60:0/2518713882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd204106f00 0x7fd204100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:05.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:05.119+0000 7fd20bb99640 1 --2- 172.21.15.60:0/2518713882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2041045d0 0x7fd2040ffc50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:05.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:05.119+0000 7fd20bb99640 1 --2- 172.21.15.60:0/2518713882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204101ca0 0x7fd204101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:05.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:05.119+0000 7fd20bb99640 1 -- 172.21.15.60:0/2518713882 >> 172.21.15.60:0/2518713882 conn(0x7fd2040fb630 msgr2=0x7fd2041026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:05.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:05.119+0000 7fd20bb99640 1 -- 172.21.15.60:0/2518713882 shutdown_connections 2024-02-26T04:55:05.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:05.119+0000 7fd20bb99640 1 -- 172.21.15.60:0/2518713882 wait complete. 2024-02-26T04:55:05.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:05 smithi156 ceph-mon[35805]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:05.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:05 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:05.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:05 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:05.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:05 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:05.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:05 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:05.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:05 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:05.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:05 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:05.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:05 smithi156 ceph-mon[35805]: Deploying daemon mds.c on smithi191 2024-02-26T04:55:05.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:05 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2518713882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:05.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:05 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.191:6834/1642858086,v1:172.21.15.191:6835/1642858086] up:boot 2024-02-26T04:55:05.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:05 smithi156 ceph-mon[35805]: fsmap 6 up:standby 2024-02-26T04:55:05.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:05 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T04:55:06.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:05 smithi060 ceph-mon[32382]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:06.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:05 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:06.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:05 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:06.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:05 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:06.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:05 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:06.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:05 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:06.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:05 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:06.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:05 smithi060 ceph-mon[32382]: Deploying daemon mds.c on smithi191 2024-02-26T04:55:06.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:05 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2518713882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:06.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:05 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.191:6834/1642858086,v1:172.21.15.191:6835/1642858086] up:boot 2024-02-26T04:55:06.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:05 smithi060 ceph-mon[32382]: fsmap 6 up:standby 2024-02-26T04:55:06.231 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:05 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T04:55:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:05 smithi191 ceph-mon[36352]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:05 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:05 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:05 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:05 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:05 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:05 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:05 smithi191 ceph-mon[36352]: Deploying daemon mds.c on smithi191 2024-02-26T04:55:06.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:05 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2518713882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:06.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:05 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.191:6834/1642858086,v1:172.21.15.191:6835/1642858086] up:boot 2024-02-26T04:55:06.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:05 smithi191 ceph-mon[36352]: fsmap 6 up:standby 2024-02-26T04:55:06.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:05 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T04:55:07.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:55:07.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.247+0000 7f59dbd99640 1 -- 172.21.15.60:0/849510597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d4100a40 msgr2=0x7f59d4100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:07.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.247+0000 7f59dbd99640 1 --2- 172.21.15.60:0/849510597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d4100a40 0x7f59d4100ec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009900 tx=0x7f59c402bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:07.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.247+0000 7f59dbd99640 1 -- 172.21.15.60:0/849510597 shutdown_connections 2024-02-26T04:55:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.247+0000 7f59dbd99640 1 --2- 172.21.15.60:0/849510597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59d4101400 0x7f59d4107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.247+0000 7f59dbd99640 1 --2- 172.21.15.60:0/849510597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d4100a40 0x7f59d4100ec0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.247+0000 7f59dbd99640 1 --2- 172.21.15.60:0/849510597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59d40ff850 0x7f59d40ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.247+0000 7f59dbd99640 1 -- 172.21.15.60:0/849510597 >> 172.21.15.60:0/849510597 conn(0x7f59d40fadc0 msgr2=0x7f59d40fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.247+0000 7f59dbd99640 1 -- 172.21.15.60:0/849510597 shutdown_connections 2024-02-26T04:55:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.247+0000 7f59dbd99640 1 -- 172.21.15.60:0/849510597 wait complete. 2024-02-26T04:55:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.247+0000 7f59dbd99640 1 Processor -- start 2024-02-26T04:55:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.247+0000 7f59dbd99640 1 -- start start 2024-02-26T04:55:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59dbd99640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59d40ff850 0x7f59d41994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59dbd99640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d4100a40 0x7f59d4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59dbd99640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59d4101400 0x7f59d41a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59dbd99640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d41a0fe0 con 0x7f59d4100a40 2024-02-26T04:55:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59dbd99640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d41a1150 con 0x7f59d4101400 2024-02-26T04:55:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59dbd99640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d41a1310 con 0x7f59d40ff850 2024-02-26T04:55:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59d9334640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d4100a40 0x7f59d4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59d9334640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d4100a40 0x7f59d4199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35900/0 (socket says 172.21.15.60:35900) 2024-02-26T04:55:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59d9334640 1 -- 172.21.15.60:0/2335311301 learned_addr learned my addr 172.21.15.60:0/2335311301 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59d9b35640 1 --2- 172.21.15.60:0/2335311301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59d40ff850 0x7f59d41994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59da336640 1 --2- 172.21.15.60:0/2335311301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59d4101400 0x7f59d41a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59d9334640 1 -- 172.21.15.60:0/2335311301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59d40ff850 msgr2=0x7f59d41994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59d9334640 1 --2- 172.21.15.60:0/2335311301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59d40ff850 0x7f59d41994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59d9334640 1 -- 172.21.15.60:0/2335311301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59d4101400 msgr2=0x7f59d41a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59d9334640 1 --2- 172.21.15.60:0/2335311301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59d4101400 0x7f59d41a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59d9334640 1 -- 172.21.15.60:0/2335311301 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c4009560 con 0x7f59d4100a40 2024-02-26T04:55:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59d9334640 1 --2- 172.21.15.60:0/2335311301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d4100a40 0x7f59d4199a30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f59c402c840 tx=0x7f59c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:07.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59caffd640 1 -- 172.21.15.60:0/2335311301 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4004a80 con 0x7f59d4100a40 2024-02-26T04:55:07.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59caffd640 1 -- 172.21.15.60:0/2335311301 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59c40052d0 con 0x7f59d4100a40 2024-02-26T04:55:07.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59dbd99640 1 -- 172.21.15.60:0/2335311301 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d41a1590 con 0x7f59d4100a40 2024-02-26T04:55:07.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59caffd640 1 -- 172.21.15.60:0/2335311301 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4003e20 con 0x7f59d4100a40 2024-02-26T04:55:07.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.248+0000 7f59dbd99640 1 -- 172.21.15.60:0/2335311301 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d41a1a70 con 0x7f59d4100a40 2024-02-26T04:55:07.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.249+0000 7f59dbd99640 1 -- 172.21.15.60:0/2335311301 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d410bb00 con 0x7f59d4100a40 2024-02-26T04:55:07.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.251+0000 7f59caffd640 1 -- 172.21.15.60:0/2335311301 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f59c40035f0 con 0x7f59d4100a40 2024-02-26T04:55:07.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.251+0000 7f59caffd640 1 --2- 172.21.15.60:0/2335311301 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f59c00a20c0 0x7f59c00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:07.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.251+0000 7f59d9b35640 1 --2- 172.21.15.60:0/2335311301 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f59c00a20c0 0x7f59c00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:07.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.251+0000 7f59caffd640 1 -- 172.21.15.60:0/2335311301 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f59c40f01a0 con 0x7f59d4100a40 2024-02-26T04:55:07.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.252+0000 7f59d9b35640 1 --2- 172.21.15.60:0/2335311301 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f59c00a20c0 0x7f59c00a4580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f59cc005fd0 tx=0x7f59cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:07.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.255+0000 7f59caffd640 1 -- 172.21.15.60:0/2335311301 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f59c40b9610 con 0x7f59d4100a40 2024-02-26T04:55:07.259 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 04:55:07 smithi191 systemd[1]: Starting Ceph mds.c for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T04:55:07.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.477+0000 7f59dbd99640 1 -- 172.21.15.60:0/2335311301 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59d41a1c90 con 0x7f59d4100a40 2024-02-26T04:55:07.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.478+0000 7f59caffd640 1 -- 172.21.15.60:0/2335311301 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5286 (secure 0 0 0) 0x7f59c40bccd0 con 0x7f59d4100a40 2024-02-26T04:55:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:07.480 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-02-26T04:55:07.481 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 7 2024-02-26T04:55:07.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.479+0000 7f59dbd99640 1 -- 172.21.15.60:0/2335311301 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f59c00a20c0 msgr2=0x7f59c00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:07.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.480+0000 7f59dbd99640 1 --2- 172.21.15.60:0/2335311301 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f59c00a20c0 0x7f59c00a4580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f59cc005fd0 tx=0x7f59cc00b410 comp rx=0 tx=0).stop 2024-02-26T04:55:07.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.480+0000 7f59dbd99640 1 -- 172.21.15.60:0/2335311301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d4100a40 msgr2=0x7f59d4199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:07.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.480+0000 7f59dbd99640 1 --2- 172.21.15.60:0/2335311301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d4100a40 0x7f59d4199a30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f59c402c840 tx=0x7f59c4004270 comp rx=0 tx=0).stop 2024-02-26T04:55:07.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.480+0000 7f59dbd99640 1 -- 172.21.15.60:0/2335311301 shutdown_connections 2024-02-26T04:55:07.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.480+0000 7f59dbd99640 1 --2- 172.21.15.60:0/2335311301 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f59c00a20c0 0x7f59c00a4580 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:07.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.480+0000 7f59dbd99640 1 --2- 172.21.15.60:0/2335311301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59d4101400 0x7f59d41a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:07.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.480+0000 7f59dbd99640 1 --2- 172.21.15.60:0/2335311301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d4100a40 0x7f59d4199a30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:07.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.480+0000 7f59dbd99640 1 --2- 172.21.15.60:0/2335311301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59d40ff850 0x7f59d41994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:07.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.480+0000 7f59dbd99640 1 -- 172.21.15.60:0/2335311301 >> 172.21.15.60:0/2335311301 conn(0x7f59d40fadc0 msgr2=0x7f59d40fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:07.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.480+0000 7f59dbd99640 1 -- 172.21.15.60:0/2335311301 shutdown_connections 2024-02-26T04:55:07.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:07.480+0000 7f59dbd99640 1 -- 172.21.15.60:0/2335311301 wait complete. 2024-02-26T04:55:07.662 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 04:55:07 smithi191 podman[66570]: 2024-02-26 04:55:07.194505805 +0000 UTC m=+0.017460390 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T04:55:07.662 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 04:55:07 smithi191 podman[66570]: 2024-02-26 04:55:07.319498238 +0000 UTC m=+0.142452799 container create 6288967d3c308315c5164fd9026c5af04a8076aa91ea73bf6c70bf329923706a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-c, org.label-schema.build-date=20240214, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-26T04:55:07.662 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 04:55:07 smithi191 podman[66570]: 2024-02-26 04:55:07.660463605 +0000 UTC m=+0.483418191 container init 6288967d3c308315c5164fd9026c5af04a8076aa91ea73bf6c70bf329923706a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-c, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T04:55:08.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:07 smithi191 ceph-mon[36352]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:08.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:07 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2335311301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:08.009 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 04:55:07 smithi191 podman[66570]: 2024-02-26 04:55:07.663099687 +0000 UTC m=+0.486054259 container start 6288967d3c308315c5164fd9026c5af04a8076aa91ea73bf6c70bf329923706a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-c, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD) 2024-02-26T04:55:08.009 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 04:55:07 smithi191 ceph-mds[66589]: -- 172.21.15.191:0/3897945081 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d3436ba1a0 con 0x55d343691000 2024-02-26T04:55:08.010 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 04:55:07 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-c[66585]: starting mds.c at 2024-02-26T04:55:08.010 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 04:55:07 smithi191 bash[66570]: 6288967d3c308315c5164fd9026c5af04a8076aa91ea73bf6c70bf329923706a 2024-02-26T04:55:08.010 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 04:55:07 smithi191 systemd[1]: Started Ceph mds.c for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:55:08.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:07 smithi156 ceph-mon[35805]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:08.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:07 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2335311301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:08.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:07 smithi060 ceph-mon[32382]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:08.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:07 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2335311301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:09.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:08 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.191:6836/3000497710,v1:172.21.15.191:6837/3000497710] up:boot 2024-02-26T04:55:09.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:08 smithi156 ceph-mon[35805]: fsmap 7 up:standby 2024-02-26T04:55:09.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:08 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T04:55:09.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:08 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:09.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:08 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:09.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:08 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:09.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:08 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:09.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:08 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:09.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:08 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:09.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:08 smithi156 ceph-mon[35805]: Deploying daemon mds.h on smithi156 2024-02-26T04:55:09.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:08 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.191:6836/3000497710,v1:172.21.15.191:6837/3000497710] up:boot 2024-02-26T04:55:09.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:08 smithi060 ceph-mon[32382]: fsmap 7 up:standby 2024-02-26T04:55:09.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T04:55:09.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:09.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:09.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:09.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:09.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:09.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:09.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:08 smithi060 ceph-mon[32382]: Deploying daemon mds.h on smithi156 2024-02-26T04:55:09.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:08 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.191:6836/3000497710,v1:172.21.15.191:6837/3000497710] up:boot 2024-02-26T04:55:09.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:08 smithi191 ceph-mon[36352]: fsmap 7 up:standby 2024-02-26T04:55:09.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T04:55:09.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:09.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:09.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:09.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:09.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:09.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:09.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:08 smithi191 ceph-mon[36352]: Deploying daemon mds.h on smithi156 2024-02-26T04:55:09.493 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:55:09.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.606+0000 7ff869205640 1 -- 172.21.15.60:0/3223082007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff864101cd0 msgr2=0x7ff864108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:09.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.606+0000 7ff869205640 1 --2- 172.21.15.60:0/3223082007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff864101cd0 0x7ff864108560 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009ca0 tx=0x7ff85c02c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:09.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.607+0000 7ff869205640 1 -- 172.21.15.60:0/3223082007 shutdown_connections 2024-02-26T04:55:09.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.607+0000 7ff869205640 1 --2- 172.21.15.60:0/3223082007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff864101cd0 0x7ff864108560 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:09.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.607+0000 7ff869205640 1 --2- 172.21.15.60:0/3223082007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff864101310 0x7ff864101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:09.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.607+0000 7ff869205640 1 --2- 172.21.15.60:0/3223082007 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff864100120 0x7ff864100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:09.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.607+0000 7ff869205640 1 -- 172.21.15.60:0/3223082007 >> 172.21.15.60:0/3223082007 conn(0x7ff8640fb6b0 msgr2=0x7ff8640fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:09.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.607+0000 7ff869205640 1 -- 172.21.15.60:0/3223082007 shutdown_connections 2024-02-26T04:55:09.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.607+0000 7ff869205640 1 -- 172.21.15.60:0/3223082007 wait complete. 2024-02-26T04:55:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.607+0000 7ff869205640 1 Processor -- start 2024-02-26T04:55:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff869205640 1 -- start start 2024-02-26T04:55:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff869205640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff864100120 0x7ff86406a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff869205640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff864101310 0x7ff86419bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff869205640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff864101cd0 0x7ff86419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:09.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff869205640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86419ed50 con 0x7ff864100120 2024-02-26T04:55:09.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff869205640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86419eec0 con 0x7ff864101310 2024-02-26T04:55:09.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff869205640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86419f080 con 0x7ff864101cd0 2024-02-26T04:55:09.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff862d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff864100120 0x7ff86406a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:09.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff862d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff864100120 0x7ff86406a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49910/0 (socket says 172.21.15.60:49910) 2024-02-26T04:55:09.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff862d9d640 1 -- 172.21.15.60:0/3970780363 learned_addr learned my addr 172.21.15.60:0/3970780363 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:09.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff86259c640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff864101310 0x7ff86419bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:09.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff86359e640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff864101cd0 0x7ff86419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:09.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff862d9d640 1 -- 172.21.15.60:0/3970780363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff864101cd0 msgr2=0x7ff86419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:09.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff862d9d640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff864101cd0 0x7ff86419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:09.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff862d9d640 1 -- 172.21.15.60:0/3970780363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff864101310 msgr2=0x7ff86419bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:09.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff862d9d640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff864101310 0x7ff86419bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:09.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff862d9d640 1 -- 172.21.15.60:0/3970780363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c009900 con 0x7ff864100120 2024-02-26T04:55:09.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff86359e640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff864101cd0 0x7ff86419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:09.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.608+0000 7ff862d9d640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff864100120 0x7ff86406a340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff85400b450 tx=0x7ff85400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:09.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.609+0000 7ff84bfff640 1 -- 172.21.15.60:0/3970780363 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854011890 con 0x7ff864100120 2024-02-26T04:55:09.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.609+0000 7ff869205640 1 -- 172.21.15.60:0/3970780363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff86419f360 con 0x7ff864100120 2024-02-26T04:55:09.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.609+0000 7ff84bfff640 1 -- 172.21.15.60:0/3970780363 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8540103e0 con 0x7ff864100120 2024-02-26T04:55:09.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.609+0000 7ff84bfff640 1 -- 172.21.15.60:0/3970780363 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85400f580 con 0x7ff864100120 2024-02-26T04:55:09.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.609+0000 7ff869205640 1 -- 172.21.15.60:0/3970780363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff86419f820 con 0x7ff864100120 2024-02-26T04:55:09.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.610+0000 7ff869205640 1 -- 172.21.15.60:0/3970780363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff864107440 con 0x7ff864100120 2024-02-26T04:55:09.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.611+0000 7ff84bfff640 1 -- 172.21.15.60:0/3970780363 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff8540108e0 con 0x7ff864100120 2024-02-26T04:55:09.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.611+0000 7ff84bfff640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff84c0a20c0 0x7ff84c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:09.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.611+0000 7ff86259c640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff84c0a20c0 0x7ff84c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:09.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.611+0000 7ff84bfff640 1 -- 172.21.15.60:0/3970780363 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff8540cd5c0 con 0x7ff864100120 2024-02-26T04:55:09.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.612+0000 7ff86259c640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff84c0a20c0 0x7ff84c0a4580 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff86406a0b0 tx=0x7ff85800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.616+0000 7ff84bfff640 1 -- 172.21.15.60:0/3970780363 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff854096a30 con 0x7ff864100120 2024-02-26T04:55:09.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.836+0000 7ff869205640 1 -- 172.21.15.60:0/3970780363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff86410bcd0 con 0x7ff864100120 2024-02-26T04:55:09.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.837+0000 7ff84bfff640 1 -- 172.21.15.60:0/3970780363 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6080 (secure 0 0 0) 0x7ff85409a0f0 con 0x7ff864100120 2024-02-26T04:55:09.839 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:09.839 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-26T04:55:09.840 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-02-26T04:55:09.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.839+0000 7ff869205640 1 -- 172.21.15.60:0/3970780363 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff84c0a20c0 msgr2=0x7ff84c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:09.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.839+0000 7ff869205640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff84c0a20c0 0x7ff84c0a4580 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff86406a0b0 tx=0x7ff85800b390 comp rx=0 tx=0).stop 2024-02-26T04:55:09.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.839+0000 7ff869205640 1 -- 172.21.15.60:0/3970780363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff864100120 msgr2=0x7ff86406a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:09.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.839+0000 7ff869205640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff864100120 0x7ff86406a340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff85400b450 tx=0x7ff85400b910 comp rx=0 tx=0).stop 2024-02-26T04:55:09.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.839+0000 7ff869205640 1 -- 172.21.15.60:0/3970780363 shutdown_connections 2024-02-26T04:55:09.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.839+0000 7ff869205640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff84c0a20c0 0x7ff84c0a4580 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:09.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.839+0000 7ff869205640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff864101cd0 0x7ff86419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:09.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.839+0000 7ff869205640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff864101310 0x7ff86419bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:09.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.839+0000 7ff869205640 1 --2- 172.21.15.60:0/3970780363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff864100120 0x7ff86406a340 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:09.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.839+0000 7ff869205640 1 -- 172.21.15.60:0/3970780363 >> 172.21.15.60:0/3970780363 conn(0x7ff8640fb6b0 msgr2=0x7ff864104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:09.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.839+0000 7ff869205640 1 -- 172.21.15.60:0/3970780363 shutdown_connections 2024-02-26T04:55:09.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:09.839+0000 7ff869205640 1 -- 172.21.15.60:0/3970780363 wait complete. 2024-02-26T04:55:10.180 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:09 smithi156 ceph-mon[35805]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:10.180 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:09 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3970780363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:10.181 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 04:55:10 smithi156 systemd[1]: Starting Ceph mds.h for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T04:55:10.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:09 smithi060 ceph-mon[32382]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:10.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:09 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3970780363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:10.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:09 smithi191 ceph-mon[36352]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:10.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:09 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3970780363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:10.546 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 04:55:10 smithi156 podman[65540]: 2024-02-26 04:55:10.280352185 +0000 UTC m=+0.018135796 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T04:55:10.546 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 04:55:10 smithi156 podman[65540]: 2024-02-26 04:55:10.420731315 +0000 UTC m=+0.158514905 container create d4a9ec314a770eb5bd26be64f11baec6f6865eea3fd7b6f1b17a5107ba69346a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-h, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-02-26T04:55:10.927 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 04:55:10 smithi156 podman[65540]: 2024-02-26 04:55:10.665584297 +0000 UTC m=+0.403367880 container init d4a9ec314a770eb5bd26be64f11baec6f6865eea3fd7b6f1b17a5107ba69346a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-h, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-26T04:55:10.928 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 04:55:10 smithi156 podman[65540]: 2024-02-26 04:55:10.668215192 +0000 UTC m=+0.405998774 container start d4a9ec314a770eb5bd26be64f11baec6f6865eea3fd7b6f1b17a5107ba69346a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-26T04:55:10.928 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 04:55:10 smithi156 ceph-mds[65559]: -- 172.21.15.156:0/657847137 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5591f20801a0 con 0x5591f2057000 2024-02-26T04:55:10.928 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 04:55:10 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-h[65555]: starting mds.h at 2024-02-26T04:55:10.928 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 04:55:10 smithi156 bash[65540]: d4a9ec314a770eb5bd26be64f11baec6f6865eea3fd7b6f1b17a5107ba69346a 2024-02-26T04:55:10.928 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 04:55:10 smithi156 systemd[1]: Started Ceph mds.h for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:55:11.853 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:55:11.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.958+0000 7f3eaeade640 1 -- 172.21.15.60:0/3548727486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea81012a0 msgr2=0x7f3ea8101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.958+0000 7f3eaeade640 1 --2- 172.21.15.60:0/3548727486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea81012a0 0x7f3ea8101720 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009900 tx=0x7f3e9002bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.958+0000 7f3eaeade640 1 -- 172.21.15.60:0/3548727486 shutdown_connections 2024-02-26T04:55:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.958+0000 7f3eaeade640 1 --2- 172.21.15.60:0/3548727486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ea8101c60 0x7f3ea81084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.958+0000 7f3eaeade640 1 --2- 172.21.15.60:0/3548727486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea81012a0 0x7f3ea8101720 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.958+0000 7f3eaeade640 1 --2- 172.21.15.60:0/3548727486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ea81000b0 0x7f3ea81004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.958+0000 7f3eaeade640 1 -- 172.21.15.60:0/3548727486 >> 172.21.15.60:0/3548727486 conn(0x7f3ea80fb660 msgr2=0x7f3ea80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.958+0000 7f3eaeade640 1 -- 172.21.15.60:0/3548727486 shutdown_connections 2024-02-26T04:55:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.958+0000 7f3eaeade640 1 -- 172.21.15.60:0/3548727486 wait complete. 2024-02-26T04:55:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.958+0000 7f3eaeade640 1 Processor -- start 2024-02-26T04:55:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.958+0000 7f3eaeade640 1 -- start start 2024-02-26T04:55:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3eaeade640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ea81000b0 0x7f3ea806a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3eaeade640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ea81012a0 0x7f3ea819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3eaeade640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8101c60 0x7f3ea819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3eaeade640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea819ed80 con 0x7f3ea8101c60 2024-02-26T04:55:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3eaeade640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea819eef0 con 0x7f3ea81012a0 2024-02-26T04:55:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3eaeade640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea819f0b0 con 0x7f3ea81000b0 2024-02-26T04:55:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3ead07b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8101c60 0x7f3ea819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3ead07b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8101c60 0x7f3ea819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49932/0 (socket says 172.21.15.60:49932) 2024-02-26T04:55:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3ead07b640 1 -- 172.21.15.60:0/4145050660 learned_addr learned my addr 172.21.15.60:0/4145050660 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3eac87a640 1 --2- 172.21.15.60:0/4145050660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ea81000b0 0x7f3ea806a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3ea7fff640 1 --2- 172.21.15.60:0/4145050660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ea81012a0 0x7f3ea819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3ead07b640 1 -- 172.21.15.60:0/4145050660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ea81000b0 msgr2=0x7f3ea806a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3ead07b640 1 --2- 172.21.15.60:0/4145050660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ea81000b0 0x7f3ea806a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3ead07b640 1 -- 172.21.15.60:0/4145050660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ea81012a0 msgr2=0x7f3ea819bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3ead07b640 1 --2- 172.21.15.60:0/4145050660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ea81012a0 0x7f3ea819bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:11.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3ead07b640 1 -- 172.21.15.60:0/4145050660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e90009560 con 0x7f3ea8101c60 2024-02-26T04:55:11.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3ead07b640 1 --2- 172.21.15.60:0/4145050660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8101c60 0x7f3ea819e5a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800d750 tx=0x7f3e9800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:11.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3ea5ffb640 1 -- 172.21.15.60:0/4145050660 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98009bd0 con 0x7f3ea8101c60 2024-02-26T04:55:11.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3ea5ffb640 1 -- 172.21.15.60:0/4145050660 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e9800de60 con 0x7f3ea8101c60 2024-02-26T04:55:11.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3eaeade640 1 -- 172.21.15.60:0/4145050660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea819f330 con 0x7f3ea8101c60 2024-02-26T04:55:11.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.959+0000 7f3ea5ffb640 1 -- 172.21.15.60:0/4145050660 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98004e70 con 0x7f3ea8101c60 2024-02-26T04:55:11.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.960+0000 7f3eaeade640 1 -- 172.21.15.60:0/4145050660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea819f840 con 0x7f3ea8101c60 2024-02-26T04:55:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.961+0000 7f3eaeade640 1 -- 172.21.15.60:0/4145050660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea81073d0 con 0x7f3ea8101c60 2024-02-26T04:55:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.961+0000 7f3ea5ffb640 1 -- 172.21.15.60:0/4145050660 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3e98005020 con 0x7f3ea8101c60 2024-02-26T04:55:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.962+0000 7f3ea5ffb640 1 --2- 172.21.15.60:0/4145050660 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3e940a2140 0x7f3e940a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.962+0000 7f3eac87a640 1 --2- 172.21.15.60:0/4145050660 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3e940a2140 0x7f3e940a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.962+0000 7f3ea5ffb640 1 -- 172.21.15.60:0/4145050660 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3e980cd120 con 0x7f3ea8101c60 2024-02-26T04:55:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.962+0000 7f3eac87a640 1 --2- 172.21.15.60:0/4145050660 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3e940a2140 0x7f3e940a4600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8101100 tx=0x7f3ea0009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:11.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:11.966+0000 7f3ea5ffb640 1 -- 172.21.15.60:0/4145050660 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e9809a0c0 con 0x7f3ea8101c60 2024-02-26T04:55:12.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:11 smithi156 ceph-mon[35805]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:12.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:12.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:12.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:12.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:12.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:12.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:12.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:11 smithi156 ceph-mon[35805]: Deploying daemon mds.i on smithi191 2024-02-26T04:55:12.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:11 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] up:boot 2024-02-26T04:55:12.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:11 smithi156 ceph-mon[35805]: fsmap 8 up:standby 2024-02-26T04:55:12.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:11 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T04:55:12.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:12.185+0000 7f3eaeade640 1 -- 172.21.15.60:0/4145050660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ea810bc90 con 0x7f3ea8101c60 2024-02-26T04:55:12.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:12.185+0000 7f3ea5ffb640 1 -- 172.21.15.60:0/4145050660 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6871 (secure 0 0 0) 0x7f3e98094e60 con 0x7f3ea8101c60 2024-02-26T04:55:12.188 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:12.188 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-02-26T04:55:12.188 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 9 2024-02-26T04:55:12.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:12.188+0000 7f3eaeade640 1 -- 172.21.15.60:0/4145050660 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3e940a2140 msgr2=0x7f3e940a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:12.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:12.188+0000 7f3eaeade640 1 --2- 172.21.15.60:0/4145050660 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3e940a2140 0x7f3e940a4600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8101100 tx=0x7f3ea0009670 comp rx=0 tx=0).stop 2024-02-26T04:55:12.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:12.188+0000 7f3eaeade640 1 -- 172.21.15.60:0/4145050660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8101c60 msgr2=0x7f3ea819e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:12.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:12.188+0000 7f3eaeade640 1 --2- 172.21.15.60:0/4145050660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8101c60 0x7f3ea819e5a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800d750 tx=0x7f3e9800dc10 comp rx=0 tx=0).stop 2024-02-26T04:55:12.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:12.188+0000 7f3eaeade640 1 -- 172.21.15.60:0/4145050660 shutdown_connections 2024-02-26T04:55:12.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:12.188+0000 7f3eaeade640 1 --2- 172.21.15.60:0/4145050660 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3e940a2140 0x7f3e940a4600 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:12.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:12.188+0000 7f3eaeade640 1 --2- 172.21.15.60:0/4145050660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ea8101c60 0x7f3ea819e5a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:12.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:12.188+0000 7f3eaeade640 1 --2- 172.21.15.60:0/4145050660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ea81012a0 0x7f3ea819bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:12.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:12.188+0000 7f3eaeade640 1 --2- 172.21.15.60:0/4145050660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ea81000b0 0x7f3ea806a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:12.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:12.188+0000 7f3eaeade640 1 -- 172.21.15.60:0/4145050660 >> 172.21.15.60:0/4145050660 conn(0x7f3ea80fb660 msgr2=0x7f3ea81045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:12.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:12.188+0000 7f3eaeade640 1 -- 172.21.15.60:0/4145050660 shutdown_connections 2024-02-26T04:55:12.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:12.188+0000 7f3eaeade640 1 -- 172.21.15.60:0/4145050660 wait complete. 2024-02-26T04:55:12.200 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:11 smithi060 ceph-mon[32382]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:12.200 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:12.201 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:12.201 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:12.201 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:12.201 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:12.201 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:12.201 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:11 smithi060 ceph-mon[32382]: Deploying daemon mds.i on smithi191 2024-02-26T04:55:12.201 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:11 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] up:boot 2024-02-26T04:55:12.201 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:11 smithi060 ceph-mon[32382]: fsmap 8 up:standby 2024-02-26T04:55:12.201 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:11 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T04:55:12.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:11 smithi191 ceph-mon[36352]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:12.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:12.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:12.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:12.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:12.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:12.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:12.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:11 smithi191 ceph-mon[36352]: Deploying daemon mds.i on smithi191 2024-02-26T04:55:12.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:11 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] up:boot 2024-02-26T04:55:12.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:11 smithi191 ceph-mon[36352]: fsmap 8 up:standby 2024-02-26T04:55:12.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:11 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T04:55:13.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:12 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4145050660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:13.173 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:12 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4145050660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:13.174 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 04:55:12 smithi191 systemd[1]: Starting Ceph mds.i for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T04:55:13.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:12 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4145050660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:13.447 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 04:55:13 smithi191 podman[66877]: 2024-02-26 04:55:13.074439202 +0000 UTC m=+0.017868163 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T04:55:13.447 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 04:55:13 smithi191 podman[66877]: 2024-02-26 04:55:13.216375933 +0000 UTC m=+0.159804868 container create e344c5b8d29b24b33a4a3c7bcf28c619abc958aeb036f82fd53b00c316d53868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-i, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-26T04:55:13.447 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 04:55:13 smithi191 podman[66877]: 2024-02-26 04:55:13.445096134 +0000 UTC m=+0.388525072 container init e344c5b8d29b24b33a4a3c7bcf28c619abc958aeb036f82fd53b00c316d53868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-26T04:55:13.759 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 04:55:13 smithi191 podman[66877]: 2024-02-26 04:55:13.447821678 +0000 UTC m=+0.391250613 container start e344c5b8d29b24b33a4a3c7bcf28c619abc958aeb036f82fd53b00c316d53868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-i, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-26T04:55:13.765 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 04:55:13 smithi191 ceph-mds[66896]: -- 172.21.15.191:0/96001587 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c907bc61a0 con 0x55c907b9cc00 2024-02-26T04:55:13.765 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 04:55:13 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-i[66892]: starting mds.i at 2024-02-26T04:55:13.765 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 04:55:13 smithi191 bash[66877]: e344c5b8d29b24b33a4a3c7bcf28c619abc958aeb036f82fd53b00c316d53868 2024-02-26T04:55:13.765 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 04:55:13 smithi191 systemd[1]: Started Ceph mds.i for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:55:14.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:13 smithi156 ceph-mon[35805]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:14.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:13 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:14.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:13 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:14.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:13 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:14.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:13 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:14.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:13 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:14.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:13 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:14.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:55:14.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:13 smithi060 ceph-mon[32382]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:14.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:13 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:14.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:13 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:14.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:13 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:14.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:13 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:14.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:13 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:14.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:13 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:14.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:13 smithi191 ceph-mon[36352]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:14.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:13 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:14.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:13 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:14.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:13 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:14.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:13 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:14.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:13 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:14.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:13 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:14.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.304+0000 7f9682cc8640 1 -- 172.21.15.60:0/780346524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f967c101ce0 msgr2=0x7f967c1040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:14.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.304+0000 7f9682cc8640 1 --2- 172.21.15.60:0/780346524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f967c101ce0 0x7f967c1040d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9670009900 tx=0x7f967002bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:14.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.304+0000 7f9682cc8640 1 -- 172.21.15.60:0/780346524 shutdown_connections 2024-02-26T04:55:14.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.304+0000 7f9682cc8640 1 --2- 172.21.15.60:0/780346524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f967c106f40 0x7f967c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:14.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.304+0000 7f9682cc8640 1 --2- 172.21.15.60:0/780346524 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f967c104610 0x7f967c106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:14.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.304+0000 7f9682cc8640 1 --2- 172.21.15.60:0/780346524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f967c101ce0 0x7f967c1040d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:14.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.304+0000 7f9682cc8640 1 -- 172.21.15.60:0/780346524 >> 172.21.15.60:0/780346524 conn(0x7f967c0fb630 msgr2=0x7f967c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.304+0000 7f9682cc8640 1 -- 172.21.15.60:0/780346524 shutdown_connections 2024-02-26T04:55:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.304+0000 7f9682cc8640 1 -- 172.21.15.60:0/780346524 wait complete. 2024-02-26T04:55:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9682cc8640 1 Processor -- start 2024-02-26T04:55:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9682cc8640 1 -- start start 2024-02-26T04:55:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9682cc8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f967c101ce0 0x7f967c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9682cc8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f967c104610 0x7f967c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9682cc8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f967c106f40 0x7f967c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9682cc8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c19edd0 con 0x7f967c101ce0 2024-02-26T04:55:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9682cc8640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c19ef40 con 0x7f967c106f40 2024-02-26T04:55:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9682cc8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c19f100 con 0x7f967c104610 2024-02-26T04:55:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9680a64640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f967c101ce0 0x7f967c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9680a64640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f967c101ce0 0x7f967c1972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49946/0 (socket says 172.21.15.60:49946) 2024-02-26T04:55:14.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9680a64640 1 -- 172.21.15.60:0/2913980015 learned_addr learned my addr 172.21.15.60:0/2913980015 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9681265640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f967c106f40 0x7f967c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f967bfff640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f967c104610 0x7f967c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9680a64640 1 -- 172.21.15.60:0/2913980015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f967c104610 msgr2=0x7f967c197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9680a64640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f967c104610 0x7f967c197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9680a64640 1 -- 172.21.15.60:0/2913980015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f967c106f40 msgr2=0x7f967c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9680a64640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f967c106f40 0x7f967c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9680a64640 1 -- 172.21.15.60:0/2913980015 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9670009560 con 0x7f967c101ce0 2024-02-26T04:55:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.305+0000 7f9681265640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f967c106f40 0x7f967c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.306+0000 7f9680a64640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f967c101ce0 0x7f967c1972e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9670006010 tx=0x7f96700040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.306+0000 7f9679ffb640 1 -- 172.21.15.60:0/2913980015 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96700044c0 con 0x7f967c101ce0 2024-02-26T04:55:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.306+0000 7f9679ffb640 1 -- 172.21.15.60:0/2913980015 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96700052d0 con 0x7f967c101ce0 2024-02-26T04:55:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.306+0000 7f9682cc8640 1 -- 172.21.15.60:0/2913980015 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967c19f380 con 0x7f967c101ce0 2024-02-26T04:55:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.306+0000 7f9679ffb640 1 -- 172.21.15.60:0/2913980015 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670003e10 con 0x7f967c101ce0 2024-02-26T04:55:14.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.306+0000 7f9682cc8640 1 -- 172.21.15.60:0/2913980015 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967c19f890 con 0x7f967c101ce0 2024-02-26T04:55:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.307+0000 7f9682cc8640 1 -- 172.21.15.60:0/2913980015 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967c0fd120 con 0x7f967c101ce0 2024-02-26T04:55:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.308+0000 7f9679ffb640 1 -- 172.21.15.60:0/2913980015 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9670005440 con 0x7f967c101ce0 2024-02-26T04:55:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.308+0000 7f9679ffb640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f966c0a20c0 0x7f966c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:14.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.309+0000 7f9679ffb640 1 -- 172.21.15.60:0/2913980015 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f96700efc70 con 0x7f967c101ce0 2024-02-26T04:55:14.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.309+0000 7f967bfff640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f966c0a20c0 0x7f966c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:14.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.309+0000 7f967bfff640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f966c0a20c0 0x7f966c0a4580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f967c1987f0 tx=0x7f9668005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:14.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.313+0000 7f9679ffb640 1 -- 172.21.15.60:0/2913980015 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f96700b90e0 con 0x7f967c101ce0 2024-02-26T04:55:14.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.536+0000 7f9682cc8640 1 -- 172.21.15.60:0/2913980015 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f967c10cdb0 con 0x7f967c101ce0 2024-02-26T04:55:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.537+0000 7f9679ffb640 1 -- 172.21.15.60:0/2913980015 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7664 (secure 0 0 0) 0x7f967c10cdb0 con 0x7f967c101ce0 2024-02-26T04:55:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:14.540 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-02-26T04:55:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 10 2024-02-26T04:55:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.539+0000 7f9682cc8640 1 -- 172.21.15.60:0/2913980015 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f966c0a20c0 msgr2=0x7f966c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.539+0000 7f9682cc8640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f966c0a20c0 0x7f966c0a4580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f967c1987f0 tx=0x7f9668005c60 comp rx=0 tx=0).stop 2024-02-26T04:55:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.539+0000 7f9682cc8640 1 -- 172.21.15.60:0/2913980015 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f967c101ce0 msgr2=0x7f967c1972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.539+0000 7f9682cc8640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f967c101ce0 0x7f967c1972e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9670006010 tx=0x7f96700040c0 comp rx=0 tx=0).stop 2024-02-26T04:55:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.540+0000 7f9682cc8640 1 -- 172.21.15.60:0/2913980015 shutdown_connections 2024-02-26T04:55:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.540+0000 7f9682cc8640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f966c0a20c0 0x7f966c0a4580 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:14.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.540+0000 7f9682cc8640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f967c106f40 0x7f967c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:14.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.540+0000 7f9682cc8640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f967c104610 0x7f967c197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:14.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.540+0000 7f9682cc8640 1 --2- 172.21.15.60:0/2913980015 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f967c101ce0 0x7f967c1972e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:14.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.540+0000 7f9682cc8640 1 -- 172.21.15.60:0/2913980015 >> 172.21.15.60:0/2913980015 conn(0x7f967c0fb630 msgr2=0x7f967c102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:14.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.540+0000 7f9682cc8640 1 -- 172.21.15.60:0/2913980015 shutdown_connections 2024-02-26T04:55:14.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:14.540+0000 7f9682cc8640 1 -- 172.21.15.60:0/2913980015 wait complete. 2024-02-26T04:55:15.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:14 smithi156 ceph-mon[35805]: Deploying daemon mds.g on smithi060 2024-02-26T04:55:15.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:14 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] up:boot 2024-02-26T04:55:15.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:14 smithi156 ceph-mon[35805]: fsmap 9 up:standby 2024-02-26T04:55:15.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:14 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T04:55:15.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:14 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2913980015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:15.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:14 smithi060 ceph-mon[32382]: Deploying daemon mds.g on smithi060 2024-02-26T04:55:15.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:14 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] up:boot 2024-02-26T04:55:15.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:14 smithi060 ceph-mon[32382]: fsmap 9 up:standby 2024-02-26T04:55:15.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:14 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T04:55:15.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:14 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2913980015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:15.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:14 smithi191 ceph-mon[36352]: Deploying daemon mds.g on smithi060 2024-02-26T04:55:15.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:14 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] up:boot 2024-02-26T04:55:15.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:14 smithi191 ceph-mon[36352]: fsmap 9 up:standby 2024-02-26T04:55:15.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:14 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T04:55:15.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:14 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2913980015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:15.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:15 smithi060 ceph-mon[32382]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:16.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:15 smithi156 ceph-mon[35805]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:16.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:15 smithi191 ceph-mon[36352]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:16.558 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:55:16.583 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 04:55:16 smithi060 systemd[1]: Started Ceph mds.g for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:55:16.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.662+0000 7efcd020f640 1 -- 172.21.15.60:0/4246444967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc8104550 msgr2=0x7efcc8106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:16.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.662+0000 7efcd020f640 1 --2- 172.21.15.60:0/4246444967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc8104550 0x7efcc8106940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009900 tx=0x7efcbc02bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:16.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.662+0000 7efcd020f640 1 -- 172.21.15.60:0/4246444967 shutdown_connections 2024-02-26T04:55:16.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.662+0000 7efcd020f640 1 --2- 172.21.15.60:0/4246444967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcc8106e80 0x7efcc81092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:16.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.662+0000 7efcd020f640 1 --2- 172.21.15.60:0/4246444967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc8104550 0x7efcc8106940 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:16.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.662+0000 7efcd020f640 1 --2- 172.21.15.60:0/4246444967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcc8101c20 0x7efcc8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.662+0000 7efcd020f640 1 -- 172.21.15.60:0/4246444967 >> 172.21.15.60:0/4246444967 conn(0x7efcc80fb610 msgr2=0x7efcc80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.663+0000 7efcd020f640 1 -- 172.21.15.60:0/4246444967 shutdown_connections 2024-02-26T04:55:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.663+0000 7efcd020f640 1 -- 172.21.15.60:0/4246444967 wait complete. 2024-02-26T04:55:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.663+0000 7efcd020f640 1 Processor -- start 2024-02-26T04:55:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.663+0000 7efcd020f640 1 -- start start 2024-02-26T04:55:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.663+0000 7efcd020f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc8101c20 0x7efcc8197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.663+0000 7efcd020f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcc8104550 0x7efcc81977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.663+0000 7efcd020f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcc8106e80 0x7efcc819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.663+0000 7efcd020f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc819ed70 con 0x7efcc8101c20 2024-02-26T04:55:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.663+0000 7efcd020f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc819eee0 con 0x7efcc8106e80 2024-02-26T04:55:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.663+0000 7efcd020f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc819f0a0 con 0x7efcc8104550 2024-02-26T04:55:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.663+0000 7efccdfab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc8101c20 0x7efcc8197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.663+0000 7efccdfab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc8101c20 0x7efcc8197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49978/0 (socket says 172.21.15.60:49978) 2024-02-26T04:55:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.663+0000 7efccdfab640 1 -- 172.21.15.60:0/347578166 learned_addr learned my addr 172.21.15.60:0/347578166 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.664+0000 7efcce7ac640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcc8106e80 0x7efcc819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.664+0000 7efccd7aa640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcc8104550 0x7efcc81977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.664+0000 7efccdfab640 1 -- 172.21.15.60:0/347578166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcc8104550 msgr2=0x7efcc81977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.664+0000 7efccdfab640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcc8104550 0x7efcc81977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.664+0000 7efccdfab640 1 -- 172.21.15.60:0/347578166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcc8106e80 msgr2=0x7efcc819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.664+0000 7efccdfab640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcc8106e80 0x7efcc819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.664+0000 7efccdfab640 1 -- 172.21.15.60:0/347578166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc009560 con 0x7efcc8101c20 2024-02-26T04:55:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.664+0000 7efccd7aa640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcc8104550 0x7efcc81977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.664+0000 7efccdfab640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc8101c20 0x7efcc8197280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efcc400e9f0 tx=0x7efcc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.664+0000 7efcbaffd640 1 -- 172.21.15.60:0/347578166 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4009700 con 0x7efcc8101c20 2024-02-26T04:55:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.664+0000 7efcd020f640 1 -- 172.21.15.60:0/347578166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc819f380 con 0x7efcc8101c20 2024-02-26T04:55:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.664+0000 7efcd020f640 1 -- 172.21.15.60:0/347578166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc819f8e0 con 0x7efcc8101c20 2024-02-26T04:55:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.664+0000 7efcbaffd640 1 -- 172.21.15.60:0/347578166 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcc400cd80 con 0x7efcc8101c20 2024-02-26T04:55:16.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.664+0000 7efcbaffd640 1 -- 172.21.15.60:0/347578166 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4018920 con 0x7efcc8101c20 2024-02-26T04:55:16.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.665+0000 7efcd020f640 1 -- 172.21.15.60:0/347578166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc80fd100 con 0x7efcc8101c20 2024-02-26T04:55:16.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.666+0000 7efcbaffd640 1 -- 172.21.15.60:0/347578166 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7efcc4018a80 con 0x7efcc8101c20 2024-02-26T04:55:16.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.667+0000 7efcbaffd640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efcb40a20c0 0x7efcb40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:16.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.667+0000 7efccd7aa640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efcb40a20c0 0x7efcb40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:16.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.667+0000 7efcbaffd640 1 -- 172.21.15.60:0/347578166 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efcc4014070 con 0x7efcc8101c20 2024-02-26T04:55:16.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.667+0000 7efccd7aa640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efcb40a20c0 0x7efcb40a4580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efcbc036040 tx=0x7efcbc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:16.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.671+0000 7efcbaffd640 1 -- 172.21.15.60:0/347578166 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efcc4052d20 con 0x7efcc8101c20 2024-02-26T04:55:16.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.895+0000 7efcd020f640 1 -- 172.21.15.60:0/347578166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efcc810cbb0 con 0x7efcc8101c20 2024-02-26T04:55:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.895+0000 7efcbaffd640 1 -- 172.21.15.60:0/347578166 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8456 (secure 0 0 0) 0x7efcc4097160 con 0x7efcc8101c20 2024-02-26T04:55:16.899 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:16.899 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-02-26T04:55:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 11 2024-02-26T04:55:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.897+0000 7efcd020f640 1 -- 172.21.15.60:0/347578166 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efcb40a20c0 msgr2=0x7efcb40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.897+0000 7efcd020f640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efcb40a20c0 0x7efcb40a4580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efcbc036040 tx=0x7efcbc000bc0 comp rx=0 tx=0).stop 2024-02-26T04:55:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.898+0000 7efcd020f640 1 -- 172.21.15.60:0/347578166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc8101c20 msgr2=0x7efcc8197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.898+0000 7efcd020f640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc8101c20 0x7efcc8197280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efcc400e9f0 tx=0x7efcc400eeb0 comp rx=0 tx=0).stop 2024-02-26T04:55:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.898+0000 7efcd020f640 1 -- 172.21.15.60:0/347578166 shutdown_connections 2024-02-26T04:55:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.898+0000 7efcd020f640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efcb40a20c0 0x7efcb40a4580 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.898+0000 7efcd020f640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcc8106e80 0x7efcc819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.898+0000 7efcd020f640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcc8104550 0x7efcc81977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.898+0000 7efcd020f640 1 --2- 172.21.15.60:0/347578166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc8101c20 0x7efcc8197280 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.898+0000 7efcd020f640 1 -- 172.21.15.60:0/347578166 >> 172.21.15.60:0/347578166 conn(0x7efcc80fb610 msgr2=0x7efcc81024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:16.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.898+0000 7efcd020f640 1 -- 172.21.15.60:0/347578166 shutdown_connections 2024-02-26T04:55:16.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:16.898+0000 7efcd020f640 1 -- 172.21.15.60:0/347578166 wait complete. 2024-02-26T04:55:17.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:17 smithi060 ceph-mon[32382]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:17.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:17.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:17.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:17.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:17.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:17.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:17.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:17 smithi060 ceph-mon[32382]: Deploying daemon mds.a on smithi060 2024-02-26T04:55:17.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:17 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.60:6838/2080933038,v1:172.21.15.60:6839/2080933038] up:boot 2024-02-26T04:55:17.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:17 smithi060 ceph-mon[32382]: fsmap 10 up:standby 2024-02-26T04:55:17.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:17 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T04:55:17.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:17 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/347578166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:17.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:17 smithi156 ceph-mon[35805]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:17.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:17 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:17.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:17 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:17.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:17 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:17.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:17 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:17.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:17 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:17.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:17 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:17.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:17 smithi156 ceph-mon[35805]: Deploying daemon mds.a on smithi060 2024-02-26T04:55:17.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:17 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.60:6838/2080933038,v1:172.21.15.60:6839/2080933038] up:boot 2024-02-26T04:55:17.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:17 smithi156 ceph-mon[35805]: fsmap 10 up:standby 2024-02-26T04:55:17.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:17 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T04:55:17.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:17 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/347578166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:17.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:17 smithi191 ceph-mon[36352]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:17.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:17.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:17.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:17.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:17.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:17.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:17.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:17 smithi191 ceph-mon[36352]: Deploying daemon mds.a on smithi060 2024-02-26T04:55:17.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:17 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.60:6838/2080933038,v1:172.21.15.60:6839/2080933038] up:boot 2024-02-26T04:55:17.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:17 smithi191 ceph-mon[36352]: fsmap 10 up:standby 2024-02-26T04:55:17.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:17 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-26T04:55:17.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:17 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/347578166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:18.730 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 04:55:18 smithi060 systemd[1]: Starting Ceph mds.a for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T04:55:18.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:55:19.011 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 04:55:18 smithi060 podman[71569]: 2024-02-26 04:55:18.64387932 +0000 UTC m=+0.017337783 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T04:55:19.011 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 04:55:18 smithi060 podman[71569]: 2024-02-26 04:55:18.769494702 +0000 UTC m=+0.142953145 container create b8df58cbd1aedaf43522b1b0ae3de334e41738289e5da0207cd2508fb73073a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-a, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-26T04:55:19.011 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 04:55:19 smithi060 podman[71569]: 2024-02-26 04:55:19.005713309 +0000 UTC m=+0.379171765 container init b8df58cbd1aedaf43522b1b0ae3de334e41738289e5da0207cd2508fb73073a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-02-26T04:55:19.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.018+0000 7f8c7e508640 1 -- 172.21.15.60:0/3262050775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c781000a0 msgr2=0x7f8c781004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:19.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.018+0000 7f8c7e508640 1 --2- 172.21.15.60:0/3262050775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c781000a0 0x7f8c781004c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009900 tx=0x7f8c6802bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:19.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.018+0000 7f8c7e508640 1 -- 172.21.15.60:0/3262050775 shutdown_connections 2024-02-26T04:55:19.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.018+0000 7f8c7e508640 1 --2- 172.21.15.60:0/3262050775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c78101c50 0x7f8c781084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:19.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.018+0000 7f8c7e508640 1 --2- 172.21.15.60:0/3262050775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c78101290 0x7f8c78101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:19.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.018+0000 7f8c7e508640 1 --2- 172.21.15.60:0/3262050775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c781000a0 0x7f8c781004c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:19.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.018+0000 7f8c7e508640 1 -- 172.21.15.60:0/3262050775 >> 172.21.15.60:0/3262050775 conn(0x7f8c780fb610 msgr2=0x7f8c780fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:19.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.018+0000 7f8c7e508640 1 -- 172.21.15.60:0/3262050775 shutdown_connections 2024-02-26T04:55:19.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.018+0000 7f8c7e508640 1 -- 172.21.15.60:0/3262050775 wait complete. 2024-02-26T04:55:19.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.019+0000 7f8c7e508640 1 Processor -- start 2024-02-26T04:55:19.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.019+0000 7f8c7e508640 1 -- start start 2024-02-26T04:55:19.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.019+0000 7f8c7e508640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c781000a0 0x7f8c78068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:19.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.019+0000 7f8c7e508640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c78101290 0x7f8c7819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:19.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.019+0000 7f8c7e508640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c78101c50 0x7f8c7819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:19.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.019+0000 7f8c7e508640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7819ed40 con 0x7f8c78101290 2024-02-26T04:55:19.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.019+0000 7f8c7e508640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7819eeb0 con 0x7f8c78101c50 2024-02-26T04:55:19.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.019+0000 7f8c7e508640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7819f070 con 0x7f8c781000a0 2024-02-26T04:55:19.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.019+0000 7f8c777fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c78101290 0x7f8c7819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:19.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.019+0000 7f8c777fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c78101290 0x7f8c7819bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49996/0 (socket says 172.21.15.60:49996) 2024-02-26T04:55:19.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.019+0000 7f8c777fe640 1 -- 172.21.15.60:0/1274782538 learned_addr learned my addr 172.21.15.60:0/1274782538 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:19.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.020+0000 7f8c77fff640 1 --2- 172.21.15.60:0/1274782538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c781000a0 0x7f8c78068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:19.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.020+0000 7f8c777fe640 1 -- 172.21.15.60:0/1274782538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c781000a0 msgr2=0x7f8c78068e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:19.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.020+0000 7f8c777fe640 1 --2- 172.21.15.60:0/1274782538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c781000a0 0x7f8c78068e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:19.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.020+0000 7f8c777fe640 1 -- 172.21.15.60:0/1274782538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c78101c50 msgr2=0x7f8c7819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:19.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.020+0000 7f8c777fe640 1 --2- 172.21.15.60:0/1274782538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c78101c50 0x7f8c7819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:19.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.020+0000 7f8c777fe640 1 -- 172.21.15.60:0/1274782538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c68009560 con 0x7f8c78101290 2024-02-26T04:55:19.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.020+0000 7f8c777fe640 1 --2- 172.21.15.60:0/1274782538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c78101290 0x7f8c7819bf70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00d950 tx=0x7f8c6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:19.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.020+0000 7f8c757fa640 1 -- 172.21.15.60:0/1274782538 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c009c30 con 0x7f8c78101290 2024-02-26T04:55:19.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.020+0000 7f8c7e508640 1 -- 172.21.15.60:0/1274782538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7819f350 con 0x7f8c78101290 2024-02-26T04:55:19.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.020+0000 7f8c757fa640 1 -- 172.21.15.60:0/1274782538 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c6c009d90 con 0x7f8c78101290 2024-02-26T04:55:19.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.020+0000 7f8c757fa640 1 -- 172.21.15.60:0/1274782538 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c00f7e0 con 0x7f8c78101290 2024-02-26T04:55:19.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.020+0000 7f8c7e508640 1 -- 172.21.15.60:0/1274782538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7819f8b0 con 0x7f8c78101290 2024-02-26T04:55:19.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.021+0000 7f8c7e508640 1 -- 172.21.15.60:0/1274782538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c78107440 con 0x7f8c78101290 2024-02-26T04:55:19.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.022+0000 7f8c757fa640 1 -- 172.21.15.60:0/1274782538 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8c6c010ad0 con 0x7f8c78101290 2024-02-26T04:55:19.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.022+0000 7f8c757fa640 1 --2- 172.21.15.60:0/1274782538 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8c600a20c0 0x7f8c600a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:19.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.023+0000 7f8c77fff640 1 --2- 172.21.15.60:0/1274782538 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8c600a20c0 0x7f8c600a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:19.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.023+0000 7f8c757fa640 1 -- 172.21.15.60:0/1274782538 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8c6c0cd5d0 con 0x7f8c78101290 2024-02-26T04:55:19.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.023+0000 7f8c77fff640 1 --2- 172.21.15.60:0/1274782538 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8c600a20c0 0x7f8c600a4580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8c6802c6a0 tx=0x7f8c68005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:19.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.027+0000 7f8c757fa640 1 -- 172.21.15.60:0/1274782538 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c6c0969a0 con 0x7f8c78101290 2024-02-26T04:55:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.256+0000 7f8c7e508640 1 -- 172.21.15.60:0/1274782538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c7810bab0 con 0x7f8c78101290 2024-02-26T04:55:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.257+0000 7f8c757fa640 1 -- 172.21.15.60:0/1274782538 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8456 (secure 0 0 0) 0x7f8c7810bab0 con 0x7f8c78101290 2024-02-26T04:55:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:19.265 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-02-26T04:55:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 11 2024-02-26T04:55:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.259+0000 7f8c7e508640 1 -- 172.21.15.60:0/1274782538 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8c600a20c0 msgr2=0x7f8c600a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.259+0000 7f8c7e508640 1 --2- 172.21.15.60:0/1274782538 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8c600a20c0 0x7f8c600a4580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8c6802c6a0 tx=0x7f8c68005fd0 comp rx=0 tx=0).stop 2024-02-26T04:55:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.259+0000 7f8c7e508640 1 -- 172.21.15.60:0/1274782538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c78101290 msgr2=0x7f8c7819bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.259+0000 7f8c7e508640 1 --2- 172.21.15.60:0/1274782538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c78101290 0x7f8c7819bf70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00d950 tx=0x7f8c6c00de10 comp rx=0 tx=0).stop 2024-02-26T04:55:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.259+0000 7f8c7e508640 1 -- 172.21.15.60:0/1274782538 shutdown_connections 2024-02-26T04:55:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.259+0000 7f8c7e508640 1 --2- 172.21.15.60:0/1274782538 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8c600a20c0 0x7f8c600a4580 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.259+0000 7f8c7e508640 1 --2- 172.21.15.60:0/1274782538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c78101c50 0x7f8c7819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.259+0000 7f8c7e508640 1 --2- 172.21.15.60:0/1274782538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c78101290 0x7f8c7819bf70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.259+0000 7f8c7e508640 1 --2- 172.21.15.60:0/1274782538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c781000a0 0x7f8c78068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.259+0000 7f8c7e508640 1 -- 172.21.15.60:0/1274782538 >> 172.21.15.60:0/1274782538 conn(0x7f8c780fb610 msgr2=0x7f8c78104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.259+0000 7f8c7e508640 1 -- 172.21.15.60:0/1274782538 shutdown_connections 2024-02-26T04:55:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:19.259+0000 7f8c7e508640 1 -- 172.21.15.60:0/1274782538 wait complete. 2024-02-26T04:55:19.274 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 04:55:19 smithi060 podman[71569]: 2024-02-26 04:55:19.009009673 +0000 UTC m=+0.382468120 container start b8df58cbd1aedaf43522b1b0ae3de334e41738289e5da0207cd2508fb73073a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214) 2024-02-26T04:55:19.274 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 04:55:19 smithi060 ceph-mds[71608]: -- 172.21.15.60:0/4259040395 <== mon.0 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f95d5621a0 con 0x55f95d539000 2024-02-26T04:55:19.274 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 04:55:19 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-a[71584]: starting mds.a at 2024-02-26T04:55:19.274 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 04:55:19 smithi060 bash[71569]: b8df58cbd1aedaf43522b1b0ae3de334e41738289e5da0207cd2508fb73073a2 2024-02-26T04:55:19.274 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 04:55:19 smithi060 systemd[1]: Started Ceph mds.a for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:55:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:20 smithi156 ceph-mon[35805]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:20.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:20.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:20.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:20 smithi156 ceph-mon[35805]: Deploying daemon mds.b on smithi156 2024-02-26T04:55:20.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:20 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1274782538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:20.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:20 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.60:6840/1896690404,v1:172.21.15.60:6841/1896690404] up:boot 2024-02-26T04:55:20.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:20 smithi156 ceph-mon[35805]: fsmap 11 up:standby 2024-02-26T04:55:20.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T04:55:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:20 smithi060 ceph-mon[32382]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:20 smithi060 ceph-mon[32382]: Deploying daemon mds.b on smithi156 2024-02-26T04:55:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:20 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1274782538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:20.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:20 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.60:6840/1896690404,v1:172.21.15.60:6841/1896690404] up:boot 2024-02-26T04:55:20.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:20 smithi060 ceph-mon[32382]: fsmap 11 up:standby 2024-02-26T04:55:20.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T04:55:20.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:20 smithi191 ceph-mon[36352]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:20.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:20.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-26T04:55:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-26T04:55:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:20 smithi191 ceph-mon[36352]: Deploying daemon mds.b on smithi156 2024-02-26T04:55:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:20 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1274782538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:20 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.60:6840/1896690404,v1:172.21.15.60:6841/1896690404] up:boot 2024-02-26T04:55:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:20 smithi191 ceph-mon[36352]: fsmap 11 up:standby 2024-02-26T04:55:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-26T04:55:21.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:55:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.389+0000 7f22e2888640 1 -- 172.21.15.60:0/1363818177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc101290 msgr2=0x7f22dc101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.389+0000 7f22e2888640 1 --2- 172.21.15.60:0/1363818177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc101290 0x7f22dc101710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009900 tx=0x7f22d002bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.389+0000 7f22e2888640 1 -- 172.21.15.60:0/1363818177 shutdown_connections 2024-02-26T04:55:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.389+0000 7f22e2888640 1 --2- 172.21.15.60:0/1363818177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22dc101c50 0x7f22dc1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.389+0000 7f22e2888640 1 --2- 172.21.15.60:0/1363818177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc101290 0x7f22dc101710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.389+0000 7f22e2888640 1 --2- 172.21.15.60:0/1363818177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22dc1000a0 0x7f22dc1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:21.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.389+0000 7f22e2888640 1 -- 172.21.15.60:0/1363818177 >> 172.21.15.60:0/1363818177 conn(0x7f22dc0fb610 msgr2=0x7f22dc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:21.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.389+0000 7f22e2888640 1 -- 172.21.15.60:0/1363818177 shutdown_connections 2024-02-26T04:55:21.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.389+0000 7f22e2888640 1 -- 172.21.15.60:0/1363818177 wait complete. 2024-02-26T04:55:21.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.389+0000 7f22e2888640 1 Processor -- start 2024-02-26T04:55:21.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.389+0000 7f22e2888640 1 -- start start 2024-02-26T04:55:21.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e2888640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22dc1000a0 0x7f22dc197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:21.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e2888640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22dc101290 0x7f22dc197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:21.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e2888640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc101c50 0x7f22dc19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:21.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e2888640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc19ed40 con 0x7f22dc101c50 2024-02-26T04:55:21.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e2888640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc19eeb0 con 0x7f22dc1000a0 2024-02-26T04:55:21.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e2888640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc19f070 con 0x7f22dc101290 2024-02-26T04:55:21.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e0e25640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc101c50 0x7f22dc19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:21.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e0e25640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc101c50 0x7f22dc19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43352/0 (socket says 172.21.15.60:43352) 2024-02-26T04:55:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e0e25640 1 -- 172.21.15.60:0/1007284395 learned_addr learned my addr 172.21.15.60:0/1007284395 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22dbfff640 1 --2- 172.21.15.60:0/1007284395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22dc1000a0 0x7f22dc197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22db7fe640 1 --2- 172.21.15.60:0/1007284395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22dc101290 0x7f22dc197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e0e25640 1 -- 172.21.15.60:0/1007284395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22dc101290 msgr2=0x7f22dc197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e0e25640 1 --2- 172.21.15.60:0/1007284395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22dc101290 0x7f22dc197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e0e25640 1 -- 172.21.15.60:0/1007284395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22dc1000a0 msgr2=0x7f22dc197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e0e25640 1 --2- 172.21.15.60:0/1007284395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22dc1000a0 0x7f22dc197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e0e25640 1 -- 172.21.15.60:0/1007284395 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d0009560 con 0x7f22dc101c50 2024-02-26T04:55:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22e0e25640 1 --2- 172.21.15.60:0/1007284395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc101c50 0x7f22dc19e800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009fd0 tx=0x7f22d400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.390+0000 7f22d97fa640 1 -- 172.21.15.60:0/1007284395 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d400bea0 con 0x7f22dc101c50 2024-02-26T04:55:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.391+0000 7f22d97fa640 1 -- 172.21.15.60:0/1007284395 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22d4009940 con 0x7f22dc101c50 2024-02-26T04:55:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.391+0000 7f22e2888640 1 -- 172.21.15.60:0/1007284395 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22dc19f350 con 0x7f22dc101c50 2024-02-26T04:55:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.391+0000 7f22d97fa640 1 -- 172.21.15.60:0/1007284395 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40106c0 con 0x7f22dc101c50 2024-02-26T04:55:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.391+0000 7f22e2888640 1 -- 172.21.15.60:0/1007284395 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc19f7e0 con 0x7f22dc101c50 2024-02-26T04:55:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.392+0000 7f22e2888640 1 -- 172.21.15.60:0/1007284395 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22dc107440 con 0x7f22dc101c50 2024-02-26T04:55:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.393+0000 7f22d97fa640 1 -- 172.21.15.60:0/1007284395 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f22d4004240 con 0x7f22dc101c50 2024-02-26T04:55:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.393+0000 7f22d97fa640 1 --2- 172.21.15.60:0/1007284395 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f22c40a20c0 0x7f22c40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.393+0000 7f22dbfff640 1 --2- 172.21.15.60:0/1007284395 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f22c40a20c0 0x7f22c40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.393+0000 7f22d97fa640 1 -- 172.21.15.60:0/1007284395 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f22d40cd460 con 0x7f22dc101c50 2024-02-26T04:55:21.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.394+0000 7f22dbfff640 1 --2- 172.21.15.60:0/1007284395 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f22c40a20c0 0x7f22c40a4580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f22dc1010f0 tx=0x7f22cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:21.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.398+0000 7f22d97fa640 1 -- 172.21.15.60:0/1007284395 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f22d4096980 con 0x7f22dc101c50 2024-02-26T04:55:21.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.618+0000 7f22e2888640 1 -- 172.21.15.60:0/1007284395 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22dc10cbb0 con 0x7f22dc101c50 2024-02-26T04:55:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.619+0000 7f22d97fa640 1 -- 172.21.15.60:0/1007284395 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9248 (secure 0 0 0) 0x7f22d409a040 con 0x7f22dc101c50 2024-02-26T04:55:21.622 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:21.622 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-02-26T04:55:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 12 2024-02-26T04:55:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.620+0000 7f22e2888640 1 -- 172.21.15.60:0/1007284395 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f22c40a20c0 msgr2=0x7f22c40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.620+0000 7f22e2888640 1 --2- 172.21.15.60:0/1007284395 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f22c40a20c0 0x7f22c40a4580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f22dc1010f0 tx=0x7f22cc005f90 comp rx=0 tx=0).stop 2024-02-26T04:55:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.620+0000 7f22e2888640 1 -- 172.21.15.60:0/1007284395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc101c50 msgr2=0x7f22dc19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.620+0000 7f22e2888640 1 --2- 172.21.15.60:0/1007284395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc101c50 0x7f22dc19e800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009fd0 tx=0x7f22d400b260 comp rx=0 tx=0).stop 2024-02-26T04:55:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.621+0000 7f22e2888640 1 -- 172.21.15.60:0/1007284395 shutdown_connections 2024-02-26T04:55:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.621+0000 7f22e2888640 1 --2- 172.21.15.60:0/1007284395 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f22c40a20c0 0x7f22c40a4580 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.621+0000 7f22e2888640 1 --2- 172.21.15.60:0/1007284395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc101c50 0x7f22dc19e800 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:21.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.621+0000 7f22e2888640 1 --2- 172.21.15.60:0/1007284395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22dc101290 0x7f22dc197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:21.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.621+0000 7f22e2888640 1 --2- 172.21.15.60:0/1007284395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22dc1000a0 0x7f22dc197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:21.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.621+0000 7f22e2888640 1 -- 172.21.15.60:0/1007284395 >> 172.21.15.60:0/1007284395 conn(0x7f22dc0fb610 msgr2=0x7f22dc104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:21.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.621+0000 7f22e2888640 1 -- 172.21.15.60:0/1007284395 shutdown_connections 2024-02-26T04:55:21.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:21.621+0000 7f22e2888640 1 -- 172.21.15.60:0/1007284395 wait complete. 2024-02-26T04:55:21.666 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 04:55:21 smithi156 systemd[1]: Starting Ceph mds.b for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T04:55:21.923 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 04:55:21 smithi156 podman[65856]: 2024-02-26 04:55:21.567114231 +0000 UTC m=+0.017511716 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T04:55:21.923 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 04:55:21 smithi156 podman[65856]: 2024-02-26 04:55:21.68702354 +0000 UTC m=+0.137421058 container create 4b36e180c94762421310b7804c0bf476f24505751f536d456b34f933343e6ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-b, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-26T04:55:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:22 smithi156 ceph-mon[35805]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:22 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1007284395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:22 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:22 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:22 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:22 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:22.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:22 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:22.178 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 04:55:21 smithi156 podman[65856]: 2024-02-26 04:55:21.921669308 +0000 UTC m=+0.372066807 container init 4b36e180c94762421310b7804c0bf476f24505751f536d456b34f933343e6ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-02-26T04:55:22.178 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 04:55:21 smithi156 podman[65856]: 2024-02-26 04:55:21.924257055 +0000 UTC m=+0.374654531 container start 4b36e180c94762421310b7804c0bf476f24505751f536d456b34f933343e6ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214) 2024-02-26T04:55:22.179 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 04:55:21 smithi156 ceph-mds[65875]: -- 172.21.15.156:0/2999644008 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ddb67ee1a0 con 0x55ddb67c4800 2024-02-26T04:55:22.179 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 04:55:21 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-b[65871]: starting mds.b at 2024-02-26T04:55:22.179 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 04:55:22 smithi156 bash[65856]: 4b36e180c94762421310b7804c0bf476f24505751f536d456b34f933343e6ab8 2024-02-26T04:55:22.179 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 04:55:22 smithi156 systemd[1]: Started Ceph mds.b for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T04:55:22.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:22 smithi060 ceph-mon[32382]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:22.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:22 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1007284395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:22.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:22 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:22.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:22 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:22.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:22 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:22.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:22 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:22.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:22 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:22.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:22 smithi191 ceph-mon[36352]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:22.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:22 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1007284395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:22.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:22 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:22.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:22 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:22.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:22 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:22.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:22 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:22.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:22 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:23.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:23 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] up:boot 2024-02-26T04:55:23.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:23 smithi156 ceph-mon[35805]: fsmap 12 up:standby 2024-02-26T04:55:23.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:23 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T04:55:23.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:23 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:55:23.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:23 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] up:boot 2024-02-26T04:55:23.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:23 smithi060 ceph-mon[32382]: fsmap 12 up:standby 2024-02-26T04:55:23.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:23 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T04:55:23.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:23 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:55:23.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:23 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] up:boot 2024-02-26T04:55:23.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:23 smithi191 ceph-mon[36352]: fsmap 12 up:standby 2024-02-26T04:55:23.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:23 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T04:55:23.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:23 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:55:23.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:55:23.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.743+0000 7f9319c36640 1 -- 172.21.15.60:0/4012712189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9314101290 msgr2=0x7f9314101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:23.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.743+0000 7f9319c36640 1 --2- 172.21.15.60:0/4012712189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9314101290 0x7f9314101710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9308009900 tx=0x7f930802bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:23.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.743+0000 7f9319c36640 1 -- 172.21.15.60:0/4012712189 shutdown_connections 2024-02-26T04:55:23.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.743+0000 7f9319c36640 1 --2- 172.21.15.60:0/4012712189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9314101c50 0x7f93141084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:23.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.743+0000 7f9319c36640 1 --2- 172.21.15.60:0/4012712189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9314101290 0x7f9314101710 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:23.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.743+0000 7f9319c36640 1 --2- 172.21.15.60:0/4012712189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93141000a0 0x7f93141004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:23.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.743+0000 7f9319c36640 1 -- 172.21.15.60:0/4012712189 >> 172.21.15.60:0/4012712189 conn(0x7f93140fb610 msgr2=0x7f93140fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:23.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.743+0000 7f9319c36640 1 -- 172.21.15.60:0/4012712189 shutdown_connections 2024-02-26T04:55:23.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.743+0000 7f9319c36640 1 -- 172.21.15.60:0/4012712189 wait complete. 2024-02-26T04:55:23.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f9319c36640 1 Processor -- start 2024-02-26T04:55:23.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f9319c36640 1 -- start start 2024-02-26T04:55:23.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f9319c36640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93141000a0 0x7f9314068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:23.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f9319c36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9314101290 0x7f931419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:23.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f9319c36640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9314101c50 0x7f931419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:23.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f9319c36640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931419edb0 con 0x7f9314101290 2024-02-26T04:55:23.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f9319c36640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931419ef20 con 0x7f9314101c50 2024-02-26T04:55:23.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f9319c36640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931419f0e0 con 0x7f93141000a0 2024-02-26T04:55:23.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f9312ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9314101290 0x7f931419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:23.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f9312ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9314101290 0x7f931419bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43376/0 (socket says 172.21.15.60:43376) 2024-02-26T04:55:23.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f93137fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93141000a0 0x7f9314068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:23.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f9312ffd640 1 -- 172.21.15.60:0/286954337 learned_addr learned my addr 172.21.15.60:0/286954337 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:23.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f9313fff640 1 --2- 172.21.15.60:0/286954337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9314101c50 0x7f931419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:23.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f93137fe640 1 -- 172.21.15.60:0/286954337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9314101c50 msgr2=0x7f931419e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:23.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f93137fe640 1 --2- 172.21.15.60:0/286954337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9314101c50 0x7f931419e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:23.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f93137fe640 1 -- 172.21.15.60:0/286954337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9314101290 msgr2=0x7f931419bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:23.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f93137fe640 1 --2- 172.21.15.60:0/286954337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9314101290 0x7f931419bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:23.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f93137fe640 1 -- 172.21.15.60:0/286954337 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9308009560 con 0x7f93141000a0 2024-02-26T04:55:23.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.744+0000 7f9312ffd640 1 --2- 172.21.15.60:0/286954337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9314101290 0x7f931419bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:55:23.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.745+0000 7f93137fe640 1 --2- 172.21.15.60:0/286954337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93141000a0 0x7f9314068e50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93141010f0 tx=0x7f930400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:23.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.745+0000 7f9310ff9640 1 -- 172.21.15.60:0/286954337 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9304009c30 con 0x7f93141000a0 2024-02-26T04:55:23.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.745+0000 7f9319c36640 1 -- 172.21.15.60:0/286954337 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931419f360 con 0x7f93141000a0 2024-02-26T04:55:23.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.745+0000 7f9310ff9640 1 -- 172.21.15.60:0/286954337 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9304009d90 con 0x7f93141000a0 2024-02-26T04:55:23.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.745+0000 7f9310ff9640 1 -- 172.21.15.60:0/286954337 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930400f6b0 con 0x7f93141000a0 2024-02-26T04:55:23.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.745+0000 7f9319c36640 1 -- 172.21.15.60:0/286954337 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931419f8c0 con 0x7f93141000a0 2024-02-26T04:55:23.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.746+0000 7f9319c36640 1 -- 172.21.15.60:0/286954337 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93141073c0 con 0x7f93141000a0 2024-02-26T04:55:23.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.747+0000 7f9310ff9640 1 -- 172.21.15.60:0/286954337 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9304010460 con 0x7f93141000a0 2024-02-26T04:55:23.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.748+0000 7f9310ff9640 1 --2- 172.21.15.60:0/286954337 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f92fc0a20c0 0x7f92fc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:23.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.748+0000 7f9312ffd640 1 --2- 172.21.15.60:0/286954337 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f92fc0a20c0 0x7f92fc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:23.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.748+0000 7f9310ff9640 1 -- 172.21.15.60:0/286954337 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f93040cd7f0 con 0x7f93141000a0 2024-02-26T04:55:23.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.748+0000 7f9312ffd640 1 --2- 172.21.15.60:0/286954337 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f92fc0a20c0 0x7f92fc0a4580 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9314068bc0 tx=0x7f930802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:23.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.753+0000 7f9310ff9640 1 -- 172.21.15.60:0/286954337 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9304096cf0 con 0x7f93141000a0 2024-02-26T04:55:24.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.998+0000 7f9319c36640 1 -- 172.21.15.60:0/286954337 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f931402ce40 con 0x7f93141000a0 2024-02-26T04:55:24.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:23.999+0000 7f9310ff9640 1 -- 172.21.15.60:0/286954337 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10043 (secure 0 0 0) 0x7f930409a3b0 con 0x7f93141000a0 2024-02-26T04:55:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:24.002 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-02-26T04:55:24.002 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 13 2024-02-26T04:55:24.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.001+0000 7f9319c36640 1 -- 172.21.15.60:0/286954337 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f92fc0a20c0 msgr2=0x7f92fc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.001+0000 7f9319c36640 1 --2- 172.21.15.60:0/286954337 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f92fc0a20c0 0x7f92fc0a4580 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9314068bc0 tx=0x7f930802c7e0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.001+0000 7f9319c36640 1 -- 172.21.15.60:0/286954337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93141000a0 msgr2=0x7f9314068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.001+0000 7f9319c36640 1 --2- 172.21.15.60:0/286954337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93141000a0 0x7f9314068e50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93141010f0 tx=0x7f930400dde0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.001+0000 7f9319c36640 1 -- 172.21.15.60:0/286954337 shutdown_connections 2024-02-26T04:55:24.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.001+0000 7f9319c36640 1 --2- 172.21.15.60:0/286954337 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f92fc0a20c0 0x7f92fc0a4580 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.001+0000 7f9319c36640 1 --2- 172.21.15.60:0/286954337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9314101c50 0x7f931419e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.001+0000 7f9319c36640 1 --2- 172.21.15.60:0/286954337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9314101290 0x7f931419bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.001+0000 7f9319c36640 1 --2- 172.21.15.60:0/286954337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93141000a0 0x7f9314068e50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.001+0000 7f9319c36640 1 -- 172.21.15.60:0/286954337 >> 172.21.15.60:0/286954337 conn(0x7f93140fb610 msgr2=0x7f9314104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:24.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.001+0000 7f9319c36640 1 -- 172.21.15.60:0/286954337 shutdown_connections 2024-02-26T04:55:24.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.001+0000 7f9319c36640 1 -- 172.21.15.60:0/286954337 wait complete. 2024-02-26T04:55:24.016 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:55:24.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.117+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/329500824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf18101c20 msgr2=0x7fdf18104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.117+0000 7fdf1f1e0640 1 --2- 172.21.15.60:0/329500824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf18101c20 0x7fdf18104010 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009900 tx=0x7fdf0c02bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:24.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.117+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/329500824 shutdown_connections 2024-02-26T04:55:24.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.117+0000 7fdf1f1e0640 1 --2- 172.21.15.60:0/329500824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf18106e80 0x7fdf181092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.117+0000 7fdf1f1e0640 1 --2- 172.21.15.60:0/329500824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf18104550 0x7fdf18106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.117+0000 7fdf1f1e0640 1 --2- 172.21.15.60:0/329500824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf18101c20 0x7fdf18104010 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.117+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/329500824 >> 172.21.15.60:0/329500824 conn(0x7fdf180fb610 msgr2=0x7fdf180fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.117+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/329500824 shutdown_connections 2024-02-26T04:55:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.118+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/329500824 wait complete. 2024-02-26T04:55:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.118+0000 7fdf1f1e0640 1 Processor -- start 2024-02-26T04:55:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.118+0000 7fdf1f1e0640 1 -- start start 2024-02-26T04:55:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.118+0000 7fdf1f1e0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf18101c20 0x7fdf18101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.118+0000 7fdf1f1e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf18104550 0x7fdf180ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.118+0000 7fdf1f1e0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf18106e80 0x7fdf18100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.118+0000 7fdf1f1e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf18100870 con 0x7fdf18104550 2024-02-26T04:55:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.118+0000 7fdf1f1e0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf181009e0 con 0x7fdf18101c20 2024-02-26T04:55:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.118+0000 7fdf1f1e0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf181a1300 con 0x7fdf18106e80 2024-02-26T04:55:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.118+0000 7fdf17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf18104550 0x7fdf180ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.118+0000 7fdf17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf18104550 0x7fdf180ffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43384/0 (socket says 172.21.15.60:43384) 2024-02-26T04:55:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.118+0000 7fdf17fff640 1 -- 172.21.15.60:0/3278012709 learned_addr learned my addr 172.21.15.60:0/3278012709 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.118+0000 7fdf1d77d640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf18106e80 0x7fdf18100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.119+0000 7fdf1cf7c640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf18101c20 0x7fdf18101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.119+0000 7fdf1d77d640 1 -- 172.21.15.60:0/3278012709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf18101c20 msgr2=0x7fdf18101590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.119+0000 7fdf1d77d640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf18101c20 0x7fdf18101590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.119+0000 7fdf1d77d640 1 -- 172.21.15.60:0/3278012709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf18104550 msgr2=0x7fdf180ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.119+0000 7fdf1d77d640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf18104550 0x7fdf180ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.119+0000 7fdf1d77d640 1 -- 172.21.15.60:0/3278012709 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c009560 con 0x7fdf18106e80 2024-02-26T04:55:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.119+0000 7fdf17fff640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf18104550 0x7fdf180ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:55:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.119+0000 7fdf1d77d640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf18106e80 0x7fdf18100110 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009fd0 tx=0x7fdf1000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.119+0000 7fdf15ffb640 1 -- 172.21.15.60:0/3278012709 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1000ce20 con 0x7fdf18106e80 2024-02-26T04:55:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.119+0000 7fdf15ffb640 1 -- 172.21.15.60:0/3278012709 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf10005020 con 0x7fdf18106e80 2024-02-26T04:55:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.119+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/3278012709 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf181a15b0 con 0x7fdf18106e80 2024-02-26T04:55:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.119+0000 7fdf15ffb640 1 -- 172.21.15.60:0/3278012709 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf10010670 con 0x7fdf18106e80 2024-02-26T04:55:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.119+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/3278012709 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf181a1a70 con 0x7fdf18106e80 2024-02-26T04:55:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.120+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/3278012709 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf180fd100 con 0x7fdf18106e80 2024-02-26T04:55:24.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.121+0000 7fdf15ffb640 1 -- 172.21.15.60:0/3278012709 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fdf10005500 con 0x7fdf18106e80 2024-02-26T04:55:24.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.122+0000 7fdf15ffb640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdf080a20c0 0x7fdf080a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:24.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.122+0000 7fdf1cf7c640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdf080a20c0 0x7fdf080a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:24.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.122+0000 7fdf15ffb640 1 -- 172.21.15.60:0/3278012709 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdf10014070 con 0x7fdf18106e80 2024-02-26T04:55:24.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.122+0000 7fdf1cf7c640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdf080a20c0 0x7fdf080a4580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c02c840 tx=0x7fdf0c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:24.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.126+0000 7fdf15ffb640 1 -- 172.21.15.60:0/3278012709 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf10097440 con 0x7fdf18106e80 2024-02-26T04:55:24.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.313+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/3278012709 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf1802ce40 con 0x7fdf18106e80 2024-02-26T04:55:24.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.315+0000 7fdf15ffb640 1 -- 172.21.15.60:0/3278012709 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18454 (secure 0 0 0) 0x7fdf10005190 con 0x7fdf18106e80 2024-02-26T04:55:24.318 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:24.318 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":81,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:54:37.620492+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:55:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.317+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/3278012709 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdf080a20c0 msgr2=0x7fdf080a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.317+0000 7fdf1f1e0640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdf080a20c0 0x7fdf080a4580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c02c840 tx=0x7fdf0c000bc0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.317+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/3278012709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf18106e80 msgr2=0x7fdf18100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.317+0000 7fdf1f1e0640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf18106e80 0x7fdf18100110 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009fd0 tx=0x7fdf1000ef10 comp rx=0 tx=0).stop 2024-02-26T04:55:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.317+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/3278012709 shutdown_connections 2024-02-26T04:55:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.317+0000 7fdf1f1e0640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdf080a20c0 0x7fdf080a4580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.317+0000 7fdf1f1e0640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf18106e80 0x7fdf18100110 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.317+0000 7fdf1f1e0640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf18104550 0x7fdf180ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.317+0000 7fdf1f1e0640 1 --2- 172.21.15.60:0/3278012709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf18101c20 0x7fdf18101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.317+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/3278012709 >> 172.21.15.60:0/3278012709 conn(0x7fdf180fb610 msgr2=0x7fdf18102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.317+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/3278012709 shutdown_connections 2024-02-26T04:55:24.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.317+0000 7fdf1f1e0640 1 -- 172.21.15.60:0/3278012709 wait complete. 2024-02-26T04:55:24.330 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:24 smithi060 ceph-mon[32382]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:24.330 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.330 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.331 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.331 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.331 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.331 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.331 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.331 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:24.331 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:55:24.331 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.331 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:24 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/286954337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:24.331 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:24 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:55:24.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T04:50:44.572921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T04:55:24.333 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T04:55:24.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:24 smithi156 ceph-mon[35805]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:24.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:55:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:24 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/286954337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:24.429 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:24 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:55:24.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.436+0000 7f6032b15640 1 -- 172.21.15.60:0/3581989896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f602c106e60 msgr2=0x7f602c1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.436+0000 7f6032b15640 1 --2- 172.21.15.60:0/3581989896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f602c106e60 0x7f602c1092b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6024009ca0 tx=0x7f602402c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.436+0000 7f6032b15640 1 -- 172.21.15.60:0/3581989896 shutdown_connections 2024-02-26T04:55:24.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.436+0000 7f6032b15640 1 --2- 172.21.15.60:0/3581989896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f602c106e60 0x7f602c1092b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.436+0000 7f6032b15640 1 --2- 172.21.15.60:0/3581989896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f602c104530 0x7f602c106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.436+0000 7f6032b15640 1 --2- 172.21.15.60:0/3581989896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f602c101c00 0x7f602c103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.436+0000 7f6032b15640 1 -- 172.21.15.60:0/3581989896 >> 172.21.15.60:0/3581989896 conn(0x7f602c0fb610 msgr2=0x7f602c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:24.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.436+0000 7f6032b15640 1 -- 172.21.15.60:0/3581989896 shutdown_connections 2024-02-26T04:55:24.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.436+0000 7f6032b15640 1 -- 172.21.15.60:0/3581989896 wait complete. 2024-02-26T04:55:24.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.436+0000 7f6032b15640 1 Processor -- start 2024-02-26T04:55:24.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.436+0000 7f6032b15640 1 -- start start 2024-02-26T04:55:24.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.436+0000 7f6032b15640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f602c101c00 0x7f602c101570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:24.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.436+0000 7f6032b15640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f602c104530 0x7f602c0ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:24.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f6032b15640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f602c106e60 0x7f602c1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:24.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f6032b15640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c100850 con 0x7f602c104530 2024-02-26T04:55:24.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f6032b15640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c1009c0 con 0x7f602c106e60 2024-02-26T04:55:24.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f6032b15640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c1a1280 con 0x7f602c101c00 2024-02-26T04:55:24.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f602bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f602c104530 0x7f602c0ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:24.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f602bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f602c104530 0x7f602c0ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43392/0 (socket says 172.21.15.60:43392) 2024-02-26T04:55:24.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f602bfff640 1 -- 172.21.15.60:0/2479860481 learned_addr learned my addr 172.21.15.60:0/2479860481 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:24.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f60310b2640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f602c106e60 0x7f602c1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:24.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f60308b1640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f602c101c00 0x7f602c101570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:24.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f602bfff640 1 -- 172.21.15.60:0/2479860481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f602c101c00 msgr2=0x7f602c101570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f602bfff640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f602c101c00 0x7f602c101570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f602bfff640 1 -- 172.21.15.60:0/2479860481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f602c106e60 msgr2=0x7f602c1000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f602bfff640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f602c106e60 0x7f602c1000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f602bfff640 1 -- 172.21.15.60:0/2479860481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6024009900 con 0x7f602c104530 2024-02-26T04:55:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f60310b2640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f602c106e60 0x7f602c1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:55:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f60308b1640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f602c101c00 0x7f602c101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:55:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f602bfff640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f602c104530 0x7f602c0ffbb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f601800ea20 tx=0x7f601800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f6029ffb640 1 -- 172.21.15.60:0/2479860481 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60180096c0 con 0x7f602c104530 2024-02-26T04:55:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f6032b15640 1 -- 172.21.15.60:0/2479860481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602c1a1530 con 0x7f602c104530 2024-02-26T04:55:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f6029ffb640 1 -- 172.21.15.60:0/2479860481 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6018005500 con 0x7f602c104530 2024-02-26T04:55:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.437+0000 7f6029ffb640 1 -- 172.21.15.60:0/2479860481 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018010430 con 0x7f602c104530 2024-02-26T04:55:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.438+0000 7f6032b15640 1 -- 172.21.15.60:0/2479860481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602c1a19f0 con 0x7f602c104530 2024-02-26T04:55:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.439+0000 7f6032b15640 1 -- 172.21.15.60:0/2479860481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602c0fd100 con 0x7f602c104530 2024-02-26T04:55:24.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.440+0000 7f6029ffb640 1 -- 172.21.15.60:0/2479860481 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6018005020 con 0x7f602c104530 2024-02-26T04:55:24.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.441+0000 7f6029ffb640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f601c0a20c0 0x7f601c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:24.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.441+0000 7f6029ffb640 1 -- 172.21.15.60:0/2479860481 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6018014070 con 0x7f602c104530 2024-02-26T04:55:24.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.441+0000 7f60308b1640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f601c0a20c0 0x7f601c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:24.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.441+0000 7f60308b1640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f601c0a20c0 0x7f601c0a4580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6020009fc0 tx=0x7f6020005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:24.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.445+0000 7f6029ffb640 1 -- 172.21.15.60:0/2479860481 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6018096fd0 con 0x7f602c104530 2024-02-26T04:55:24.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:24 smithi191 ceph-mon[36352]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:24.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:24.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:55:24.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:24.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:24 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/286954337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:24.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:24 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:55:24.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.640+0000 7f6032b15640 1 -- 172.21.15.60:0/2479860481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f602c10cde0 con 0x7f602c104530 2024-02-26T04:55:24.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.641+0000 7f6029ffb640 1 -- 172.21.15.60:0/2479860481 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f601809a690 con 0x7f602c104530 2024-02-26T04:55:24.644 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T04:55:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.643+0000 7f6032b15640 1 -- 172.21.15.60:0/2479860481 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f601c0a20c0 msgr2=0x7f601c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.643+0000 7f6032b15640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f601c0a20c0 0x7f601c0a4580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6020009fc0 tx=0x7f6020005f90 comp rx=0 tx=0).stop 2024-02-26T04:55:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.643+0000 7f6032b15640 1 -- 172.21.15.60:0/2479860481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f602c104530 msgr2=0x7f602c0ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.643+0000 7f6032b15640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f602c104530 0x7f602c0ffbb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f601800ea20 tx=0x7f601800eee0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.643+0000 7f6032b15640 1 -- 172.21.15.60:0/2479860481 shutdown_connections 2024-02-26T04:55:24.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.643+0000 7f6032b15640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f601c0a20c0 0x7f601c0a4580 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.643+0000 7f6032b15640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f602c106e60 0x7f602c1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.643+0000 7f6032b15640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f602c104530 0x7f602c0ffbb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.643+0000 7f6032b15640 1 --2- 172.21.15.60:0/2479860481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f602c101c00 0x7f602c101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.643+0000 7f6032b15640 1 -- 172.21.15.60:0/2479860481 >> 172.21.15.60:0/2479860481 conn(0x7f602c0fb610 msgr2=0x7f602c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:24.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.643+0000 7f6032b15640 1 -- 172.21.15.60:0/2479860481 shutdown_connections 2024-02-26T04:55:24.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.643+0000 7f6032b15640 1 -- 172.21.15.60:0/2479860481 wait complete. 2024-02-26T04:55:24.659 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-26T04:55:24.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.763+0000 7f9344f55640 1 -- 172.21.15.60:0/2099446761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9340101310 msgr2=0x7f9340101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.763+0000 7f9344f55640 1 --2- 172.21.15.60:0/2099446761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9340101310 0x7f9340101790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9330009900 tx=0x7f933002bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:24.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.763+0000 7f9344f55640 1 -- 172.21.15.60:0/2099446761 shutdown_connections 2024-02-26T04:55:24.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.763+0000 7f9344f55640 1 --2- 172.21.15.60:0/2099446761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9340101cd0 0x7f9340108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.763+0000 7f9344f55640 1 --2- 172.21.15.60:0/2099446761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9340101310 0x7f9340101790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.763+0000 7f9344f55640 1 --2- 172.21.15.60:0/2099446761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9340100120 0x7f9340100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.763+0000 7f9344f55640 1 -- 172.21.15.60:0/2099446761 >> 172.21.15.60:0/2099446761 conn(0x7f93400fb6b0 msgr2=0x7f93400fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:24.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.763+0000 7f9344f55640 1 -- 172.21.15.60:0/2099446761 shutdown_connections 2024-02-26T04:55:24.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.763+0000 7f9344f55640 1 -- 172.21.15.60:0/2099446761 wait complete. 2024-02-26T04:55:24.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f9344f55640 1 Processor -- start 2024-02-26T04:55:24.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f9344f55640 1 -- start start 2024-02-26T04:55:24.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f9344f55640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9340100120 0x7f9340196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:24.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f9344f55640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9340101310 0x7f93401966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:24.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f9344f55640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9340101cd0 0x7f934019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:24.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f9344f55640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934019dc50 con 0x7f9340101cd0 2024-02-26T04:55:24.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f9344f55640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934019ddc0 con 0x7f9340101310 2024-02-26T04:55:24.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f9344f55640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934019df80 con 0x7f9340100120 2024-02-26T04:55:24.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f933ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9340101cd0 0x7f934019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:24.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f933ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9340101cd0 0x7f934019d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43404/0 (socket says 172.21.15.60:43404) 2024-02-26T04:55:24.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f933ed9d640 1 -- 172.21.15.60:0/4171796423 learned_addr learned my addr 172.21.15.60:0/4171796423 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:24.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f933e59c640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9340100120 0x7f9340196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:24.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f933ed9d640 1 -- 172.21.15.60:0/4171796423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9340100120 msgr2=0x7f9340196160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f933ed9d640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9340100120 0x7f9340196160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f933ed9d640 1 -- 172.21.15.60:0/4171796423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9340101310 msgr2=0x7f93401966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f933dd9b640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9340101310 0x7f93401966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:24.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f933ed9d640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9340101310 0x7f93401966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f933ed9d640 1 -- 172.21.15.60:0/4171796423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9330009560 con 0x7f9340101cd0 2024-02-26T04:55:24.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f933e59c640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9340100120 0x7f9340196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:24.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.764+0000 7f933ed9d640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9340101cd0 0x7f934019d710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9338009fd0 tx=0x7f933800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:24.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.765+0000 7f93377fe640 1 -- 172.21.15.60:0/4171796423 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933800bea0 con 0x7f9340101cd0 2024-02-26T04:55:24.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.765+0000 7f9344f55640 1 -- 172.21.15.60:0/4171796423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934019e260 con 0x7f9340101cd0 2024-02-26T04:55:24.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.765+0000 7f93377fe640 1 -- 172.21.15.60:0/4171796423 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9338009940 con 0x7f9340101cd0 2024-02-26T04:55:24.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.765+0000 7f93377fe640 1 -- 172.21.15.60:0/4171796423 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93380106c0 con 0x7f9340101cd0 2024-02-26T04:55:24.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.765+0000 7f9344f55640 1 -- 172.21.15.60:0/4171796423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934019e770 con 0x7f9340101cd0 2024-02-26T04:55:24.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.766+0000 7f9344f55640 1 -- 172.21.15.60:0/4171796423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93401074c0 con 0x7f9340101cd0 2024-02-26T04:55:24.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.767+0000 7f93377fe640 1 -- 172.21.15.60:0/4171796423 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9338004240 con 0x7f9340101cd0 2024-02-26T04:55:24.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.767+0000 7f93377fe640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f932c0a20c0 0x7f932c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:24.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.767+0000 7f933e59c640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f932c0a20c0 0x7f932c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:24.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.767+0000 7f93377fe640 1 -- 172.21.15.60:0/4171796423 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f93380cd5c0 con 0x7f9340101cd0 2024-02-26T04:55:24.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.768+0000 7f933e59c640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f932c0a20c0 0x7f932c0a4580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9340101170 tx=0x7f9328005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:24.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.772+0000 7f93377fe640 1 -- 172.21.15.60:0/4171796423 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9338096a30 con 0x7f9340101cd0 2024-02-26T04:55:24.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.954+0000 7f9344f55640 1 -- 172.21.15.60:0/4171796423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f934002cdd0 con 0x7f9340101cd0 2024-02-26T04:55:24.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.955+0000 7f93377fe640 1 -- 172.21.15.60:0/4171796423 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f933809a0f0 con 0x7f9340101cd0 2024-02-26T04:55:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:24.958 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-02-26T04:55:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:55:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-02-26T04:55:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-02-26T04:55:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:55:24.959 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-02-26T04:55:24.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.957+0000 7f9344f55640 1 -- 172.21.15.60:0/4171796423 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f932c0a20c0 msgr2=0x7f932c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.957+0000 7f9344f55640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f932c0a20c0 0x7f932c0a4580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9340101170 tx=0x7f9328005f90 comp rx=0 tx=0).stop 2024-02-26T04:55:24.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.957+0000 7f9344f55640 1 -- 172.21.15.60:0/4171796423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9340101cd0 msgr2=0x7f934019d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:24.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.957+0000 7f9344f55640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9340101cd0 0x7f934019d710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9338009fd0 tx=0x7f933800b260 comp rx=0 tx=0).stop 2024-02-26T04:55:24.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.957+0000 7f9344f55640 1 -- 172.21.15.60:0/4171796423 shutdown_connections 2024-02-26T04:55:24.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.957+0000 7f9344f55640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f932c0a20c0 0x7f932c0a4580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.957+0000 7f9344f55640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9340101cd0 0x7f934019d710 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.957+0000 7f9344f55640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9340101310 0x7f93401966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.957+0000 7f9344f55640 1 --2- 172.21.15.60:0/4171796423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9340100120 0x7f9340196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:24.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.957+0000 7f9344f55640 1 -- 172.21.15.60:0/4171796423 >> 172.21.15.60:0/4171796423 conn(0x7f93400fb6b0 msgr2=0x7f9340104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:24.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.957+0000 7f9344f55640 1 -- 172.21.15.60:0/4171796423 shutdown_connections 2024-02-26T04:55:24.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:24.957+0000 7f9344f55640 1 -- 172.21.15.60:0/4171796423 wait complete. 2024-02-26T04:55:24.971 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-26T04:55:24.971 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-26T04:55:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.077+0000 7fc78779c640 1 -- 172.21.15.60:0/1025857712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780100080 msgr2=0x7fc7801004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.077+0000 7fc78779c640 1 --2- 172.21.15.60:0/1025857712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780100080 0x7fc7801004a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc778009900 tx=0x7fc77802bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.077+0000 7fc78779c640 1 -- 172.21.15.60:0/1025857712 shutdown_connections 2024-02-26T04:55:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.077+0000 7fc78779c640 1 --2- 172.21.15.60:0/1025857712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc780101c30 0x7fc7801084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.077+0000 7fc78779c640 1 --2- 172.21.15.60:0/1025857712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc780101270 0x7fc7801016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.077+0000 7fc78779c640 1 --2- 172.21.15.60:0/1025857712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780100080 0x7fc7801004a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.077+0000 7fc78779c640 1 -- 172.21.15.60:0/1025857712 >> 172.21.15.60:0/1025857712 conn(0x7fc7800fb610 msgr2=0x7fc7800fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.078+0000 7fc78779c640 1 -- 172.21.15.60:0/1025857712 shutdown_connections 2024-02-26T04:55:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.078+0000 7fc78779c640 1 -- 172.21.15.60:0/1025857712 wait complete. 2024-02-26T04:55:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.078+0000 7fc78779c640 1 Processor -- start 2024-02-26T04:55:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.078+0000 7fc78779c640 1 -- start start 2024-02-26T04:55:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.078+0000 7fc78779c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc780100080 0x7fc780104010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.078+0000 7fc78779c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780101270 0x7fc780102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.078+0000 7fc78779c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc780101c30 0x7fc780102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.078+0000 7fc78779c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7801032f0 con 0x7fc780101270 2024-02-26T04:55:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.078+0000 7fc78779c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc780103460 con 0x7fc780101c30 2024-02-26T04:55:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.078+0000 7fc78779c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7801a1330 con 0x7fc780100080 2024-02-26T04:55:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.078+0000 7fc784d37640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780101270 0x7fc780102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.078+0000 7fc784d37640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780101270 0x7fc780102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43426/0 (socket says 172.21.15.60:43426) 2024-02-26T04:55:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.078+0000 7fc784d37640 1 -- 172.21.15.60:0/3105189170 learned_addr learned my addr 172.21.15.60:0/3105189170 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.079+0000 7fc785d39640 1 --2- 172.21.15.60:0/3105189170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc780101c30 0x7fc780102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.079+0000 7fc785538640 1 --2- 172.21.15.60:0/3105189170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc780100080 0x7fc780104010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.079+0000 7fc784d37640 1 -- 172.21.15.60:0/3105189170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc780100080 msgr2=0x7fc780104010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.079+0000 7fc784d37640 1 --2- 172.21.15.60:0/3105189170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc780100080 0x7fc780104010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.079+0000 7fc784d37640 1 -- 172.21.15.60:0/3105189170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc780101c30 msgr2=0x7fc780102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.079+0000 7fc784d37640 1 --2- 172.21.15.60:0/3105189170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc780101c30 0x7fc780102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.079+0000 7fc784d37640 1 -- 172.21.15.60:0/3105189170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc778009560 con 0x7fc780101270 2024-02-26T04:55:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.079+0000 7fc784d37640 1 --2- 172.21.15.60:0/3105189170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780101270 0x7fc780102650 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc77400d7a0 tx=0x7fc77400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.079+0000 7fc77e7fc640 1 -- 172.21.15.60:0/3105189170 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774009c30 con 0x7fc780101270 2024-02-26T04:55:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.079+0000 7fc77e7fc640 1 -- 172.21.15.60:0/3105189170 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc77400deb0 con 0x7fc780101270 2024-02-26T04:55:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.079+0000 7fc78779c640 1 -- 172.21.15.60:0/3105189170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7801a15e0 con 0x7fc780101270 2024-02-26T04:55:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.079+0000 7fc77e7fc640 1 -- 172.21.15.60:0/3105189170 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774004ea0 con 0x7fc780101270 2024-02-26T04:55:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.079+0000 7fc78779c640 1 -- 172.21.15.60:0/3105189170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7801a1aa0 con 0x7fc780101270 2024-02-26T04:55:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.080+0000 7fc78779c640 1 -- 172.21.15.60:0/3105189170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78010ba90 con 0x7fc780101270 2024-02-26T04:55:25.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.081+0000 7fc77e7fc640 1 -- 172.21.15.60:0/3105189170 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc774005020 con 0x7fc780101270 2024-02-26T04:55:25.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.082+0000 7fc77e7fc640 1 --2- 172.21.15.60:0/3105189170 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc7680a20c0 0x7fc7680a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:25.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.082+0000 7fc785538640 1 --2- 172.21.15.60:0/3105189170 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc7680a20c0 0x7fc7680a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:25.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.082+0000 7fc77e7fc640 1 -- 172.21.15.60:0/3105189170 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc7740cd230 con 0x7fc780101270 2024-02-26T04:55:25.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.082+0000 7fc785538640 1 --2- 172.21.15.60:0/3105189170 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc7680a20c0 0x7fc7680a4580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc778005ab0 tx=0x7fc778005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:25.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.086+0000 7fc77e7fc640 1 -- 172.21.15.60:0/3105189170 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc7740966b0 con 0x7fc780101270 2024-02-26T04:55:25.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:25 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3278012709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:25.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:25 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2479860481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:55:25.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:25 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4171796423' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-26T04:55:25.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:25 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3278012709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:25.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:25 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2479860481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:55:25.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:25 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4171796423' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-26T04:55:25.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:25.274+0000 7fc78779c640 1 -- 172.21.15.60:0/3105189170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc78010bc20 con 0x7fc780101270 2024-02-26T04:55:25.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:25 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3278012709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:25.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:25 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2479860481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:55:25.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:25 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4171796423' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-26T04:55:26.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.174+0000 7fc77e7fc640 1 -- 172.21.15.60:0/3105189170 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc774099d70 con 0x7fc780101270 2024-02-26T04:55:26.178 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-02-26T04:55:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.177+0000 7fc78779c640 1 -- 172.21.15.60:0/3105189170 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc7680a20c0 msgr2=0x7fc7680a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.177+0000 7fc78779c640 1 --2- 172.21.15.60:0/3105189170 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc7680a20c0 0x7fc7680a4580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc778005ab0 tx=0x7fc778005fd0 comp rx=0 tx=0).stop 2024-02-26T04:55:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.177+0000 7fc78779c640 1 -- 172.21.15.60:0/3105189170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780101270 msgr2=0x7fc780102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.177+0000 7fc78779c640 1 --2- 172.21.15.60:0/3105189170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780101270 0x7fc780102650 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc77400d7a0 tx=0x7fc77400dc60 comp rx=0 tx=0).stop 2024-02-26T04:55:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.177+0000 7fc78779c640 1 -- 172.21.15.60:0/3105189170 shutdown_connections 2024-02-26T04:55:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.177+0000 7fc78779c640 1 --2- 172.21.15.60:0/3105189170 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fc7680a20c0 0x7fc7680a4580 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.177+0000 7fc78779c640 1 --2- 172.21.15.60:0/3105189170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc780101c30 0x7fc780102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.177+0000 7fc78779c640 1 --2- 172.21.15.60:0/3105189170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc780101270 0x7fc780102650 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.177+0000 7fc78779c640 1 --2- 172.21.15.60:0/3105189170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc780100080 0x7fc780104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.177+0000 7fc78779c640 1 -- 172.21.15.60:0/3105189170 >> 172.21.15.60:0/3105189170 conn(0x7fc7800fb610 msgr2=0x7fc7800fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.177+0000 7fc78779c640 1 -- 172.21.15.60:0/3105189170 shutdown_connections 2024-02-26T04:55:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.177+0000 7fc78779c640 1 -- 172.21.15.60:0/3105189170 wait complete. 2024-02-26T04:55:26.194 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-02-26T04:55:26.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.301+0000 7f1fbc98c640 1 -- 172.21.15.60:0/1395039157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb8100090 msgr2=0x7f1fb81004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:26.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.301+0000 7f1fbc98c640 1 --2- 172.21.15.60:0/1395039157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb8100090 0x7f1fb81004b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009900 tx=0x7f1fa802bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:26.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.301+0000 7f1fbc98c640 1 -- 172.21.15.60:0/1395039157 shutdown_connections 2024-02-26T04:55:26.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.301+0000 7f1fbc98c640 1 --2- 172.21.15.60:0/1395039157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fb8101c40 0x7f1fb81084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:26.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.301+0000 7f1fbc98c640 1 --2- 172.21.15.60:0/1395039157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fb8101280 0x7f1fb8101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:26.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.301+0000 7f1fbc98c640 1 --2- 172.21.15.60:0/1395039157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb8100090 0x7f1fb81004b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:26.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.301+0000 7f1fbc98c640 1 -- 172.21.15.60:0/1395039157 >> 172.21.15.60:0/1395039157 conn(0x7f1fb80fb600 msgr2=0x7f1fb80fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:26.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.301+0000 7f1fbc98c640 1 -- 172.21.15.60:0/1395039157 shutdown_connections 2024-02-26T04:55:26.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.301+0000 7f1fbc98c640 1 -- 172.21.15.60:0/1395039157 wait complete. 2024-02-26T04:55:26.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fbc98c640 1 Processor -- start 2024-02-26T04:55:26.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fbc98c640 1 -- start start 2024-02-26T04:55:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fbc98c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fb8100090 0x7f1fb806a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fbc98c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fb8101280 0x7f1fb819bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fbc98c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb8101c40 0x7f1fb819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fbc98c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb819ed40 con 0x7f1fb8101c40 2024-02-26T04:55:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fbc98c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb819eeb0 con 0x7f1fb8100090 2024-02-26T04:55:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fbc98c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb819f070 con 0x7f1fb8101280 2024-02-26T04:55:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fb6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb8101c40 0x7f1fb819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fb6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb8101c40 0x7f1fb819e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43450/0 (socket says 172.21.15.60:43450) 2024-02-26T04:55:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fb6d9d640 1 -- 172.21.15.60:0/833253730 learned_addr learned my addr 172.21.15.60:0/833253730 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1faffff640 1 --2- 172.21.15.60:0/833253730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fb8101280 0x7f1fb819bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fb659c640 1 --2- 172.21.15.60:0/833253730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fb8100090 0x7f1fb806a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fb6d9d640 1 -- 172.21.15.60:0/833253730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fb8101280 msgr2=0x7f1fb819bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fb6d9d640 1 --2- 172.21.15.60:0/833253730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fb8101280 0x7f1fb819bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fb6d9d640 1 -- 172.21.15.60:0/833253730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fb8100090 msgr2=0x7f1fb806a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fb6d9d640 1 --2- 172.21.15.60:0/833253730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fb8100090 0x7f1fb806a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fb6d9d640 1 -- 172.21.15.60:0/833253730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa8009560 con 0x7f1fb8101c40 2024-02-26T04:55:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fb6d9d640 1 --2- 172.21.15.60:0/833253730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb8101c40 0x7f1fb819e560 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000eb10 tx=0x7f1fb000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1faf7fe640 1 -- 172.21.15.60:0/833253730 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0004b20 con 0x7f1fb8101c40 2024-02-26T04:55:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fbc98c640 1 -- 172.21.15.60:0/833253730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb819f2f0 con 0x7f1fb8101c40 2024-02-26T04:55:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1faf7fe640 1 -- 172.21.15.60:0/833253730 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fb0004c80 con 0x7f1fb8101c40 2024-02-26T04:55:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1faf7fe640 1 -- 172.21.15.60:0/833253730 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0010670 con 0x7f1fb8101c40 2024-02-26T04:55:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.303+0000 7f1fbc98c640 1 -- 172.21.15.60:0/833253730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb819f850 con 0x7f1fb8101c40 2024-02-26T04:55:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.304+0000 7f1fbc98c640 1 -- 172.21.15.60:0/833253730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb8107430 con 0x7f1fb8101c40 2024-02-26T04:55:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.305+0000 7f1faf7fe640 1 -- 172.21.15.60:0/833253730 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1fb0005020 con 0x7f1fb8101c40 2024-02-26T04:55:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.305+0000 7f1faf7fe640 1 --2- 172.21.15.60:0/833253730 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1fa40a20c0 0x7f1fa40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.306+0000 7f1fb659c640 1 --2- 172.21.15.60:0/833253730 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1fa40a20c0 0x7f1fa40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.306+0000 7f1faf7fe640 1 -- 172.21.15.60:0/833253730 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f1fb0014070 con 0x7f1fb8101c40 2024-02-26T04:55:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.306+0000 7f1fb659c640 1 --2- 172.21.15.60:0/833253730 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1fa40a20c0 0x7f1fa40a4580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8006010 tx=0x7f1fa8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:26.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.311+0000 7f1faf7fe640 1 -- 172.21.15.60:0/833253730 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1fb0097240 con 0x7f1fb8101c40 2024-02-26T04:55:26.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:26 smithi156 ceph-mon[35805]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:26.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:26 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3105189170' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T04:55:26.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:55:26.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:26 smithi060 ceph-mon[32382]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:26 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3105189170' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T04:55:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:55:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:26.492+0000 7f1fbc98c640 1 -- 172.21.15.60:0/833253730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f1fb802ce40 con 0x7f1fb8101c40 2024-02-26T04:55:26.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:26 smithi191 ceph-mon[36352]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:26.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:26 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3105189170' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-26T04:55:26.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:26.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:55:26.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:55:26.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:55:27.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.180+0000 7f1faf7fe640 1 -- 172.21.15.60:0/833253730 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) v1 ==== 171+0+0 (secure 0 0 0) 0x7f1fb009a900 con 0x7f1fb8101c40 2024-02-26T04:55:27.183 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-02-26T04:55:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.183+0000 7f1fbc98c640 1 -- 172.21.15.60:0/833253730 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1fa40a20c0 msgr2=0x7f1fa40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.183+0000 7f1fbc98c640 1 --2- 172.21.15.60:0/833253730 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1fa40a20c0 0x7f1fa40a4580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8006010 tx=0x7f1fa8000bc0 comp rx=0 tx=0).stop 2024-02-26T04:55:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.183+0000 7f1fbc98c640 1 -- 172.21.15.60:0/833253730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb8101c40 msgr2=0x7f1fb819e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.183+0000 7f1fbc98c640 1 --2- 172.21.15.60:0/833253730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb8101c40 0x7f1fb819e560 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000eb10 tx=0x7f1fb000c2d0 comp rx=0 tx=0).stop 2024-02-26T04:55:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.183+0000 7f1fbc98c640 1 -- 172.21.15.60:0/833253730 shutdown_connections 2024-02-26T04:55:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.183+0000 7f1fbc98c640 1 --2- 172.21.15.60:0/833253730 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1fa40a20c0 0x7f1fa40a4580 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.183+0000 7f1fbc98c640 1 --2- 172.21.15.60:0/833253730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb8101c40 0x7f1fb819e560 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.183+0000 7f1fbc98c640 1 --2- 172.21.15.60:0/833253730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fb8101280 0x7f1fb819bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.183+0000 7f1fbc98c640 1 --2- 172.21.15.60:0/833253730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fb8100090 0x7f1fb806a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.184+0000 7f1fbc98c640 1 -- 172.21.15.60:0/833253730 >> 172.21.15.60:0/833253730 conn(0x7f1fb80fb600 msgr2=0x7f1fb8104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.184+0000 7f1fbc98c640 1 -- 172.21.15.60:0/833253730 shutdown_connections 2024-02-26T04:55:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.184+0000 7f1fbc98c640 1 -- 172.21.15.60:0/833253730 wait complete. 2024-02-26T04:55:27.212 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-26T04:55:27.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 -- 172.21.15.60:0/1335900973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1150102e00 msgr2=0x7f1150109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:27.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 --2- 172.21.15.60:0/1335900973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1150102e00 0x7f1150109690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1148009ca0 tx=0x7f114802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:27.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 -- 172.21.15.60:0/1335900973 shutdown_connections 2024-02-26T04:55:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 --2- 172.21.15.60:0/1335900973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1150102e00 0x7f1150109690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 --2- 172.21.15.60:0/1335900973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1150102440 0x7f11501028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 --2- 172.21.15.60:0/1335900973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11500ff470 0x7f11500ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 -- 172.21.15.60:0/1335900973 >> 172.21.15.60:0/1335900973 conn(0x7f11500fad80 msgr2=0x7f11500fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 -- 172.21.15.60:0/1335900973 shutdown_connections 2024-02-26T04:55:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 -- 172.21.15.60:0/1335900973 wait complete. 2024-02-26T04:55:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 Processor -- start 2024-02-26T04:55:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 -- start start 2024-02-26T04:55:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11500ff470 0x7f1150197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1150102440 0x7f1150197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1150102e00 0x7f115019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115019ed00 con 0x7f1150102440 2024-02-26T04:55:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115019ee70 con 0x7f1150102e00 2024-02-26T04:55:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115019f030 con 0x7f11500ff470 2024-02-26T04:55:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f115543c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1150102440 0x7f1150197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f115543c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1150102440 0x7f1150197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43472/0 (socket says 172.21.15.60:43472) 2024-02-26T04:55:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f115543c640 1 -- 172.21.15.60:0/3415983884 learned_addr learned my addr 172.21.15.60:0/3415983884 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1155c3d640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11500ff470 0x7f1150197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f115643e640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1150102e00 0x7f115019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f115543c640 1 -- 172.21.15.60:0/3415983884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11500ff470 msgr2=0x7f1150197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f115543c640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11500ff470 0x7f1150197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f115543c640 1 -- 172.21.15.60:0/3415983884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1150102e00 msgr2=0x7f115019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f115543c640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1150102e00 0x7f115019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f115543c640 1 -- 172.21.15.60:0/3415983884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1148009900 con 0x7f1150102440 2024-02-26T04:55:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f115643e640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1150102e00 0x7f115019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f115543c640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1150102440 0x7f1150197750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f114000c890 tx=0x7f114000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1146ffd640 1 -- 172.21.15.60:0/3415983884 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140004220 con 0x7f1150102440 2024-02-26T04:55:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 -- 172.21.15.60:0/3415983884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115019f310 con 0x7f1150102440 2024-02-26T04:55:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1146ffd640 1 -- 172.21.15.60:0/3415983884 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1140007e90 con 0x7f1150102440 2024-02-26T04:55:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1146ffd640 1 -- 172.21.15.60:0/3415983884 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140004a00 con 0x7f1150102440 2024-02-26T04:55:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.323+0000 7f1157ea1640 1 -- 172.21.15.60:0/3415983884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115019f870 con 0x7f1150102440 2024-02-26T04:55:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.325+0000 7f1157ea1640 1 -- 172.21.15.60:0/3415983884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1150108510 con 0x7f1150102440 2024-02-26T04:55:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.325+0000 7f1146ffd640 1 -- 172.21.15.60:0/3415983884 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f11400079b0 con 0x7f1150102440 2024-02-26T04:55:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.327+0000 7f1146ffd640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f113c0a20c0 0x7f113c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.327+0000 7f1155c3d640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f113c0a20c0 0x7f113c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.327+0000 7f1146ffd640 1 -- 172.21.15.60:0/3415983884 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f11400cdcc0 con 0x7f1150102440 2024-02-26T04:55:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.328+0000 7f1155c3d640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f113c0a20c0 0x7f113c0a4580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1150069590 tx=0x7f114c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:27.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.330+0000 7f1146ffd640 1 -- 172.21.15.60:0/3415983884 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1140096da0 con 0x7f1150102440 2024-02-26T04:55:27.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:27 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3105189170' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-26T04:55:27.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:27 smithi060 ceph-mon[32382]: osdmap e82: 12 total, 12 up, 12 in 2024-02-26T04:55:27.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:27 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/833253730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-26T04:55:27.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:27 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3105189170' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-26T04:55:27.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:27 smithi191 ceph-mon[36352]: osdmap e82: 12 total, 12 up, 12 in 2024-02-26T04:55:27.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:27 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/833253730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-26T04:55:27.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:27.552+0000 7f1157ea1640 1 -- 172.21.15.60:0/3415983884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f115010cbd0 con 0x7f1150102440 2024-02-26T04:55:27.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:27 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3105189170' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-26T04:55:27.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:27 smithi156 ceph-mon[35805]: osdmap e82: 12 total, 12 up, 12 in 2024-02-26T04:55:27.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:27 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/833253730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-26T04:55:28.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.182+0000 7f1146ffd640 1 -- 172.21.15.60:0/3415983884 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f114001d020 con 0x7f1150102440 2024-02-26T04:55:28.186 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 2 and data pool 3 2024-02-26T04:55:28.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.186+0000 7f1157ea1640 1 -- 172.21.15.60:0/3415983884 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f113c0a20c0 msgr2=0x7f113c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:28.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.186+0000 7f1157ea1640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f113c0a20c0 0x7f113c0a4580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1150069590 tx=0x7f114c005f90 comp rx=0 tx=0).stop 2024-02-26T04:55:28.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.186+0000 7f1157ea1640 1 -- 172.21.15.60:0/3415983884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1150102440 msgr2=0x7f1150197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:28.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.186+0000 7f1157ea1640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1150102440 0x7f1150197750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f114000c890 tx=0x7f114000cd50 comp rx=0 tx=0).stop 2024-02-26T04:55:28.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.186+0000 7f1157ea1640 1 -- 172.21.15.60:0/3415983884 shutdown_connections 2024-02-26T04:55:28.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.186+0000 7f1157ea1640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f113c0a20c0 0x7f113c0a4580 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:28.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.186+0000 7f1157ea1640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1150102e00 0x7f115019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:28.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.186+0000 7f1157ea1640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1150102440 0x7f1150197750 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:28.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.186+0000 7f1157ea1640 1 --2- 172.21.15.60:0/3415983884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11500ff470 0x7f1150197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:28.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.186+0000 7f1157ea1640 1 -- 172.21.15.60:0/3415983884 >> 172.21.15.60:0/3415983884 conn(0x7f11500fad80 msgr2=0x7f11501056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:28.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.186+0000 7f1157ea1640 1 -- 172.21.15.60:0/3415983884 shutdown_connections 2024-02-26T04:55:28.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.186+0000 7f1157ea1640 1 -- 172.21.15.60:0/3415983884 wait complete. 2024-02-26T04:55:28.211 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-02-26T04:55:28.212 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-02-26T04:55:28.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.324+0000 7f600ba60640 1 -- 172.21.15.60:0/3141639658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6004102d60 msgr2=0x7f60041095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:28.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.324+0000 7f600ba60640 1 --2- 172.21.15.60:0/3141639658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6004102d60 0x7f60041095f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009ca0 tx=0x7f5ffc02c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:28.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.324+0000 7f600ba60640 1 -- 172.21.15.60:0/3141639658 shutdown_connections 2024-02-26T04:55:28.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.324+0000 7f600ba60640 1 --2- 172.21.15.60:0/3141639658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6004102d60 0x7f60041095f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.324+0000 7f600ba60640 1 --2- 172.21.15.60:0/3141639658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60041023a0 0x7f6004102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.324+0000 7f600ba60640 1 --2- 172.21.15.60:0/3141639658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60041011b0 0x7f60041015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.324+0000 7f600ba60640 1 -- 172.21.15.60:0/3141639658 >> 172.21.15.60:0/3141639658 conn(0x7f60040fc740 msgr2=0x7f60040feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.324+0000 7f600ba60640 1 -- 172.21.15.60:0/3141639658 shutdown_connections 2024-02-26T04:55:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.324+0000 7f600ba60640 1 -- 172.21.15.60:0/3141639658 wait complete. 2024-02-26T04:55:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.324+0000 7f600ba60640 1 Processor -- start 2024-02-26T04:55:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f600ba60640 1 -- start start 2024-02-26T04:55:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f600ba60640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60041011b0 0x7f6004197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f600ba60640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60041023a0 0x7f6004197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f600ba60640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6004102d60 0x7f600419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f600ba60640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600419ec40 con 0x7f6004102d60 2024-02-26T04:55:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f600ba60640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600419edb0 con 0x7f60041011b0 2024-02-26T04:55:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f600ba60640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600419ef70 con 0x7f60041023a0 2024-02-26T04:55:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f6009ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6004102d60 0x7f600419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f6009ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6004102d60 0x7f600419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43490/0 (socket says 172.21.15.60:43490) 2024-02-26T04:55:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f6009ffd640 1 -- 172.21.15.60:0/1057793230 learned_addr learned my addr 172.21.15.60:0/1057793230 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f60097fc640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60041011b0 0x7f6004197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f6008ffb640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60041023a0 0x7f6004197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f6009ffd640 1 -- 172.21.15.60:0/1057793230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60041023a0 msgr2=0x7f6004197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f6009ffd640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60041023a0 0x7f6004197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f6009ffd640 1 -- 172.21.15.60:0/1057793230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60041011b0 msgr2=0x7f6004197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f6009ffd640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60041011b0 0x7f6004197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f6009ffd640 1 -- 172.21.15.60:0/1057793230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc009900 con 0x7f6004102d60 2024-02-26T04:55:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f6008ffb640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60041023a0 0x7f6004197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f6009ffd640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6004102d60 0x7f600419e700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc037040 tx=0x7f5ffc034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f5ffa7fc640 1 -- 172.21.15.60:0/1057793230 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc032eb0 con 0x7f6004102d60 2024-02-26T04:55:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f5ffa7fc640 1 -- 172.21.15.60:0/1057793230 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ffc003e50 con 0x7f6004102d60 2024-02-26T04:55:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f600ba60640 1 -- 172.21.15.60:0/1057793230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600419f1f0 con 0x7f6004102d60 2024-02-26T04:55:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f5ffa7fc640 1 -- 172.21.15.60:0/1057793230 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc040450 con 0x7f6004102d60 2024-02-26T04:55:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.325+0000 7f600ba60640 1 -- 172.21.15.60:0/1057793230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600419f6f0 con 0x7f6004102d60 2024-02-26T04:55:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.326+0000 7f600ba60640 1 -- 172.21.15.60:0/1057793230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6004108400 con 0x7f6004102d60 2024-02-26T04:55:28.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.327+0000 7f5ffa7fc640 1 -- 172.21.15.60:0/1057793230 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5ffc00a040 con 0x7f6004102d60 2024-02-26T04:55:28.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.328+0000 7f5ffa7fc640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ff00a2140 0x7f5ff00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:28.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.328+0000 7f60097fc640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ff00a2140 0x7f5ff00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:28.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.328+0000 7f5ffa7fc640 1 -- 172.21.15.60:0/1057793230 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5ffc0f06d0 con 0x7f6004102d60 2024-02-26T04:55:28.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.328+0000 7f60097fc640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ff00a2140 0x7f5ff00a4600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6004102200 tx=0x7f5ff4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:28.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.332+0000 7f5ffa7fc640 1 -- 172.21.15.60:0/1057793230 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ffc0b97f0 con 0x7f6004102d60 2024-02-26T04:55:28.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:28 smithi060 ceph-mon[32382]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:28.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:28 smithi060 ceph-mon[32382]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T04:55:28.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:28 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/833253730' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-26T04:55:28.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:28 smithi060 ceph-mon[32382]: osdmap e83: 12 total, 12 up, 12 in 2024-02-26T04:55:28.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:28 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3415983884' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T04:55:28.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:28 smithi060 ceph-mon[32382]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-26T04:55:28.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:28 smithi191 ceph-mon[36352]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:28.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:28 smithi191 ceph-mon[36352]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T04:55:28.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:28 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/833253730' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-26T04:55:28.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:28 smithi191 ceph-mon[36352]: osdmap e83: 12 total, 12 up, 12 in 2024-02-26T04:55:28.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:28 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3415983884' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T04:55:28.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:28 smithi191 ceph-mon[36352]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-26T04:55:28.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:28.522+0000 7f600ba60640 1 -- 172.21.15.60:0/1057793230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f600410cd90 con 0x7f6004102d60 2024-02-26T04:55:28.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:28 smithi156 ceph-mon[35805]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:28.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:28 smithi156 ceph-mon[35805]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T04:55:28.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:28 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/833253730' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-26T04:55:28.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:28 smithi156 ceph-mon[35805]: osdmap e83: 12 total, 12 up, 12 in 2024-02-26T04:55:28.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:28 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3415983884' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-26T04:55:28.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:28 smithi156 ceph-mon[35805]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-26T04:55:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.199+0000 7f5ffa7fc640 1 -- 172.21.15.60:0/1057793230 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) v1 ==== 157+0+0 (secure 0 0 0) 0x7f5ffc0bceb0 con 0x7f6004102d60 2024-02-26T04:55:29.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.202+0000 7f600ba60640 1 -- 172.21.15.60:0/1057793230 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ff00a2140 msgr2=0x7f5ff00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:29.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.202+0000 7f600ba60640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ff00a2140 0x7f5ff00a4600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6004102200 tx=0x7f5ff4005f90 comp rx=0 tx=0).stop 2024-02-26T04:55:29.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.202+0000 7f600ba60640 1 -- 172.21.15.60:0/1057793230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6004102d60 msgr2=0x7f600419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:29.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.202+0000 7f600ba60640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6004102d60 0x7f600419e700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc037040 tx=0x7f5ffc034d40 comp rx=0 tx=0).stop 2024-02-26T04:55:29.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.202+0000 7f600ba60640 1 -- 172.21.15.60:0/1057793230 shutdown_connections 2024-02-26T04:55:29.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.202+0000 7f600ba60640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ff00a2140 0x7f5ff00a4600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:29.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.203+0000 7f600ba60640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6004102d60 0x7f600419e700 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:29.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.203+0000 7f600ba60640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60041023a0 0x7f6004197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:29.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.203+0000 7f600ba60640 1 --2- 172.21.15.60:0/1057793230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60041011b0 0x7f6004197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:29.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.203+0000 7f600ba60640 1 -- 172.21.15.60:0/1057793230 >> 172.21.15.60:0/1057793230 conn(0x7f60040fc740 msgr2=0x7f60041055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:29.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.203+0000 7f600ba60640 1 -- 172.21.15.60:0/1057793230 shutdown_connections 2024-02-26T04:55:29.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.203+0000 7f600ba60640 1 -- 172.21.15.60:0/1057793230 wait complete. 2024-02-26T04:55:29.218 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-02-26T04:55:29.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.333+0000 7f2002a36640 1 -- 172.21.15.60:0/3176048543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ffc1011a0 msgr2=0x7f1ffc1015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:29.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.333+0000 7f2002a36640 1 --2- 172.21.15.60:0/3176048543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ffc1011a0 0x7f1ffc1015c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009860 tx=0x7f1fec02bb70 comp rx=0 tx=0).stop 2024-02-26T04:55:29.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.333+0000 7f2002a36640 1 -- 172.21.15.60:0/3176048543 shutdown_connections 2024-02-26T04:55:29.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.333+0000 7f2002a36640 1 --2- 172.21.15.60:0/3176048543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ffc102d50 0x7f1ffc1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:29.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.333+0000 7f2002a36640 1 --2- 172.21.15.60:0/3176048543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ffc102390 0x7f1ffc102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:29.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.333+0000 7f2002a36640 1 --2- 172.21.15.60:0/3176048543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ffc1011a0 0x7f1ffc1015c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:29.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.333+0000 7f2002a36640 1 -- 172.21.15.60:0/3176048543 >> 172.21.15.60:0/3176048543 conn(0x7f1ffc0fc730 msgr2=0x7f1ffc0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:29.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.333+0000 7f2002a36640 1 -- 172.21.15.60:0/3176048543 shutdown_connections 2024-02-26T04:55:29.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.333+0000 7f2002a36640 1 -- 172.21.15.60:0/3176048543 wait complete. 2024-02-26T04:55:29.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.334+0000 7f2002a36640 1 Processor -- start 2024-02-26T04:55:29.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.334+0000 7f2002a36640 1 -- start start 2024-02-26T04:55:29.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.334+0000 7f2002a36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ffc1011a0 0x7f1ffc197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:29.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.334+0000 7f2002a36640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ffc102390 0x7f1ffc1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:29.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.334+0000 7f2002a36640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ffc102d50 0x7f1ffc19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:29.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.334+0000 7f2002a36640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc19ed50 con 0x7f1ffc1011a0 2024-02-26T04:55:29.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.334+0000 7f2002a36640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc19eec0 con 0x7f1ffc102390 2024-02-26T04:55:29.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.334+0000 7f2002a36640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc19f080 con 0x7f1ffc102d50 2024-02-26T04:55:29.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.334+0000 7f1ffbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ffc1011a0 0x7f1ffc197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:29.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.334+0000 7f1ffbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ffc1011a0 0x7f1ffc197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43512/0 (socket says 172.21.15.60:43512) 2024-02-26T04:55:29.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.334+0000 7f1ffbfff640 1 -- 172.21.15.60:0/3474971188 learned_addr learned my addr 172.21.15.60:0/3474971188 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:29.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.334+0000 7f2000fd3640 1 --2- 172.21.15.60:0/3474971188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ffc102d50 0x7f1ffc19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:29.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.334+0000 7f1ffb7fe640 1 --2- 172.21.15.60:0/3474971188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ffc102390 0x7f1ffc1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:29.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.335+0000 7f1ffbfff640 1 -- 172.21.15.60:0/3474971188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ffc102d50 msgr2=0x7f1ffc19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:29.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.335+0000 7f1ffbfff640 1 --2- 172.21.15.60:0/3474971188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ffc102d50 0x7f1ffc19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:29.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.335+0000 7f1ffbfff640 1 -- 172.21.15.60:0/3474971188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ffc102390 msgr2=0x7f1ffc1977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:29.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.335+0000 7f1ffbfff640 1 --2- 172.21.15.60:0/3474971188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ffc102390 0x7f1ffc1977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:29.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.335+0000 7f1ffbfff640 1 -- 172.21.15.60:0/3474971188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff4009900 con 0x7f1ffc1011a0 2024-02-26T04:55:29.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.335+0000 7f1ffbfff640 1 --2- 172.21.15.60:0/3474971188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ffc1011a0 0x7f1ffc197260 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1fec02c0b0 tx=0x7f1fec034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:29.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.335+0000 7f1ff97fa640 1 -- 172.21.15.60:0/3474971188 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec02c510 con 0x7f1ffc1011a0 2024-02-26T04:55:29.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.335+0000 7f2002a36640 1 -- 172.21.15.60:0/3474971188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fec009490 con 0x7f1ffc1011a0 2024-02-26T04:55:29.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.335+0000 7f1ff97fa640 1 -- 172.21.15.60:0/3474971188 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fec02cb50 con 0x7f1ffc1011a0 2024-02-26T04:55:29.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.335+0000 7f1ff97fa640 1 -- 172.21.15.60:0/3474971188 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec003ea0 con 0x7f1ffc1011a0 2024-02-26T04:55:29.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.335+0000 7f2002a36640 1 -- 172.21.15.60:0/3474971188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ffc19f530 con 0x7f1ffc1011a0 2024-02-26T04:55:29.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.336+0000 7f2002a36640 1 -- 172.21.15.60:0/3474971188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ffc108540 con 0x7f1ffc1011a0 2024-02-26T04:55:29.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.337+0000 7f1ff97fa640 1 -- 172.21.15.60:0/3474971188 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1fec03b040 con 0x7f1ffc1011a0 2024-02-26T04:55:29.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.338+0000 7f1ff97fa640 1 --2- 172.21.15.60:0/3474971188 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1fe40a20c0 0x7f1fe40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:29.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.338+0000 7f1ffb7fe640 1 --2- 172.21.15.60:0/3474971188 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1fe40a20c0 0x7f1fe40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:29.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.338+0000 7f1ff97fa640 1 -- 172.21.15.60:0/3474971188 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f1fec0efdb0 con 0x7f1ffc1011a0 2024-02-26T04:55:29.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.338+0000 7f1ffb7fe640 1 --2- 172.21.15.60:0/3474971188 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1fe40a20c0 0x7f1fe40a4580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc198770 tx=0x7f1ff000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:29.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.341+0000 7f1ff97fa640 1 -- 172.21.15.60:0/3474971188 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1fec0b8da0 con 0x7f1ffc1011a0 2024-02-26T04:55:29.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:29 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3415983884' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-26T04:55:29.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:29 smithi060 ceph-mon[32382]: osdmap e84: 12 total, 12 up, 12 in 2024-02-26T04:55:29.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:29 smithi060 ceph-mon[32382]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-26T04:55:29.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:29 smithi060 ceph-mon[32382]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-26T04:55:29.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:29 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1057793230' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-26T04:55:29.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:29 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] up:active 2024-02-26T04:55:29.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:29 smithi060 ceph-mon[32382]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T04:55:29.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:29 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3415983884' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-26T04:55:29.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:29 smithi191 ceph-mon[36352]: osdmap e84: 12 total, 12 up, 12 in 2024-02-26T04:55:29.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:29 smithi191 ceph-mon[36352]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-26T04:55:29.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:29 smithi191 ceph-mon[36352]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-26T04:55:29.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:29 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1057793230' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-26T04:55:29.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:29 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] up:active 2024-02-26T04:55:29.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:29 smithi191 ceph-mon[36352]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T04:55:29.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:29.522+0000 7f2002a36640 1 -- 172.21.15.60:0/3474971188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1ffc107fb0 con 0x7f1ffc1011a0 2024-02-26T04:55:29.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:29 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3415983884' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-26T04:55:29.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:29 smithi156 ceph-mon[35805]: osdmap e84: 12 total, 12 up, 12 in 2024-02-26T04:55:29.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:29 smithi156 ceph-mon[35805]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-26T04:55:29.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:29 smithi156 ceph-mon[35805]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-26T04:55:29.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:29 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1057793230' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-26T04:55:29.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:29 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] up:active 2024-02-26T04:55:29.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:29 smithi156 ceph-mon[35805]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T04:55:30.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:30 smithi060 ceph-mon[32382]: pgmap v289: 97 pgs: 18 creating+peering, 75 unknown, 4 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:30.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:30 smithi060 ceph-mon[32382]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T04:55:30.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:30 smithi060 ceph-mon[32382]: Cluster is now healthy 2024-02-26T04:55:30.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:30 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1057793230' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-26T04:55:30.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:30 smithi060 ceph-mon[32382]: osdmap e85: 12 total, 12 up, 12 in 2024-02-26T04:55:30.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:30 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3474971188' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T04:55:30.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:30 smithi191 ceph-mon[36352]: pgmap v289: 97 pgs: 18 creating+peering, 75 unknown, 4 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:30.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:30 smithi191 ceph-mon[36352]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T04:55:30.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:30 smithi191 ceph-mon[36352]: Cluster is now healthy 2024-02-26T04:55:30.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:30 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1057793230' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-26T04:55:30.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:30 smithi191 ceph-mon[36352]: osdmap e85: 12 total, 12 up, 12 in 2024-02-26T04:55:30.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:30 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3474971188' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T04:55:30.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:30 smithi156 ceph-mon[35805]: pgmap v289: 97 pgs: 18 creating+peering, 75 unknown, 4 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T04:55:30.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:30 smithi156 ceph-mon[35805]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T04:55:30.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:30 smithi156 ceph-mon[35805]: Cluster is now healthy 2024-02-26T04:55:30.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:30 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1057793230' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-26T04:55:30.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:30 smithi156 ceph-mon[35805]: osdmap e85: 12 total, 12 up, 12 in 2024-02-26T04:55:30.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:30 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3474971188' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-26T04:55:31.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.206+0000 7f1ff97fa640 1 -- 172.21.15.60:0/3474971188 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) v1 ==== 228+0+0 (secure 0 0 0) 0x7f1fec041100 con 0x7f1ffc1011a0 2024-02-26T04:55:31.209 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data_ec' created 2024-02-26T04:55:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.211+0000 7f2002a36640 1 -- 172.21.15.60:0/3474971188 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1fe40a20c0 msgr2=0x7f1fe40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.211+0000 7f2002a36640 1 --2- 172.21.15.60:0/3474971188 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1fe40a20c0 0x7f1fe40a4580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc198770 tx=0x7f1ff000b390 comp rx=0 tx=0).stop 2024-02-26T04:55:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.211+0000 7f2002a36640 1 -- 172.21.15.60:0/3474971188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ffc1011a0 msgr2=0x7f1ffc197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:31.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.211+0000 7f2002a36640 1 --2- 172.21.15.60:0/3474971188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ffc1011a0 0x7f1ffc197260 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1fec02c0b0 tx=0x7f1fec034440 comp rx=0 tx=0).stop 2024-02-26T04:55:31.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.211+0000 7f2002a36640 1 -- 172.21.15.60:0/3474971188 shutdown_connections 2024-02-26T04:55:31.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.211+0000 7f2002a36640 1 --2- 172.21.15.60:0/3474971188 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1fe40a20c0 0x7f1fe40a4580 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:31.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.211+0000 7f2002a36640 1 --2- 172.21.15.60:0/3474971188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ffc102d50 0x7f1ffc19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:31.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.211+0000 7f2002a36640 1 --2- 172.21.15.60:0/3474971188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ffc102390 0x7f1ffc1977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:31.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.211+0000 7f2002a36640 1 --2- 172.21.15.60:0/3474971188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ffc1011a0 0x7f1ffc197260 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:31.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.211+0000 7f2002a36640 1 -- 172.21.15.60:0/3474971188 >> 172.21.15.60:0/3474971188 conn(0x7f1ffc0fc730 msgr2=0x7f1ffc105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:31.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.211+0000 7f2002a36640 1 -- 172.21.15.60:0/3474971188 shutdown_connections 2024-02-26T04:55:31.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.211+0000 7f2002a36640 1 -- 172.21.15.60:0/3474971188 wait complete. 2024-02-26T04:55:31.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-02-26T04:55:31.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.341+0000 7f0411684640 1 -- 172.21.15.60:0/4194958119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f040c101270 msgr2=0x7f040c1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:31.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.341+0000 7f0411684640 1 --2- 172.21.15.60:0/4194958119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f040c101270 0x7f040c1016f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009900 tx=0x7f03f402bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.341+0000 7f0411684640 1 -- 172.21.15.60:0/4194958119 shutdown_connections 2024-02-26T04:55:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.341+0000 7f0411684640 1 --2- 172.21.15.60:0/4194958119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f040c101c30 0x7f040c1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.341+0000 7f0411684640 1 --2- 172.21.15.60:0/4194958119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f040c101270 0x7f040c1016f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.341+0000 7f0411684640 1 --2- 172.21.15.60:0/4194958119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f040c100080 0x7f040c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.341+0000 7f0411684640 1 -- 172.21.15.60:0/4194958119 >> 172.21.15.60:0/4194958119 conn(0x7f040c0fb610 msgr2=0x7f040c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.341+0000 7f0411684640 1 -- 172.21.15.60:0/4194958119 shutdown_connections 2024-02-26T04:55:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.341+0000 7f0411684640 1 -- 172.21.15.60:0/4194958119 wait complete. 2024-02-26T04:55:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.342+0000 7f0411684640 1 Processor -- start 2024-02-26T04:55:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.342+0000 7f0411684640 1 -- start start 2024-02-26T04:55:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.342+0000 7f0411684640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f040c100080 0x7f040c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.342+0000 7f0411684640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f040c101270 0x7f040c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.342+0000 7f0411684640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f040c101c30 0x7f040c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.342+0000 7f0411684640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c19ed30 con 0x7f040c101270 2024-02-26T04:55:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.342+0000 7f0411684640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c19eea0 con 0x7f040c101c30 2024-02-26T04:55:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.342+0000 7f0411684640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c19f060 con 0x7f040c100080 2024-02-26T04:55:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.342+0000 7f040a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f040c101270 0x7f040c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.342+0000 7f040a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f040c101270 0x7f040c197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43958/0 (socket says 172.21.15.60:43958) 2024-02-26T04:55:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.342+0000 7f040a7fc640 1 -- 172.21.15.60:0/122026384 learned_addr learned my addr 172.21.15.60:0/122026384 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.343+0000 7f040affd640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f040c100080 0x7f040c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.343+0000 7f040b7fe640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f040c101c30 0x7f040c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.343+0000 7f040a7fc640 1 -- 172.21.15.60:0/122026384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f040c100080 msgr2=0x7f040c197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.343+0000 7f040a7fc640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f040c100080 0x7f040c197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.343+0000 7f040a7fc640 1 -- 172.21.15.60:0/122026384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f040c101c30 msgr2=0x7f040c19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.343+0000 7f040a7fc640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f040c101c30 0x7f040c19e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.343+0000 7f040a7fc640 1 -- 172.21.15.60:0/122026384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f4009560 con 0x7f040c101270 2024-02-26T04:55:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.343+0000 7f040b7fe640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f040c101c30 0x7f040c19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.343+0000 7f040a7fc640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f040c101270 0x7f040c197780 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f03f4006010 tx=0x7f03f40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.343+0000 7f0403fff640 1 -- 172.21.15.60:0/122026384 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f40044c0 con 0x7f040c101270 2024-02-26T04:55:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.343+0000 7f0403fff640 1 -- 172.21.15.60:0/122026384 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03f40052d0 con 0x7f040c101270 2024-02-26T04:55:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.343+0000 7f0411684640 1 -- 172.21.15.60:0/122026384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040c19f2e0 con 0x7f040c101270 2024-02-26T04:55:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.343+0000 7f0403fff640 1 -- 172.21.15.60:0/122026384 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4003e10 con 0x7f040c101270 2024-02-26T04:55:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.343+0000 7f0411684640 1 -- 172.21.15.60:0/122026384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040c19f7f0 con 0x7f040c101270 2024-02-26T04:55:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.344+0000 7f0411684640 1 -- 172.21.15.60:0/122026384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040c10ba90 con 0x7f040c101270 2024-02-26T04:55:31.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.345+0000 7f0403fff640 1 -- 172.21.15.60:0/122026384 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f03f4005440 con 0x7f040c101270 2024-02-26T04:55:31.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.346+0000 7f0403fff640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f03f80a20c0 0x7f03f80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:31.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.346+0000 7f040affd640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f03f80a20c0 0x7f03f80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:31.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.346+0000 7f0403fff640 1 -- 172.21.15.60:0/122026384 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f03f40f08a0 con 0x7f040c101270 2024-02-26T04:55:31.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.346+0000 7f040affd640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f03f80a20c0 0x7f03f80a4580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f040c1010d0 tx=0x7f03fc009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:31.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.350+0000 7f0403fff640 1 -- 172.21.15.60:0/122026384 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f03f40b95f0 con 0x7f040c101270 2024-02-26T04:55:31.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:31.530+0000 7f0411684640 1 -- 172.21.15.60:0/122026384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f040c10bc20 con 0x7f040c101270 2024-02-26T04:55:31.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:31 smithi060 ceph-mon[32382]: pgmap v292: 97 pgs: 18 creating+peering, 46 unknown, 33 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 8 op/s 2024-02-26T04:55:31.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:31 smithi060 ceph-mon[32382]: osdmap e86: 12 total, 12 up, 12 in 2024-02-26T04:55:31.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:31 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3474971188' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-26T04:55:31.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:31 smithi060 ceph-mon[32382]: osdmap e87: 12 total, 12 up, 12 in 2024-02-26T04:55:31.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:31 smithi191 ceph-mon[36352]: pgmap v292: 97 pgs: 18 creating+peering, 46 unknown, 33 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 8 op/s 2024-02-26T04:55:31.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:31 smithi191 ceph-mon[36352]: osdmap e86: 12 total, 12 up, 12 in 2024-02-26T04:55:31.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:31 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3474971188' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-26T04:55:31.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:31 smithi191 ceph-mon[36352]: osdmap e87: 12 total, 12 up, 12 in 2024-02-26T04:55:31.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:31 smithi156 ceph-mon[35805]: pgmap v292: 97 pgs: 18 creating+peering, 46 unknown, 33 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 8 op/s 2024-02-26T04:55:31.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:31 smithi156 ceph-mon[35805]: osdmap e86: 12 total, 12 up, 12 in 2024-02-26T04:55:31.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:31 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3474971188' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-26T04:55:31.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:31 smithi156 ceph-mon[35805]: osdmap e87: 12 total, 12 up, 12 in 2024-02-26T04:55:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.209+0000 7f0403fff640 1 -- 172.21.15.60:0/122026384 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7f03f40bccb0 con 0x7f040c101270 2024-02-26T04:55:32.211 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 4 allow_ec_overwrites to true 2024-02-26T04:55:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.212+0000 7f0411684640 1 -- 172.21.15.60:0/122026384 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f03f80a20c0 msgr2=0x7f03f80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.212+0000 7f0411684640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f03f80a20c0 0x7f03f80a4580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f040c1010d0 tx=0x7f03fc009670 comp rx=0 tx=0).stop 2024-02-26T04:55:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.212+0000 7f0411684640 1 -- 172.21.15.60:0/122026384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f040c101270 msgr2=0x7f040c197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.212+0000 7f0411684640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f040c101270 0x7f040c197780 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f03f4006010 tx=0x7f03f40040c0 comp rx=0 tx=0).stop 2024-02-26T04:55:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.212+0000 7f0411684640 1 -- 172.21.15.60:0/122026384 shutdown_connections 2024-02-26T04:55:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.212+0000 7f0411684640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f03f80a20c0 0x7f03f80a4580 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.212+0000 7f0411684640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f040c101c30 0x7f040c19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.212+0000 7f0411684640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f040c101270 0x7f040c197780 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.212+0000 7f0411684640 1 --2- 172.21.15.60:0/122026384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f040c100080 0x7f040c197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.212+0000 7f0411684640 1 -- 172.21.15.60:0/122026384 >> 172.21.15.60:0/122026384 conn(0x7f040c0fb610 msgr2=0x7f040c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.213+0000 7f0411684640 1 -- 172.21.15.60:0/122026384 shutdown_connections 2024-02-26T04:55:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.213+0000 7f0411684640 1 -- 172.21.15.60:0/122026384 wait complete. 2024-02-26T04:55:32.241 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-02-26T04:55:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.348+0000 7f8545f1e640 1 -- 172.21.15.60:0/4098006824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8540104550 msgr2=0x7f8540106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.348+0000 7f8545f1e640 1 --2- 172.21.15.60:0/4098006824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8540104550 0x7f8540106940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8534009900 tx=0x7f853402bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.348+0000 7f8545f1e640 1 -- 172.21.15.60:0/4098006824 shutdown_connections 2024-02-26T04:55:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.348+0000 7f8545f1e640 1 --2- 172.21.15.60:0/4098006824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8540106e80 0x7f85401092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.348+0000 7f8545f1e640 1 --2- 172.21.15.60:0/4098006824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8540104550 0x7f8540106940 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.348+0000 7f8545f1e640 1 --2- 172.21.15.60:0/4098006824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8540101c20 0x7f8540104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.348+0000 7f8545f1e640 1 -- 172.21.15.60:0/4098006824 >> 172.21.15.60:0/4098006824 conn(0x7f85400fb650 msgr2=0x7f85400fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.348+0000 7f8545f1e640 1 -- 172.21.15.60:0/4098006824 shutdown_connections 2024-02-26T04:55:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.348+0000 7f8545f1e640 1 -- 172.21.15.60:0/4098006824 wait complete. 2024-02-26T04:55:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f8545f1e640 1 Processor -- start 2024-02-26T04:55:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f8545f1e640 1 -- start start 2024-02-26T04:55:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f8545f1e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8540101c20 0x7f85401972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f8545f1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8540104550 0x7f8540197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f8545f1e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8540106e80 0x7f854019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f8545f1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854019edc0 con 0x7f8540104550 2024-02-26T04:55:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f8545f1e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854019ef30 con 0x7f8540106e80 2024-02-26T04:55:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f8545f1e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854019f0f0 con 0x7f8540101c20 2024-02-26T04:55:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f853effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8540104550 0x7f8540197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f853effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8540104550 0x7f8540197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43982/0 (socket says 172.21.15.60:43982) 2024-02-26T04:55:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f853effd640 1 -- 172.21.15.60:0/2698441341 learned_addr learned my addr 172.21.15.60:0/2698441341 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f853f7fe640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8540101c20 0x7f85401972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f853ffff640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8540106e80 0x7f854019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f853effd640 1 -- 172.21.15.60:0/2698441341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8540101c20 msgr2=0x7f85401972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f853effd640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8540101c20 0x7f85401972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f853effd640 1 -- 172.21.15.60:0/2698441341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8540106e80 msgr2=0x7f854019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f853effd640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8540106e80 0x7f854019e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.349+0000 7f853effd640 1 -- 172.21.15.60:0/2698441341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8534009560 con 0x7f8540104550 2024-02-26T04:55:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.350+0000 7f853f7fe640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8540101c20 0x7f85401972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.350+0000 7f853effd640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8540104550 0x7f8540197810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8534036040 tx=0x7f85340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.350+0000 7f853cff9640 1 -- 172.21.15.60:0/2698441341 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85340044e0 con 0x7f8540104550 2024-02-26T04:55:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.350+0000 7f853cff9640 1 -- 172.21.15.60:0/2698441341 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85340054e0 con 0x7f8540104550 2024-02-26T04:55:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.350+0000 7f8545f1e640 1 -- 172.21.15.60:0/2698441341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854019f370 con 0x7f8540104550 2024-02-26T04:55:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.350+0000 7f853cff9640 1 -- 172.21.15.60:0/2698441341 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853403e450 con 0x7f8540104550 2024-02-26T04:55:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.350+0000 7f8545f1e640 1 -- 172.21.15.60:0/2698441341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f854019f900 con 0x7f8540104550 2024-02-26T04:55:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.351+0000 7f8545f1e640 1 -- 172.21.15.60:0/2698441341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85400fd140 con 0x7f8540104550 2024-02-26T04:55:32.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.352+0000 7f853cff9640 1 -- 172.21.15.60:0/2698441341 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8534003710 con 0x7f8540104550 2024-02-26T04:55:32.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.352+0000 7f853cff9640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f85280a20c0 0x7f85280a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:32.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.353+0000 7f853cff9640 1 -- 172.21.15.60:0/2698441341 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f85340f03d0 con 0x7f8540104550 2024-02-26T04:55:32.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.353+0000 7f853f7fe640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f85280a20c0 0x7f85280a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:32.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.353+0000 7f853f7fe640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f85280a20c0 0x7f85280a4580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8530009930 tx=0x7f8530008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:32.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.357+0000 7f853cff9640 1 -- 172.21.15.60:0/2698441341 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f85340b9180 con 0x7f8540104550 2024-02-26T04:55:32.427 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 04:55:32 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-b[65871]: 2024-02-26T04:55:32.157+0000 7ff16f12d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T04:55:32.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:32.580+0000 7f8545f1e640 1 -- 172.21.15.60:0/2698441341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f854002cdd0 con 0x7f8540104550 2024-02-26T04:55:32.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:32 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/122026384' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-26T04:55:32.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:32 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/122026384' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-26T04:55:32.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:32 smithi060 ceph-mon[32382]: osdmap e88: 12 total, 12 up, 12 in 2024-02-26T04:55:32.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:32 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/122026384' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-26T04:55:32.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:32 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/122026384' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-26T04:55:32.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:32 smithi191 ceph-mon[36352]: osdmap e88: 12 total, 12 up, 12 in 2024-02-26T04:55:32.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:32 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/122026384' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-26T04:55:32.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:32 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/122026384' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-26T04:55:32.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:32 smithi156 ceph-mon[35805]: osdmap e88: 12 total, 12 up, 12 in 2024-02-26T04:55:33.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.462+0000 7f853cff9640 1 -- 172.21.15.60:0/2698441341 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) v1 ==== 137+0+0 (secure 0 0 0) 0x7f85340bc840 con 0x7f8540104550 2024-02-26T04:55:33.465 INFO:teuthology.orchestra.run.smithi060.stderr:added data pool 4 to fsmap 2024-02-26T04:55:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.465+0000 7f8545f1e640 1 -- 172.21.15.60:0/2698441341 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f85280a20c0 msgr2=0x7f85280a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.465+0000 7f8545f1e640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f85280a20c0 0x7f85280a4580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8530009930 tx=0x7f8530008040 comp rx=0 tx=0).stop 2024-02-26T04:55:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.465+0000 7f8545f1e640 1 -- 172.21.15.60:0/2698441341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8540104550 msgr2=0x7f8540197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.465+0000 7f8545f1e640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8540104550 0x7f8540197810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8534036040 tx=0x7f85340042f0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.465+0000 7f8545f1e640 1 -- 172.21.15.60:0/2698441341 shutdown_connections 2024-02-26T04:55:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.465+0000 7f8545f1e640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f85280a20c0 0x7f85280a4580 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.465+0000 7f8545f1e640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8540106e80 0x7f854019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.465+0000 7f8545f1e640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8540104550 0x7f8540197810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.465+0000 7f8545f1e640 1 --2- 172.21.15.60:0/2698441341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8540101c20 0x7f85401972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.465+0000 7f8545f1e640 1 -- 172.21.15.60:0/2698441341 >> 172.21.15.60:0/2698441341 conn(0x7f85400fb650 msgr2=0x7f8540102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.465+0000 7f8545f1e640 1 -- 172.21.15.60:0/2698441341 shutdown_connections 2024-02-26T04:55:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.465+0000 7f8545f1e640 1 -- 172.21.15.60:0/2698441341 wait complete. 2024-02-26T04:55:33.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:55:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.589+0000 7fe799cde640 1 -- 172.21.15.60:0/664771096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7941011a0 msgr2=0x7fe7941015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.589+0000 7fe799cde640 1 --2- 172.21.15.60:0/664771096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7941011a0 0x7fe7941015c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe784009900 tx=0x7fe78402bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.589+0000 7fe799cde640 1 -- 172.21.15.60:0/664771096 shutdown_connections 2024-02-26T04:55:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.589+0000 7fe799cde640 1 --2- 172.21.15.60:0/664771096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe794102d50 0x7fe7941095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.589+0000 7fe799cde640 1 --2- 172.21.15.60:0/664771096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe794102390 0x7fe794102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.589+0000 7fe799cde640 1 --2- 172.21.15.60:0/664771096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7941011a0 0x7fe7941015c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.589+0000 7fe799cde640 1 -- 172.21.15.60:0/664771096 >> 172.21.15.60:0/664771096 conn(0x7fe7940fc730 msgr2=0x7fe7940feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.589+0000 7fe799cde640 1 -- 172.21.15.60:0/664771096 shutdown_connections 2024-02-26T04:55:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.589+0000 7fe799cde640 1 -- 172.21.15.60:0/664771096 wait complete. 2024-02-26T04:55:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.589+0000 7fe799cde640 1 Processor -- start 2024-02-26T04:55:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.589+0000 7fe799cde640 1 -- start start 2024-02-26T04:55:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe799cde640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7941011a0 0x7fe7941971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe799cde640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe794102390 0x7fe7941976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe799cde640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe794102d50 0x7fe79419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe799cde640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79419ec90 con 0x7fe7941011a0 2024-02-26T04:55:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe799cde640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79419ee00 con 0x7fe794102390 2024-02-26T04:55:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe799cde640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79419efc0 con 0x7fe794102d50 2024-02-26T04:55:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe7937fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7941011a0 0x7fe7941971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe7937fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7941011a0 0x7fe7941971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44010/0 (socket says 172.21.15.60:44010) 2024-02-26T04:55:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe7937fe640 1 -- 172.21.15.60:0/2912323221 learned_addr learned my addr 172.21.15.60:0/2912323221 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe78affd640 1 --2- 172.21.15.60:0/2912323221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe794102390 0x7fe7941976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe793fff640 1 --2- 172.21.15.60:0/2912323221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe794102d50 0x7fe79419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe7937fe640 1 -- 172.21.15.60:0/2912323221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe794102d50 msgr2=0x7fe79419e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe7937fe640 1 --2- 172.21.15.60:0/2912323221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe794102d50 0x7fe79419e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe7937fe640 1 -- 172.21.15.60:0/2912323221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe794102390 msgr2=0x7fe7941976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe7937fe640 1 --2- 172.21.15.60:0/2912323221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe794102390 0x7fe7941976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe7937fe640 1 -- 172.21.15.60:0/2912323221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe784009560 con 0x7fe7941011a0 2024-02-26T04:55:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe7937fe640 1 --2- 172.21.15.60:0/2912323221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7941011a0 0x7fe7941971a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe784005ab0 tx=0x7fe784004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe7917fa640 1 -- 172.21.15.60:0/2912323221 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784004aa0 con 0x7fe7941011a0 2024-02-26T04:55:33.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe799cde640 1 -- 172.21.15.60:0/2912323221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79419f240 con 0x7fe7941011a0 2024-02-26T04:55:33.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe7917fa640 1 -- 172.21.15.60:0/2912323221 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7840052d0 con 0x7fe7941011a0 2024-02-26T04:55:33.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe7917fa640 1 -- 172.21.15.60:0/2912323221 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784003e80 con 0x7fe7941011a0 2024-02-26T04:55:33.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.590+0000 7fe799cde640 1 -- 172.21.15.60:0/2912323221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79419f7a0 con 0x7fe7941011a0 2024-02-26T04:55:33.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.592+0000 7fe799cde640 1 -- 172.21.15.60:0/2912323221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7941084c0 con 0x7fe7941011a0 2024-02-26T04:55:33.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.593+0000 7fe7917fa640 1 -- 172.21.15.60:0/2912323221 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe784005440 con 0x7fe7941011a0 2024-02-26T04:55:33.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.594+0000 7fe7917fa640 1 --2- 172.21.15.60:0/2912323221 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe7800a20c0 0x7fe7800a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:33.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.594+0000 7fe78affd640 1 --2- 172.21.15.60:0/2912323221 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe7800a20c0 0x7fe7800a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:33.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.594+0000 7fe7917fa640 1 -- 172.21.15.60:0/2912323221 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe7840f0a60 con 0x7fe7941011a0 2024-02-26T04:55:33.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.594+0000 7fe78affd640 1 --2- 172.21.15.60:0/2912323221 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe7800a20c0 0x7fe7800a4580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe77c005e10 tx=0x7fe77c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:33.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.597+0000 7fe7917fa640 1 -- 172.21.15.60:0/2912323221 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe7840b9790 con 0x7fe7941011a0 2024-02-26T04:55:33.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:33 smithi060 ceph-mon[32382]: pgmap v295: 129 pgs: 18 creating+peering, 27 unknown, 84 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-02-26T04:55:33.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:33 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2698441341' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-26T04:55:33.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:33 smithi060 ceph-mon[32382]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T04:55:33.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:33 smithi060 ceph-mon[32382]: osdmap e89: 12 total, 12 up, 12 in 2024-02-26T04:55:33.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:33 smithi191 ceph-mon[36352]: pgmap v295: 129 pgs: 18 creating+peering, 27 unknown, 84 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-02-26T04:55:33.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:33 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2698441341' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-26T04:55:33.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:33 smithi191 ceph-mon[36352]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T04:55:33.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:33 smithi191 ceph-mon[36352]: osdmap e89: 12 total, 12 up, 12 in 2024-02-26T04:55:33.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.816+0000 7fe799cde640 1 -- 172.21.15.60:0/2912323221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe79410cbb0 con 0x7fe7941011a0 2024-02-26T04:55:33.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.816+0000 7fe7917fa640 1 -- 172.21.15.60:0/2912323221 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+11286 (secure 0 0 0) 0x7fe7840bce50 con 0x7fe7941011a0 2024-02-26T04:55:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:33.819 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:55:32.581648+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-26T04:55:33.819 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2024-02-26T04:55:33.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.818+0000 7fe799cde640 1 -- 172.21.15.60:0/2912323221 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe7800a20c0 msgr2=0x7fe7800a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:33.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.818+0000 7fe799cde640 1 --2- 172.21.15.60:0/2912323221 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe7800a20c0 0x7fe7800a4580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe77c005e10 tx=0x7fe77c005da0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.818+0000 7fe799cde640 1 -- 172.21.15.60:0/2912323221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7941011a0 msgr2=0x7fe7941971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:33.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.818+0000 7fe799cde640 1 --2- 172.21.15.60:0/2912323221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7941011a0 0x7fe7941971a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe784005ab0 tx=0x7fe784004270 comp rx=0 tx=0).stop 2024-02-26T04:55:33.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.819+0000 7fe799cde640 1 -- 172.21.15.60:0/2912323221 shutdown_connections 2024-02-26T04:55:33.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.819+0000 7fe799cde640 1 --2- 172.21.15.60:0/2912323221 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe7800a20c0 0x7fe7800a4580 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.819+0000 7fe799cde640 1 --2- 172.21.15.60:0/2912323221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe794102d50 0x7fe79419e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.819+0000 7fe799cde640 1 --2- 172.21.15.60:0/2912323221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe794102390 0x7fe7941976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.819+0000 7fe799cde640 1 --2- 172.21.15.60:0/2912323221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7941011a0 0x7fe7941971a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.819+0000 7fe799cde640 1 -- 172.21.15.60:0/2912323221 >> 172.21.15.60:0/2912323221 conn(0x7fe7940fc730 msgr2=0x7fe794105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:33.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.819+0000 7fe799cde640 1 -- 172.21.15.60:0/2912323221 shutdown_connections 2024-02-26T04:55:33.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.819+0000 7fe799cde640 1 -- 172.21.15.60:0/2912323221 wait complete. 2024-02-26T04:55:33.832 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:55:33.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:33 smithi156 ceph-mon[35805]: pgmap v295: 129 pgs: 18 creating+peering, 27 unknown, 84 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-02-26T04:55:33.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:33 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2698441341' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-26T04:55:33.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:33 smithi156 ceph-mon[35805]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-26T04:55:33.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:33 smithi156 ceph-mon[35805]: osdmap e89: 12 total, 12 up, 12 in 2024-02-26T04:55:33.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.932+0000 7f5e98d55640 1 -- 172.21.15.60:0/237994336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e941000a0 msgr2=0x7f5e941004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:33.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.932+0000 7f5e98d55640 1 --2- 172.21.15.60:0/237994336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e941000a0 0x7f5e941004c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009900 tx=0x7f5e8402bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:33.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.933+0000 7f5e98d55640 1 -- 172.21.15.60:0/237994336 shutdown_connections 2024-02-26T04:55:33.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.933+0000 7f5e98d55640 1 --2- 172.21.15.60:0/237994336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e94101c50 0x7f5e941084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.933+0000 7f5e98d55640 1 --2- 172.21.15.60:0/237994336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e94101290 0x7f5e94101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.933+0000 7f5e98d55640 1 --2- 172.21.15.60:0/237994336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e941000a0 0x7f5e941004c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.933+0000 7f5e98d55640 1 -- 172.21.15.60:0/237994336 >> 172.21.15.60:0/237994336 conn(0x7f5e940fb610 msgr2=0x7f5e940fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:33.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.933+0000 7f5e98d55640 1 -- 172.21.15.60:0/237994336 shutdown_connections 2024-02-26T04:55:33.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.933+0000 7f5e98d55640 1 -- 172.21.15.60:0/237994336 wait complete. 2024-02-26T04:55:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.933+0000 7f5e98d55640 1 Processor -- start 2024-02-26T04:55:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.933+0000 7f5e98d55640 1 -- start start 2024-02-26T04:55:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.933+0000 7f5e98d55640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e941000a0 0x7f5e941972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.933+0000 7f5e98d55640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e94101290 0x7f5e94197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e98d55640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e94101c50 0x7f5e9419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e98d55640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9419edd0 con 0x7f5e94101290 2024-02-26T04:55:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e98d55640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9419ef40 con 0x7f5e94101c50 2024-02-26T04:55:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e98d55640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9419f100 con 0x7f5e941000a0 2024-02-26T04:55:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e8bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e94101290 0x7f5e94197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e8bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e94101290 0x7f5e94197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44028/0 (socket says 172.21.15.60:44028) 2024-02-26T04:55:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e8bfff640 1 -- 172.21.15.60:0/238378866 learned_addr learned my addr 172.21.15.60:0/238378866 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e92d9d640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e94101c50 0x7f5e9419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e9259c640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e941000a0 0x7f5e941972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e8bfff640 1 -- 172.21.15.60:0/238378866 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e941000a0 msgr2=0x7f5e941972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e8bfff640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e941000a0 0x7f5e941972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e8bfff640 1 -- 172.21.15.60:0/238378866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e94101c50 msgr2=0x7f5e9419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e8bfff640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e94101c50 0x7f5e9419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e8bfff640 1 -- 172.21.15.60:0/238378866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e84009560 con 0x7f5e94101290 2024-02-26T04:55:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e9259c640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e941000a0 0x7f5e941972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e8bfff640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e94101290 0x7f5e94197820 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00b4a0 tx=0x7f5e7c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e8b7fe640 1 -- 172.21.15.60:0/238378866 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c010610 con 0x7f5e94101290 2024-02-26T04:55:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e8b7fe640 1 -- 172.21.15.60:0/238378866 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e7c010c50 con 0x7f5e94101290 2024-02-26T04:55:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e8b7fe640 1 -- 172.21.15.60:0/238378866 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c017e20 con 0x7f5e94101290 2024-02-26T04:55:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e98d55640 1 -- 172.21.15.60:0/238378866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9419f3e0 con 0x7f5e94101290 2024-02-26T04:55:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.934+0000 7f5e98d55640 1 -- 172.21.15.60:0/238378866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9419f920 con 0x7f5e94101290 2024-02-26T04:55:33.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.935+0000 7f5e98d55640 1 -- 172.21.15.60:0/238378866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e9410baf0 con 0x7f5e94101290 2024-02-26T04:55:33.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.936+0000 7f5e8b7fe640 1 -- 172.21.15.60:0/238378866 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5e7c017480 con 0x7f5e94101290 2024-02-26T04:55:33.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.937+0000 7f5e8b7fe640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5e800a20c0 0x7f5e800a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:33.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.937+0000 7f5e9259c640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5e800a20c0 0x7f5e800a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:33.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.937+0000 7f5e8b7fe640 1 -- 172.21.15.60:0/238378866 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f5e7c0cd540 con 0x7f5e94101290 2024-02-26T04:55:33.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.937+0000 7f5e9259c640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5e800a20c0 0x7f5e800a4580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5e84006010 tx=0x7f5e84005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:33.941+0000 7f5e8b7fe640 1 -- 172.21.15.60:0/238378866 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5e7c0962d0 con 0x7f5e94101290 2024-02-26T04:55:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.123+0000 7f5e98d55640 1 -- 172.21.15.60:0/238378866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e9410bc80 con 0x7f5e94101290 2024-02-26T04:55:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.125+0000 7f5e8b7fe640 1 -- 172.21.15.60:0/238378866 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24126 (secure 0 0 0) 0x7f5e7c099990 con 0x7f5e94101290 2024-02-26T04:55:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:34.131 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:55:32.581481+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:55:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.127+0000 7f5e98d55640 1 -- 172.21.15.60:0/238378866 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5e800a20c0 msgr2=0x7f5e800a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.127+0000 7f5e98d55640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5e800a20c0 0x7f5e800a4580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5e84006010 tx=0x7f5e84005be0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.127+0000 7f5e98d55640 1 -- 172.21.15.60:0/238378866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e94101290 msgr2=0x7f5e94197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.127+0000 7f5e98d55640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e94101290 0x7f5e94197820 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00b4a0 tx=0x7f5e7c00b960 comp rx=0 tx=0).stop 2024-02-26T04:55:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.127+0000 7f5e98d55640 1 -- 172.21.15.60:0/238378866 shutdown_connections 2024-02-26T04:55:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.127+0000 7f5e98d55640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5e800a20c0 0x7f5e800a4580 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.127+0000 7f5e98d55640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e94101c50 0x7f5e9419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.127+0000 7f5e98d55640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e94101290 0x7f5e94197820 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.127+0000 7f5e98d55640 1 --2- 172.21.15.60:0/238378866 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e941000a0 0x7f5e941972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.127+0000 7f5e98d55640 1 -- 172.21.15.60:0/238378866 >> 172.21.15.60:0/238378866 conn(0x7f5e940fb610 msgr2=0x7f5e940fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.127+0000 7f5e98d55640 1 -- 172.21.15.60:0/238378866 shutdown_connections 2024-02-26T04:55:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.127+0000 7f5e98d55640 1 -- 172.21.15.60:0/238378866 wait complete. 2024-02-26T04:55:34.142 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:55:34.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.242+0000 7f9f4a799640 1 -- 172.21.15.60:0/4063249610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f441045d0 msgr2=0x7f9f441069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.242+0000 7f9f4a799640 1 --2- 172.21.15.60:0/4063249610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f441045d0 0x7f9f441069c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009900 tx=0x7f9f3802bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:34.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.242+0000 7f9f4a799640 1 -- 172.21.15.60:0/4063249610 shutdown_connections 2024-02-26T04:55:34.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.242+0000 7f9f4a799640 1 --2- 172.21.15.60:0/4063249610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f44106f00 0x7f9f44109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.242+0000 7f9f4a799640 1 --2- 172.21.15.60:0/4063249610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f441045d0 0x7f9f441069c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.242+0000 7f9f4a799640 1 --2- 172.21.15.60:0/4063249610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44101ca0 0x7f9f44104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.242+0000 7f9f4a799640 1 -- 172.21.15.60:0/4063249610 >> 172.21.15.60:0/4063249610 conn(0x7f9f440fb6b0 msgr2=0x7f9f440fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.242+0000 7f9f4a799640 1 -- 172.21.15.60:0/4063249610 shutdown_connections 2024-02-26T04:55:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.242+0000 7f9f4a799640 1 -- 172.21.15.60:0/4063249610 wait complete. 2024-02-26T04:55:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.242+0000 7f9f4a799640 1 Processor -- start 2024-02-26T04:55:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.242+0000 7f9f4a799640 1 -- start start 2024-02-26T04:55:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f4a799640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f44101ca0 0x7f9f44199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f4a799640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f441045d0 0x7f9f441999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f4a799640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44106f00 0x7f9f441a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f4a799640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f441a0f80 con 0x7f9f441045d0 2024-02-26T04:55:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f4a799640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f441a10f0 con 0x7f9f44101ca0 2024-02-26T04:55:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f4a799640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f441a12b0 con 0x7f9f44106f00 2024-02-26T04:55:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f437fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f441045d0 0x7f9f441999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f437fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f441045d0 0x7f9f441999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44048/0 (socket says 172.21.15.60:44048) 2024-02-26T04:55:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f437fe640 1 -- 172.21.15.60:0/2015325954 learned_addr learned my addr 172.21.15.60:0/2015325954 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f48d36640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44106f00 0x7f9f441a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f43fff640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f44101ca0 0x7f9f44199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f437fe640 1 -- 172.21.15.60:0/2015325954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44106f00 msgr2=0x7f9f441a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f437fe640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44106f00 0x7f9f441a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f437fe640 1 -- 172.21.15.60:0/2015325954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f44101ca0 msgr2=0x7f9f44199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f437fe640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f44101ca0 0x7f9f44199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f437fe640 1 -- 172.21.15.60:0/2015325954 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f38009560 con 0x7f9f441045d0 2024-02-26T04:55:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f43fff640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f44101ca0 0x7f9f44199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f437fe640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f441045d0 0x7f9f441999d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9f38036040 tx=0x7f9f380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:34.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.243+0000 7f9f417fa640 1 -- 172.21.15.60:0/2015325954 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f380044e0 con 0x7f9f441045d0 2024-02-26T04:55:34.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.244+0000 7f9f4a799640 1 -- 172.21.15.60:0/2015325954 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f441a1530 con 0x7f9f441045d0 2024-02-26T04:55:34.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.244+0000 7f9f417fa640 1 -- 172.21.15.60:0/2015325954 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f380054e0 con 0x7f9f441045d0 2024-02-26T04:55:34.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.244+0000 7f9f417fa640 1 -- 172.21.15.60:0/2015325954 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3803e450 con 0x7f9f441045d0 2024-02-26T04:55:34.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.244+0000 7f9f4a799640 1 -- 172.21.15.60:0/2015325954 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f441a1a90 con 0x7f9f441045d0 2024-02-26T04:55:34.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.245+0000 7f9f4a799640 1 -- 172.21.15.60:0/2015325954 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f44108a50 con 0x7f9f441045d0 2024-02-26T04:55:34.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.245+0000 7f9f417fa640 1 -- 172.21.15.60:0/2015325954 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9f38003710 con 0x7f9f441045d0 2024-02-26T04:55:34.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.246+0000 7f9f417fa640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9f2c0a20c0 0x7f9f2c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:34.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.246+0000 7f9f417fa640 1 -- 172.21.15.60:0/2015325954 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9f380f03a0 con 0x7f9f441045d0 2024-02-26T04:55:34.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.246+0000 7f9f43fff640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9f2c0a20c0 0x7f9f2c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:34.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.246+0000 7f9f43fff640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9f2c0a20c0 0x7f9f2c0a4580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9f340058c0 tx=0x7f9f34005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:34.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.250+0000 7f9f417fa640 1 -- 172.21.15.60:0/2015325954 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9f380b90d0 con 0x7f9f441045d0 2024-02-26T04:55:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.434+0000 7f9f4a799640 1 -- 172.21.15.60:0/2015325954 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f441084c0 con 0x7f9f441045d0 2024-02-26T04:55:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.435+0000 7f9f417fa640 1 -- 172.21.15.60:0/2015325954 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24126 (secure 0 0 0) 0x7f9f380bc790 con 0x7f9f441045d0 2024-02-26T04:55:34.438 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:34.438 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:55:32.581481+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:55:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.437+0000 7f9f4a799640 1 -- 172.21.15.60:0/2015325954 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9f2c0a20c0 msgr2=0x7f9f2c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.437+0000 7f9f4a799640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9f2c0a20c0 0x7f9f2c0a4580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9f340058c0 tx=0x7f9f34005d30 comp rx=0 tx=0).stop 2024-02-26T04:55:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.437+0000 7f9f4a799640 1 -- 172.21.15.60:0/2015325954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f441045d0 msgr2=0x7f9f441999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.437+0000 7f9f4a799640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f441045d0 0x7f9f441999d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9f38036040 tx=0x7f9f380042f0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.437+0000 7f9f4a799640 1 -- 172.21.15.60:0/2015325954 shutdown_connections 2024-02-26T04:55:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.437+0000 7f9f4a799640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9f2c0a20c0 0x7f9f2c0a4580 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.437+0000 7f9f4a799640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44106f00 0x7f9f441a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.437+0000 7f9f4a799640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f441045d0 0x7f9f441999d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.437+0000 7f9f4a799640 1 --2- 172.21.15.60:0/2015325954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f44101ca0 0x7f9f44199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.438+0000 7f9f4a799640 1 -- 172.21.15.60:0/2015325954 >> 172.21.15.60:0/2015325954 conn(0x7f9f440fb6b0 msgr2=0x7f9f441013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:34.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.438+0000 7f9f4a799640 1 -- 172.21.15.60:0/2015325954 shutdown_connections 2024-02-26T04:55:34.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.438+0000 7f9f4a799640 1 -- 172.21.15.60:0/2015325954 wait complete. 2024-02-26T04:55:34.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:55:34.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 -- 172.21.15.60:0/3901987322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7138102d60 msgr2=0x7f71381095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 --2- 172.21.15.60:0/3901987322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7138102d60 0x7f71381095f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7130009ca0 tx=0x7f713002c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 -- 172.21.15.60:0/3901987322 shutdown_connections 2024-02-26T04:55:34.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 --2- 172.21.15.60:0/3901987322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7138102d60 0x7f71381095f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 --2- 172.21.15.60:0/3901987322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71381023a0 0x7f7138102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 --2- 172.21.15.60:0/3901987322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71381011b0 0x7f71381015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 -- 172.21.15.60:0/3901987322 >> 172.21.15.60:0/3901987322 conn(0x7f71380fc760 msgr2=0x7f71380feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:34.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 -- 172.21.15.60:0/3901987322 shutdown_connections 2024-02-26T04:55:34.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 -- 172.21.15.60:0/3901987322 wait complete. 2024-02-26T04:55:34.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 Processor -- start 2024-02-26T04:55:34.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 -- start start 2024-02-26T04:55:34.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71381011b0 0x7f7138074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:34.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71381023a0 0x7f7138073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:34.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7138102d60 0x7f7138073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:34.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7138074210 con 0x7f71381023a0 2024-02-26T04:55:34.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7138074380 con 0x7f7138102d60 2024-02-26T04:55:34.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f713f236640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71381a1220 con 0x7f71381011b0 2024-02-26T04:55:34.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.583+0000 7f7137fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71381023a0 0x7f7138073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:34.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f7137fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71381023a0 0x7f7138073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44074/0 (socket says 172.21.15.60:44074) 2024-02-26T04:55:34.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f7137fff640 1 -- 172.21.15.60:0/2902340490 learned_addr learned my addr 172.21.15.60:0/2902340490 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:34.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f713d7d3640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7138102d60 0x7f7138073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:34.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f713cfd2640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71381011b0 0x7f7138074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:34.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f7137fff640 1 -- 172.21.15.60:0/2902340490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71381011b0 msgr2=0x7f7138074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f7137fff640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71381011b0 0x7f7138074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f7137fff640 1 -- 172.21.15.60:0/2902340490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7138102d60 msgr2=0x7f7138073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f7137fff640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7138102d60 0x7f7138073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f7137fff640 1 -- 172.21.15.60:0/2902340490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7130009900 con 0x7f71381023a0 2024-02-26T04:55:34.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f713d7d3640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7138102d60 0x7f7138073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:34.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f7137fff640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71381023a0 0x7f7138073570 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f712400c8c0 tx=0x7f712400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:34.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f7135ffb640 1 -- 172.21.15.60:0/2902340490 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124004220 con 0x7f71381023a0 2024-02-26T04:55:34.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f7135ffb640 1 -- 172.21.15.60:0/2902340490 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7124007e90 con 0x7f71381023a0 2024-02-26T04:55:34.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f713f236640 1 -- 172.21.15.60:0/2902340490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71381a14d0 con 0x7f71381023a0 2024-02-26T04:55:34.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f7135ffb640 1 -- 172.21.15.60:0/2902340490 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124004a00 con 0x7f71381023a0 2024-02-26T04:55:34.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.584+0000 7f713f236640 1 -- 172.21.15.60:0/2902340490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71381a19e0 con 0x7f71381023a0 2024-02-26T04:55:34.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.585+0000 7f713f236640 1 -- 172.21.15.60:0/2902340490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7138108470 con 0x7f71381023a0 2024-02-26T04:55:34.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.586+0000 7f7135ffb640 1 -- 172.21.15.60:0/2902340490 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f71240079b0 con 0x7f71381023a0 2024-02-26T04:55:34.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.587+0000 7f7135ffb640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f71280a20c0 0x7f71280a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:34.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.587+0000 7f713cfd2640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f71280a20c0 0x7f71280a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:34.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.587+0000 7f7135ffb640 1 -- 172.21.15.60:0/2902340490 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f71240cdce0 con 0x7f71381023a0 2024-02-26T04:55:34.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.588+0000 7f713cfd2640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f71280a20c0 0x7f71280a4580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f712c005fd0 tx=0x7f712c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:34.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.592+0000 7f7135ffb640 1 -- 172.21.15.60:0/2902340490 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7124096a10 con 0x7f71381023a0 2024-02-26T04:55:34.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:34 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2698441341' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-26T04:55:34.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:34 smithi060 ceph-mon[32382]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T04:55:34.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:34 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2912323221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:34.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:34 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/238378866' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:34.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:34 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2015325954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:34.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:34 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2698441341' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-26T04:55:34.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:34 smithi191 ceph-mon[36352]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T04:55:34.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:34 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2912323221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:34.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:34 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/238378866' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:34.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:34 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2015325954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:34.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.812+0000 7f713f236640 1 -- 172.21.15.60:0/2902340490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f713810cbc0 con 0x7f71381023a0 2024-02-26T04:55:34.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.813+0000 7f7135ffb640 1 -- 172.21.15.60:0/2902340490 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+11286 (secure 0 0 0) 0x7f712409a0d0 con 0x7f71381023a0 2024-02-26T04:55:34.815 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:34.815 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:55:32.581648+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-26T04:55:34.815 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2024-02-26T04:55:34.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.815+0000 7f713f236640 1 -- 172.21.15.60:0/2902340490 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f71280a20c0 msgr2=0x7f71280a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.815+0000 7f713f236640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f71280a20c0 0x7f71280a4580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f712c005fd0 tx=0x7f712c005850 comp rx=0 tx=0).stop 2024-02-26T04:55:34.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.815+0000 7f713f236640 1 -- 172.21.15.60:0/2902340490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71381023a0 msgr2=0x7f7138073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.815+0000 7f713f236640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71381023a0 0x7f7138073570 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f712400c8c0 tx=0x7f712400cd80 comp rx=0 tx=0).stop 2024-02-26T04:55:34.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.815+0000 7f713f236640 1 -- 172.21.15.60:0/2902340490 shutdown_connections 2024-02-26T04:55:34.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.815+0000 7f713f236640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f71280a20c0 0x7f71280a4580 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.815+0000 7f713f236640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7138102d60 0x7f7138073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.815+0000 7f713f236640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71381023a0 0x7f7138073570 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.815+0000 7f713f236640 1 --2- 172.21.15.60:0/2902340490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71381011b0 0x7f7138074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.815+0000 7f713f236640 1 -- 172.21.15.60:0/2902340490 >> 172.21.15.60:0/2902340490 conn(0x7f71380fc760 msgr2=0x7f7138105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:34.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.815+0000 7f713f236640 1 -- 172.21.15.60:0/2902340490 shutdown_connections 2024-02-26T04:55:34.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.815+0000 7f713f236640 1 -- 172.21.15.60:0/2902340490 wait complete. 2024-02-26T04:55:34.828 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:55:34.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:34 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2698441341' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-26T04:55:34.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:34 smithi156 ceph-mon[35805]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T04:55:34.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:34 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2912323221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:34.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:34 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/238378866' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:34.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:34 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2015325954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:34.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.928+0000 7ff0074de640 1 -- 172.21.15.60:0/769681542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff000101c60 msgr2=0x7ff0001084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.928+0000 7ff0074de640 1 --2- 172.21.15.60:0/769681542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff000101c60 0x7ff0001084f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feff8009ca0 tx=0x7feff802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.928+0000 7ff0074de640 1 -- 172.21.15.60:0/769681542 shutdown_connections 2024-02-26T04:55:34.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.928+0000 7ff0074de640 1 --2- 172.21.15.60:0/769681542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff000101c60 0x7ff0001084f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.928+0000 7ff0074de640 1 --2- 172.21.15.60:0/769681542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0001012a0 0x7ff000101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.928+0000 7ff0074de640 1 --2- 172.21.15.60:0/769681542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0001000b0 0x7ff0001004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.928+0000 7ff0074de640 1 -- 172.21.15.60:0/769681542 >> 172.21.15.60:0/769681542 conn(0x7ff0000fb660 msgr2=0x7ff0000fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:34.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.928+0000 7ff0074de640 1 -- 172.21.15.60:0/769681542 shutdown_connections 2024-02-26T04:55:34.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.928+0000 7ff0074de640 1 -- 172.21.15.60:0/769681542 wait complete. 2024-02-26T04:55:34.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.928+0000 7ff0074de640 1 Processor -- start 2024-02-26T04:55:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.928+0000 7ff0074de640 1 -- start start 2024-02-26T04:55:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff0074de640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0001000b0 0x7ff00006a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff0074de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0001012a0 0x7ff00019bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff0074de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff000101c60 0x7ff00019e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff0074de640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00019ecd0 con 0x7ff000101c60 2024-02-26T04:55:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff0074de640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00019ee40 con 0x7ff0001000b0 2024-02-26T04:55:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff0074de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00019f000 con 0x7ff0001012a0 2024-02-26T04:55:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff005a7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff000101c60 0x7ff00019e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff005a7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff000101c60 0x7ff00019e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44098/0 (socket says 172.21.15.60:44098) 2024-02-26T04:55:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff005a7b640 1 -- 172.21.15.60:0/2188864281 learned_addr learned my addr 172.21.15.60:0/2188864281 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff00527a640 1 --2- 172.21.15.60:0/2188864281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0001000b0 0x7ff00006a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff004a79640 1 --2- 172.21.15.60:0/2188864281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0001012a0 0x7ff00019bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff005a7b640 1 -- 172.21.15.60:0/2188864281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0001012a0 msgr2=0x7ff00019bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff005a7b640 1 --2- 172.21.15.60:0/2188864281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0001012a0 0x7ff00019bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff005a7b640 1 -- 172.21.15.60:0/2188864281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0001000b0 msgr2=0x7ff00006a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff005a7b640 1 --2- 172.21.15.60:0/2188864281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0001000b0 0x7ff00006a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff005a7b640 1 -- 172.21.15.60:0/2188864281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff8009900 con 0x7ff000101c60 2024-02-26T04:55:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff005a7b640 1 --2- 172.21.15.60:0/2188864281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff000101c60 0x7ff00019e540 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feff802c630 tx=0x7feff8034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7feff67fc640 1 -- 172.21.15.60:0/2188864281 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8032eb0 con 0x7ff000101c60 2024-02-26T04:55:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7feff67fc640 1 -- 172.21.15.60:0/2188864281 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feff8003e50 con 0x7ff000101c60 2024-02-26T04:55:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff0074de640 1 -- 172.21.15.60:0/2188864281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00019f280 con 0x7ff000101c60 2024-02-26T04:55:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7feff67fc640 1 -- 172.21.15.60:0/2188864281 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8040450 con 0x7ff000101c60 2024-02-26T04:55:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.929+0000 7ff0074de640 1 -- 172.21.15.60:0/2188864281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00019f730 con 0x7ff000101c60 2024-02-26T04:55:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.930+0000 7ff0074de640 1 -- 172.21.15.60:0/2188864281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0001073d0 con 0x7ff000101c60 2024-02-26T04:55:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.931+0000 7feff67fc640 1 -- 172.21.15.60:0/2188864281 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7feff800a040 con 0x7ff000101c60 2024-02-26T04:55:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.932+0000 7feff67fc640 1 --2- 172.21.15.60:0/2188864281 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fefec0a20c0 0x7fefec0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.932+0000 7ff00527a640 1 --2- 172.21.15.60:0/2188864281 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fefec0a20c0 0x7fefec0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.932+0000 7feff67fc640 1 -- 172.21.15.60:0/2188864281 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7feff80f1200 con 0x7ff000101c60 2024-02-26T04:55:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.932+0000 7ff00527a640 1 --2- 172.21.15.60:0/2188864281 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fefec0a20c0 0x7fefec0a4580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff000101100 tx=0x7feffc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:34.935+0000 7feff67fc640 1 -- 172.21.15.60:0/2188864281 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feff80ba030 con 0x7ff000101c60 2024-02-26T04:55:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.118+0000 7ff0074de640 1 -- 172.21.15.60:0/2188864281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff00002ce40 con 0x7ff000101c60 2024-02-26T04:55:35.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.119+0000 7feff67fc640 1 -- 172.21.15.60:0/2188864281 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24126 (secure 0 0 0) 0x7feff80bd6f0 con 0x7ff000101c60 2024-02-26T04:55:35.122 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:35.122 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:55:32.581481+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:55:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.121+0000 7ff0074de640 1 -- 172.21.15.60:0/2188864281 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fefec0a20c0 msgr2=0x7fefec0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.121+0000 7ff0074de640 1 --2- 172.21.15.60:0/2188864281 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fefec0a20c0 0x7fefec0a4580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff000101100 tx=0x7feffc005f90 comp rx=0 tx=0).stop 2024-02-26T04:55:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.121+0000 7ff0074de640 1 -- 172.21.15.60:0/2188864281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff000101c60 msgr2=0x7ff00019e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.121+0000 7ff0074de640 1 --2- 172.21.15.60:0/2188864281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff000101c60 0x7ff00019e540 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feff802c630 tx=0x7feff8034cc0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.121+0000 7ff0074de640 1 -- 172.21.15.60:0/2188864281 shutdown_connections 2024-02-26T04:55:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.121+0000 7ff0074de640 1 --2- 172.21.15.60:0/2188864281 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fefec0a20c0 0x7fefec0a4580 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.121+0000 7ff0074de640 1 --2- 172.21.15.60:0/2188864281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff000101c60 0x7ff00019e540 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.121+0000 7ff0074de640 1 --2- 172.21.15.60:0/2188864281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0001012a0 0x7ff00019bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.121+0000 7ff0074de640 1 --2- 172.21.15.60:0/2188864281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0001000b0 0x7ff00006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.121+0000 7ff0074de640 1 -- 172.21.15.60:0/2188864281 >> 172.21.15.60:0/2188864281 conn(0x7ff0000fb660 msgr2=0x7ff0001045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:35.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.122+0000 7ff0074de640 1 -- 172.21.15.60:0/2188864281 shutdown_connections 2024-02-26T04:55:35.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.122+0000 7ff0074de640 1 -- 172.21.15.60:0/2188864281 wait complete. 2024-02-26T04:55:35.136 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-26T04:55:27.553789+0000', 'modified': '2024-02-26T04:55:32.581648+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24524}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.156:6839/1768629491', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 1768629491}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 1768629491}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-02-26T04:55:35.136 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-26T04:55:35.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-26T04:55:35.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.238+0000 7f284f18c640 1 -- 172.21.15.60:0/2731491281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2848104540 msgr2=0x7f2848106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:35.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.238+0000 7f284f18c640 1 --2- 172.21.15.60:0/2731491281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2848104540 0x7f2848106930 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2838009ca0 tx=0x7f283802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.238+0000 7f284f18c640 1 -- 172.21.15.60:0/2731491281 shutdown_connections 2024-02-26T04:55:35.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.238+0000 7f284f18c640 1 --2- 172.21.15.60:0/2731491281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2848106e70 0x7f28481092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.238+0000 7f284f18c640 1 --2- 172.21.15.60:0/2731491281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2848104540 0x7f2848106930 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.238+0000 7f284f18c640 1 --2- 172.21.15.60:0/2731491281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2848101c10 0x7f2848104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.238+0000 7f284f18c640 1 -- 172.21.15.60:0/2731491281 >> 172.21.15.60:0/2731491281 conn(0x7f28480fb620 msgr2=0x7f28480fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.238+0000 7f284f18c640 1 -- 172.21.15.60:0/2731491281 shutdown_connections 2024-02-26T04:55:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.238+0000 7f284f18c640 1 -- 172.21.15.60:0/2731491281 wait complete. 2024-02-26T04:55:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284f18c640 1 Processor -- start 2024-02-26T04:55:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284f18c640 1 -- start start 2024-02-26T04:55:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284f18c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2848101c10 0x7f28481972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284f18c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2848104540 0x7f28481977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284f18c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2848106e70 0x7f284819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284f18c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284819eda0 con 0x7f2848106e70 2024-02-26T04:55:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284f18c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284819ef10 con 0x7f2848104540 2024-02-26T04:55:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284f18c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284819f0d0 con 0x7f2848101c10 2024-02-26T04:55:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284d729640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2848106e70 0x7f284819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284d729640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2848106e70 0x7f284819e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44108/0 (socket says 172.21.15.60:44108) 2024-02-26T04:55:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284d729640 1 -- 172.21.15.60:0/828209658 learned_addr learned my addr 172.21.15.60:0/828209658 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f2847fff640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2848104540 0x7f28481977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284cf28640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2848101c10 0x7f28481972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284d729640 1 -- 172.21.15.60:0/828209658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2848101c10 msgr2=0x7f28481972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284d729640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2848101c10 0x7f28481972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284d729640 1 -- 172.21.15.60:0/828209658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2848104540 msgr2=0x7f28481977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284d729640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2848104540 0x7f28481977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284d729640 1 -- 172.21.15.60:0/828209658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2838009900 con 0x7f2848106e70 2024-02-26T04:55:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f2847fff640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2848104540 0x7f28481977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284cf28640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2848101c10 0x7f28481972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f284d729640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2848106e70 0x7f284819e860 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f283000c970 tx=0x7f283000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.239+0000 7f2845ffb640 1 -- 172.21.15.60:0/828209658 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2830007bb0 con 0x7f2848106e70 2024-02-26T04:55:35.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.240+0000 7f2845ffb640 1 -- 172.21.15.60:0/828209658 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2830007d10 con 0x7f2848106e70 2024-02-26T04:55:35.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.240+0000 7f284f18c640 1 -- 172.21.15.60:0/828209658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284819f3b0 con 0x7f2848106e70 2024-02-26T04:55:35.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.240+0000 7f2845ffb640 1 -- 172.21.15.60:0/828209658 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000f450 con 0x7f2848106e70 2024-02-26T04:55:35.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.240+0000 7f284f18c640 1 -- 172.21.15.60:0/828209658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284819f840 con 0x7f2848106e70 2024-02-26T04:55:35.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.242+0000 7f2845ffb640 1 -- 172.21.15.60:0/828209658 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2830004f00 con 0x7f2848106e70 2024-02-26T04:55:35.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.242+0000 7f2851f64740 1 Processor -- start 2024-02-26T04:55:35.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.242+0000 7f2845ffb640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f28340a20c0 0x7f28340a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.242+0000 7f2851f64740 1 -- start start 2024-02-26T04:55:35.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.242+0000 7f2847fff640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f28340a20c0 0x7f28340a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.242+0000 7f2851f64740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb9619f50 0x560cb961a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.242+0000 7f2851f64740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb961a940 0x560cb961f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.242+0000 7f2845ffb640 1 -- 172.21.15.60:0/828209658 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f28300cdc30 con 0x7f2848106e70 2024-02-26T04:55:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.242+0000 7f2851f64740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961fab0 0x560cb9621f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.242+0000 7f2851f64740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb95d5ca0 con 0x560cb961fab0 2024-02-26T04:55:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.242+0000 7f2851f64740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb9588ef0 con 0x560cb961a940 2024-02-26T04:55:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.242+0000 7f2851f64740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb9586420 con 0x560cb9619f50 2024-02-26T04:55:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f284d729640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961fab0 0x560cb9621f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f284d729640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961fab0 0x560cb9621f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44124/0 (socket says 172.21.15.60:44124) 2024-02-26T04:55:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f284d729640 1 -- 172.21.15.60:0/3556572906 learned_addr learned my addr 172.21.15.60:0/3556572906 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f284cf28640 1 --2- 172.21.15.60:0/3556572906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb9619f50 0x560cb961a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f284d729640 1 -- 172.21.15.60:0/3556572906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb9619f50 msgr2=0x560cb961a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:55:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f284d729640 1 --2- 172.21.15.60:0/3556572906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb9619f50 0x560cb961a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f284d729640 1 -- 172.21.15.60:0/3556572906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb961a940 msgr2=0x560cb961f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:55:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f284d729640 1 --2- 172.21.15.60:0/3556572906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb961a940 0x560cb961f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f284d729640 1 -- 172.21.15.60:0/3556572906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cb9536ef0 con 0x560cb961fab0 2024-02-26T04:55:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f284cf28640 1 --2- 172.21.15.60:0/3556572906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb9619f50 0x560cb961a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f2847fff640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f28340a20c0 0x7f28340a4580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f283802c700 tx=0x7f2838032e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f284d729640 1 --2- 172.21.15.60:0/3556572906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961fab0 0x560cb9621f40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2830096b80 tx=0x7f283009a260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a05eb444d1bd4b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f283effd640 1 -- 172.21.15.60:0/3556572906 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283009a720 con 0x560cb961fab0 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f283effd640 1 -- 172.21.15.60:0/3556572906 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28300b8e90 con 0x560cb961fab0 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.243+0000 7f283effd640 1 -- 172.21.15.60:0/3556572906 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2830098b00 con 0x560cb961fab0 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.244+0000 7f2851f64740 1 -- 172.21.15.60:0/3556572906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961fab0 msgr2=0x560cb9621f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.244+0000 7f2851f64740 1 --2- 172.21.15.60:0/3556572906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961fab0 0x560cb9621f40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2830096b80 tx=0x7f283009a260 comp rx=0 tx=0).stop 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.244+0000 7f2851f64740 1 -- 172.21.15.60:0/3556572906 shutdown_connections 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.244+0000 7f2851f64740 1 --2- 172.21.15.60:0/3556572906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961fab0 0x560cb9621f40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.244+0000 7f2851f64740 1 --2- 172.21.15.60:0/3556572906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb961a940 0x560cb961f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.244+0000 7f2851f64740 1 --2- 172.21.15.60:0/3556572906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb9619f50 0x560cb961a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.244+0000 7f2851f64740 1 -- 172.21.15.60:0/3556572906 >> 172.21.15.60:0/3556572906 conn(0x560cb96154c0 msgr2=0x560cb96158c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.244+0000 7f2851f64740 1 -- 172.21.15.60:0/3556572906 shutdown_connections 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.244+0000 7f2851f64740 1 -- 172.21.15.60:0/3556572906 wait complete. 2024-02-26T04:55:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.244+0000 7f2851f64740 1 -- start start 2024-02-26T04:55:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.244+0000 7f2851f64740 1 Processor -- start 2024-02-26T04:55:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f2851f64740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb9619f50 0x560cb9624140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f2851f64740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961a940 0x560cb9624680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f2851f64740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb961fab0 0x560cb962b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f2851f64740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb9581c00 con 0x560cb9619f50 2024-02-26T04:55:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f2851f64740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb9526d00 con 0x560cb961fab0 2024-02-26T04:55:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f2851f64740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb9255310 con 0x560cb961a940 2024-02-26T04:55:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f2851f64740 10 client.0 _ll_register_callbacks cb 0x560cb952ed10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:55:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f284cf28640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb9619f50 0x560cb9624140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f284cf28640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb9619f50 0x560cb9624140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44140/0 (socket says 172.21.15.60:44140) 2024-02-26T04:55:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f284cf28640 1 -- 172.21.15.60:0/2703721363 learned_addr learned my addr 172.21.15.60:0/2703721363 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f284d729640 1 --2- 172.21.15.60:0/2703721363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb961fab0 0x560cb962b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f2847fff640 1 --2- 172.21.15.60:0/2703721363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961a940 0x560cb9624680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f284cf28640 1 -- 172.21.15.60:0/2703721363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961a940 msgr2=0x560cb9624680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:55:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f284cf28640 1 --2- 172.21.15.60:0/2703721363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961a940 0x560cb9624680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f284cf28640 1 -- 172.21.15.60:0/2703721363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb961fab0 msgr2=0x560cb962b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:55:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f284cf28640 1 --2- 172.21.15.60:0/2703721363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb961fab0 0x560cb962b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f284cf28640 1 -- 172.21.15.60:0/2703721363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284819f840 con 0x560cb9619f50 2024-02-26T04:55:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f284cf28640 1 --2- 172.21.15.60:0/2703721363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb9619f50 0x560cb9624140 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2840013710 tx=0x7f2840013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6b3221c7a75563 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f2827fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:55:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f2827fff640 1 -- 172.21.15.60:0/2703721363 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840011d70 con 0x560cb9619f50 2024-02-26T04:55:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f2827fff640 1 -- 172.21.15.60:0/2703721363 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f284000f450 con 0x560cb9619f50 2024-02-26T04:55:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.245+0000 7f2851f64740 1 -- 172.21.15.60:0/2703721363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cb92816c0 con 0x560cb9619f50 2024-02-26T04:55:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.246+0000 7f2827fff640 1 -- 172.21.15.60:0/2703721363 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2840011ed0 con 0x560cb9619f50 2024-02-26T04:55:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.246+0000 7f2851f64740 10 client.14742 fetch_fsmap learned FSMap version 16 2024-02-26T04:55:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.246+0000 7f2851f64740 1 -- 172.21.15.60:0/2703721363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x560cb9580280 con 0x560cb9619f50 2024-02-26T04:55:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.246+0000 7f2827fff640 1 -- 172.21.15.60:0/2703721363 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560cb92816c0 con 0x560cb9619f50 2024-02-26T04:55:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.246+0000 7f2827fff640 1 -- 172.21.15.60:0/2703721363 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 16) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f284000e850 con 0x560cb9619f50 2024-02-26T04:55:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.246+0000 7f2851f64740 10 client.14742 fetch_fsmap finished waiting for FSMap version 16 2024-02-26T04:55:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.246+0000 7f2851f64740 10 client.14742 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T04:55:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.246+0000 7f2851f64740 20 client.14742 populate_metadata read hostname 'smithi060' 2024-02-26T04:55:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.246+0000 7f2851f64740 1 --2- 172.21.15.60:0/2703721363 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x560cb9630b90 0x560cb962fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.246+0000 7f2851f64740 4 client.14742 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:55:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.246+0000 7f2851f64740 1 -- 172.21.15.60:0/2703721363 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cb92d2270 con 0x560cb9630b90 2024-02-26T04:55:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.246+0000 7f2847fff640 1 --2- 172.21.15.60:0/2703721363 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x560cb9630b90 0x560cb962fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.247+0000 7f2847fff640 1 --2- 172.21.15.60:0/2703721363 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x560cb9630b90 0x560cb962fe40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b583247c93b2794 server_cookie=326164c966baf5b9 in_seq=0 out_seq=0 2024-02-26T04:55:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.247+0000 7f2827fff640 10 client.14742 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:55:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.247+0000 7f2827fff640 1 -- 172.21.15.60:0/2703721363 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560cb92d2270 con 0x560cb9630b90 2024-02-26T04:55:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.247+0000 7f2827fff640 10 client.14742 handle_command_reply: tid=0 2024-02-26T04:55:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.247+0000 7f2851f64740 1 client.14742 shutdown 2024-02-26T04:55:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.248+0000 7f2851f64740 1 -- 172.21.15.60:0/2703721363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb9619f50 msgr2=0x560cb9624140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.248+0000 7f2851f64740 1 --2- 172.21.15.60:0/2703721363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb9619f50 0x560cb9624140 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2840013710 tx=0x7f2840013bd0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.248+0000 7f2851f64740 1 -- 172.21.15.60:0/2703721363 shutdown_connections 2024-02-26T04:55:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.248+0000 7f2851f64740 1 --2- 172.21.15.60:0/2703721363 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x560cb9630b90 0x560cb962fe40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.248+0000 7f2851f64740 1 --2- 172.21.15.60:0/2703721363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb961fab0 0x560cb962b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.248+0000 7f2827fff640 0 client.14742 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:55:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.248+0000 7f2851f64740 1 --2- 172.21.15.60:0/2703721363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961a940 0x560cb9624680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.248+0000 7f2851f64740 1 --2- 172.21.15.60:0/2703721363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb9619f50 0x560cb9624140 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.248+0000 7f2851f64740 1 -- 172.21.15.60:0/2703721363 >> 172.21.15.60:0/2703721363 conn(0x560cb96154c0 msgr2=0x560cb96147a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.248+0000 7f2851f64740 1 -- 172.21.15.60:0/2703721363 shutdown_connections 2024-02-26T04:55:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.248+0000 7f2851f64740 1 -- 172.21.15.60:0/2703721363 wait complete. 2024-02-26T04:55:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.248+0000 7f2851f64740 20 client.14742 trim_cache size 0 max 16384 2024-02-26T04:55:35.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.263+0000 7f2851f64740 1 Processor -- start 2024-02-26T04:55:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.263+0000 7f2851f64740 1 -- start start 2024-02-26T04:55:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2851f64740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb9619f50 0x560cb962c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2851f64740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961a940 0x560cb962b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2851f64740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961fab0 0x560cb961be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2851f64740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb9507460 con 0x560cb961a940 2024-02-26T04:55:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2851f64740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb9615f10 con 0x560cb9619f50 2024-02-26T04:55:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2851f64740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb9619b10 con 0x560cb961fab0 2024-02-26T04:55:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2847fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961a940 0x560cb962b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2847fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961a940 0x560cb962b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44142/0 (socket says 172.21.15.60:44142) 2024-02-26T04:55:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2847fff640 1 -- 172.21.15.60:0/2322669905 learned_addr learned my addr 172.21.15.60:0/2322669905 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:35.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f284cf28640 1 --2- 172.21.15.60:0/2322669905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb9619f50 0x560cb962c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f284d729640 1 --2- 172.21.15.60:0/2322669905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961fab0 0x560cb961be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2847fff640 1 -- 172.21.15.60:0/2322669905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961fab0 msgr2=0x560cb961be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:55:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2847fff640 1 --2- 172.21.15.60:0/2322669905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961fab0 0x560cb961be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2847fff640 1 -- 172.21.15.60:0/2322669905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb9619f50 msgr2=0x560cb962c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:55:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2847fff640 1 --2- 172.21.15.60:0/2322669905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb9619f50 0x560cb962c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2847fff640 1 -- 172.21.15.60:0/2322669905 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cb961f840 con 0x560cb961a940 2024-02-26T04:55:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f284d729640 1 --2- 172.21.15.60:0/2322669905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961fab0 0x560cb961be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f2847fff640 1 --2- 172.21.15.60:0/2322669905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961a940 0x560cb962b5e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2838032540 tx=0x7f2838040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67ba124e334fa02 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f283e7fc640 1 -- 172.21.15.60:0/2322669905 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838034c90 con 0x560cb961a940 2024-02-26T04:55:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f283e7fc640 1 -- 172.21.15.60:0/2322669905 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2838003e50 con 0x560cb961a940 2024-02-26T04:55:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.264+0000 7f283e7fc640 1 -- 172.21.15.60:0/2322669905 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838004800 con 0x560cb961a940 2024-02-26T04:55:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.265+0000 7f2851f64740 1 -- 172.21.15.60:0/2322669905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961a940 msgr2=0x560cb962b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:35.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.265+0000 7f2851f64740 1 --2- 172.21.15.60:0/2322669905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961a940 0x560cb962b5e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2838032540 tx=0x7f2838040000 comp rx=0 tx=0).stop 2024-02-26T04:55:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.265+0000 7f2851f64740 1 -- 172.21.15.60:0/2322669905 shutdown_connections 2024-02-26T04:55:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.265+0000 7f2851f64740 1 --2- 172.21.15.60:0/2322669905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961fab0 0x560cb961be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.265+0000 7f2851f64740 1 --2- 172.21.15.60:0/2322669905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961a940 0x560cb962b5e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.265+0000 7f2851f64740 1 --2- 172.21.15.60:0/2322669905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb9619f50 0x560cb962c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.265+0000 7f2851f64740 1 -- 172.21.15.60:0/2322669905 >> 172.21.15.60:0/2322669905 conn(0x560cb96154c0 msgr2=0x560cb9622c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.265+0000 7f2851f64740 1 -- 172.21.15.60:0/2322669905 shutdown_connections 2024-02-26T04:55:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.265+0000 7f2851f64740 1 -- 172.21.15.60:0/2322669905 wait complete. 2024-02-26T04:55:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.265+0000 7f2851f64740 1 -- start start 2024-02-26T04:55:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.265+0000 7f2851f64740 1 Processor -- start 2024-02-26T04:55:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2851f64740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb9619f50 0x560cb9628770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2851f64740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961a940 0x560cb9628cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2851f64740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961fab0 0x560cb9629cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2851f64740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb962a200 con 0x560cb961fab0 2024-02-26T04:55:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2851f64740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb962a370 con 0x560cb9619f50 2024-02-26T04:55:35.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2851f64740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb962a530 con 0x560cb961a940 2024-02-26T04:55:35.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2851f64740 10 client.0 _ll_register_callbacks cb 0x7f2830004840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:55:35.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f284d729640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961fab0 0x560cb9629cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f284d729640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961fab0 0x560cb9629cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44158/0 (socket says 172.21.15.60:44158) 2024-02-26T04:55:35.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f284d729640 1 -- 172.21.15.60:0/2977701437 learned_addr learned my addr 172.21.15.60:0/2977701437 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:35.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2847fff640 1 --2- 172.21.15.60:0/2977701437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961a940 0x560cb9628cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f284cf28640 1 --2- 172.21.15.60:0/2977701437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb9619f50 0x560cb9628770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f284d729640 1 -- 172.21.15.60:0/2977701437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961a940 msgr2=0x560cb9628cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:55:35.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f284d729640 1 --2- 172.21.15.60:0/2977701437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961a940 0x560cb9628cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f284d729640 1 -- 172.21.15.60:0/2977701437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb9619f50 msgr2=0x560cb9628770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:55:35.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f284d729640 1 --2- 172.21.15.60:0/2977701437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb9619f50 0x560cb9628770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f284d729640 1 -- 172.21.15.60:0/2977701437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2848109ea0 con 0x560cb961fab0 2024-02-26T04:55:35.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f284cf28640 1 --2- 172.21.15.60:0/2977701437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb9619f50 0x560cb9628770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:35.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f284d729640 1 --2- 172.21.15.60:0/2977701437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961fab0 0x560cb9629cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2830007590 tx=0x7f28300048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7076aedd3dfcdcb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:35.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2827fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:55:35.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2827fff640 1 -- 172.21.15.60:0/2977701437 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283009a720 con 0x560cb961fab0 2024-02-26T04:55:35.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2827fff640 1 -- 172.21.15.60:0/2977701437 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f28300a2ec0 con 0x560cb961fab0 2024-02-26T04:55:35.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2851f64740 1 -- 172.21.15.60:0/2977701437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cb9631560 con 0x560cb961fab0 2024-02-26T04:55:35.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2827fff640 1 -- 172.21.15.60:0/2977701437 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f283009e870 con 0x560cb961fab0 2024-02-26T04:55:35.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2851f64740 10 client.14748 fetch_fsmap learned FSMap version 16 2024-02-26T04:55:35.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.266+0000 7f2851f64740 1 -- 172.21.15.60:0/2977701437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x560cb951c9f0 con 0x560cb961fab0 2024-02-26T04:55:35.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.267+0000 7f2827fff640 1 -- 172.21.15.60:0/2977701437 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f283007f020 con 0x560cb961fab0 2024-02-26T04:55:35.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.267+0000 7f2827fff640 1 -- 172.21.15.60:0/2977701437 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 16) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2830016030 con 0x560cb961fab0 2024-02-26T04:55:35.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.267+0000 7f2851f64740 10 client.14748 fetch_fsmap finished waiting for FSMap version 16 2024-02-26T04:55:35.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.267+0000 7f2851f64740 10 client.14748 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T04:55:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.267+0000 7f2851f64740 20 client.14748 populate_metadata read hostname 'smithi060' 2024-02-26T04:55:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.267+0000 7f2851f64740 1 --2- 172.21.15.60:0/2977701437 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x560cb968c0d0 0x560cb96887a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.267+0000 7f2851f64740 4 client.14748 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T04:55:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.267+0000 7f2851f64740 1 -- 172.21.15.60:0/2977701437 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f28300b8e90 con 0x560cb968c0d0 2024-02-26T04:55:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.267+0000 7f284cf28640 1 --2- 172.21.15.60:0/2977701437 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x560cb968c0d0 0x560cb96887a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.267+0000 7f2827fff640 10 client.14748 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:55:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.267+0000 7f284cf28640 1 --2- 172.21.15.60:0/2977701437 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x560cb968c0d0 0x560cb96887a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=633bf8385f5c1638 server_cookie=c719547e839284c8 in_seq=0 out_seq=0 2024-02-26T04:55:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2827fff640 1 -- 172.21.15.60:0/2977701437 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x7f28300b8e90 con 0x560cb968c0d0 2024-02-26T04:55:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2827fff640 10 client.14748 handle_command_reply: tid=0 2024-02-26T04:55:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2851f64740 1 client.14748 shutdown 2024-02-26T04:55:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2851f64740 1 -- 172.21.15.60:0/2977701437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961fab0 msgr2=0x560cb9629cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2851f64740 1 --2- 172.21.15.60:0/2977701437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961fab0 0x560cb9629cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2830007590 tx=0x7f28300048f0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2851f64740 1 -- 172.21.15.60:0/2977701437 shutdown_connections 2024-02-26T04:55:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2851f64740 1 --2- 172.21.15.60:0/2977701437 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x560cb968c0d0 0x560cb96887a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2851f64740 1 --2- 172.21.15.60:0/2977701437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560cb961fab0 0x560cb9629cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2827fff640 0 client.14748 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:55:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2851f64740 1 --2- 172.21.15.60:0/2977701437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cb961a940 0x560cb9628cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2851f64740 1 --2- 172.21.15.60:0/2977701437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560cb9619f50 0x560cb9628770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2851f64740 1 -- 172.21.15.60:0/2977701437 >> 172.21.15.60:0/2977701437 conn(0x560cb96154c0 msgr2=0x560cb961be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2851f64740 1 -- 172.21.15.60:0/2977701437 shutdown_connections 2024-02-26T04:55:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2851f64740 1 -- 172.21.15.60:0/2977701437 wait complete. 2024-02-26T04:55:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f2851f64740 20 client.14748 trim_cache size 0 max 16384 2024-02-26T04:55:35.279 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:55:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "f12e2206-d461-11ee-95c3-87774f69a715", 2024-02-26T04:55:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-02-26T04:55:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24524, 2024-02-26T04:55:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-02-26T04:55:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-02-26T04:55:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-02-26T04:55:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 7.083304826, 2024-02-26T04:55:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 16, 2024-02-26T04:55:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 89, 2024-02-26T04:55:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 89, 2024-02-26T04:55:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 13.318835738000001 2024-02-26T04:55:35.280 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:55:35.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f284f18c640 1 -- 172.21.15.60:0/828209658 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f28340a20c0 msgr2=0x7f28340a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:35.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f284f18c640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f28340a20c0 0x7f28340a4580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f283802c700 tx=0x7f2838032e50 comp rx=0 tx=0).stop 2024-02-26T04:55:35.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f284f18c640 1 -- 172.21.15.60:0/828209658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2848106e70 msgr2=0x7f284819e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:35.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f284f18c640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2848106e70 0x7f284819e860 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f283000c970 tx=0x7f283000ce30 comp rx=0 tx=0).stop 2024-02-26T04:55:35.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f284f18c640 1 -- 172.21.15.60:0/828209658 shutdown_connections 2024-02-26T04:55:35.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f284f18c640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f28340a20c0 0x7f28340a4580 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f284f18c640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2848106e70 0x7f284819e860 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f284f18c640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2848104540 0x7f28481977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f284f18c640 1 --2- 172.21.15.60:0/828209658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2848101c10 0x7f28481972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:35.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.268+0000 7f284f18c640 1 -- 172.21.15.60:0/828209658 >> 172.21.15.60:0/828209658 conn(0x7f28480fb620 msgr2=0x7f2848102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:35.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.269+0000 7f284f18c640 1 -- 172.21.15.60:0/828209658 shutdown_connections 2024-02-26T04:55:35.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:35.269+0000 7f284f18c640 1 -- 172.21.15.60:0/828209658 wait complete. 2024-02-26T04:55:35.283 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-26T04:55:35.283 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-02-26T04:55:35.283 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph fs ls 2024-02-26T04:55:35.523 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:55:35.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:35 smithi060 ceph-mon[32382]: pgmap v297: 129 pgs: 5 creating+peering, 20 unknown, 104 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 748 B/s wr, 3 op/s 2024-02-26T04:55:35.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:35 smithi060 ceph-mon[32382]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T04:55:35.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:35 smithi060 ceph-mon[32382]: Cluster is now healthy 2024-02-26T04:55:35.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:35 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2902340490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:35.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:35 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2188864281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:35 smithi191 ceph-mon[36352]: pgmap v297: 129 pgs: 5 creating+peering, 20 unknown, 104 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 748 B/s wr, 3 op/s 2024-02-26T04:55:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:35 smithi191 ceph-mon[36352]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T04:55:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:35 smithi191 ceph-mon[36352]: Cluster is now healthy 2024-02-26T04:55:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:35 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2902340490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:35.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:35 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2188864281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:35.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:35 smithi156 ceph-mon[35805]: pgmap v297: 129 pgs: 5 creating+peering, 20 unknown, 104 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 748 B/s wr, 3 op/s 2024-02-26T04:55:35.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:35 smithi156 ceph-mon[35805]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-26T04:55:35.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:35 smithi156 ceph-mon[35805]: Cluster is now healthy 2024-02-26T04:55:35.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:35 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2902340490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:35.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:35 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2188864281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.249+0000 7f3c240ef700 1 -- 172.21.15.60:0/4209225998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c1c06a5d0 msgr2=0x7f3c1c06a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.249+0000 7f3c240ef700 1 --2- 172.21.15.60:0/4209225998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c1c06a5d0 0x7f3c1c06a9b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00b0a0 tx=0x7f3c0c00b3b0 comp rx=0 tx=0).stop 2024-02-26T04:55:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.249+0000 7f3c240ef700 1 -- 172.21.15.60:0/4209225998 shutdown_connections 2024-02-26T04:55:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.249+0000 7f3c240ef700 1 --2- 172.21.15.60:0/4209225998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c1c106fb0 0x7f3c1c10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.249+0000 7f3c240ef700 1 --2- 172.21.15.60:0/4209225998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c1c06a5d0 0x7f3c1c06a9b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.249+0000 7f3c240ef700 1 --2- 172.21.15.60:0/4209225998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c1c06af80 0x7f3c1c1068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.249+0000 7f3c240ef700 1 -- 172.21.15.60:0/4209225998 >> 172.21.15.60:0/4209225998 conn(0x7f3c1c076bc0 msgr2=0x7f3c1c076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.249+0000 7f3c240ef700 1 -- 172.21.15.60:0/4209225998 shutdown_connections 2024-02-26T04:55:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.249+0000 7f3c240ef700 1 -- 172.21.15.60:0/4209225998 wait complete. 2024-02-26T04:55:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c240ef700 1 Processor -- start 2024-02-26T04:55:37.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c240ef700 1 -- start start 2024-02-26T04:55:37.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c240ef700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c1c06a5d0 0x7f3c1c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:37.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c240ef700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c1c06af80 0x7f3c1c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:37.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c240ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c1c106fb0 0x7f3c1c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:37.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c240ef700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c19f310 con 0x7f3c1c06af80 2024-02-26T04:55:37.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c240ef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c19f480 con 0x7f3c1c106fb0 2024-02-26T04:55:37.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c240ef700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c19f640 con 0x7f3c1c06a5d0 2024-02-26T04:55:37.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c2168a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c1c06af80 0x7f3c1c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:37.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c21e8b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c1c06a5d0 0x7f3c1c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:37.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c21e8b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c1c06a5d0 0x7f3c1c19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.60:36904/0 (socket says 172.21.15.60:36904) 2024-02-26T04:55:37.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c21e8b700 1 -- 172.21.15.60:0/164111676 learned_addr learned my addr 172.21.15.60:0/164111676 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c2268c700 1 --2- 172.21.15.60:0/164111676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c1c106fb0 0x7f3c1c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c2168a700 1 -- 172.21.15.60:0/164111676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c1c06a5d0 msgr2=0x7f3c1c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c2168a700 1 --2- 172.21.15.60:0/164111676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c1c06a5d0 0x7f3c1c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c2168a700 1 -- 172.21.15.60:0/164111676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c1c106fb0 msgr2=0x7f3c1c19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c2168a700 1 --2- 172.21.15.60:0/164111676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c1c106fb0 0x7f3c1c19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.250+0000 7f3c2168a700 1 -- 172.21.15.60:0/164111676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0c009d00 con 0x7f3c1c06af80 2024-02-26T04:55:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.251+0000 7f3c2168a700 1 --2- 172.21.15.60:0/164111676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c1c06af80 0x7f3c1c19a970 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c06a160 tx=0x7f3c1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.251+0000 7f3c1affd700 1 -- 172.21.15.60:0/164111676 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1000f890 con 0x7f3c1c06af80 2024-02-26T04:55:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.251+0000 7f3c1affd700 1 -- 172.21.15.60:0/164111676 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c10010460 con 0x7f3c1c06af80 2024-02-26T04:55:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.251+0000 7f3c240ef700 1 -- 172.21.15.60:0/164111676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c1c19f920 con 0x7f3c1c06af80 2024-02-26T04:55:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.251+0000 7f3c1affd700 1 -- 172.21.15.60:0/164111676 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1000e610 con 0x7f3c1c06af80 2024-02-26T04:55:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.251+0000 7f3c240ef700 1 -- 172.21.15.60:0/164111676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1c1a7180 con 0x7f3c1c06af80 2024-02-26T04:55:37.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.252+0000 7f3c240ef700 1 -- 172.21.15.60:0/164111676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1c068930 con 0x7f3c1c06af80 2024-02-26T04:55:37.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.253+0000 7f3c1affd700 1 -- 172.21.15.60:0/164111676 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3c10010920 con 0x7f3c1c06af80 2024-02-26T04:55:37.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.253+0000 7f3c1affd700 1 --2- 172.21.15.60:0/164111676 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3c040a21d0 0x7f3c040a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:37.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.254+0000 7f3c21e8b700 1 --2- 172.21.15.60:0/164111676 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3c040a21d0 0x7f3c040a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:37.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.254+0000 7f3c1affd700 1 -- 172.21.15.60:0/164111676 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f3c100cdaa0 con 0x7f3c1c06af80 2024-02-26T04:55:37.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.254+0000 7f3c21e8b700 1 --2- 172.21.15.60:0/164111676 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3c040a21d0 0x7f3c040a4690 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009fd0 tx=0x7f3c0c006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:37.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.258+0000 7f3c1affd700 1 -- 172.21.15.60:0/164111676 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3c100967d0 con 0x7f3c1c06af80 2024-02-26T04:55:37.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.501+0000 7f3c240ef700 1 -- 172.21.15.60:0/164111676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3c1c111d10 con 0x7f3c1c06af80 2024-02-26T04:55:37.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:37 smithi060 ceph-mon[32382]: pgmap v298: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-02-26T04:55:37.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.502+0000 7f3c1affd700 1 -- 172.21.15.60:0/164111676 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3c10099e90 con 0x7f3c1c06af80 2024-02-26T04:55:37.505 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-26T04:55:37.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.504+0000 7f3c240ef700 1 -- 172.21.15.60:0/164111676 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3c040a21d0 msgr2=0x7f3c040a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:37.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.504+0000 7f3c240ef700 1 --2- 172.21.15.60:0/164111676 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3c040a21d0 0x7f3c040a4690 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009fd0 tx=0x7f3c0c006210 comp rx=0 tx=0).stop 2024-02-26T04:55:37.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.504+0000 7f3c240ef700 1 -- 172.21.15.60:0/164111676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c1c06af80 msgr2=0x7f3c1c19a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:37.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.504+0000 7f3c240ef700 1 --2- 172.21.15.60:0/164111676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c1c06af80 0x7f3c1c19a970 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c06a160 tx=0x7f3c1000dc30 comp rx=0 tx=0).stop 2024-02-26T04:55:37.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.504+0000 7f3c240ef700 1 -- 172.21.15.60:0/164111676 shutdown_connections 2024-02-26T04:55:37.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.504+0000 7f3c240ef700 1 --2- 172.21.15.60:0/164111676 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3c040a21d0 0x7f3c040a4690 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:37.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.504+0000 7f3c240ef700 1 --2- 172.21.15.60:0/164111676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c1c106fb0 0x7f3c1c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:37.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.504+0000 7f3c240ef700 1 --2- 172.21.15.60:0/164111676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c1c06a5d0 0x7f3c1c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:37.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.504+0000 7f3c240ef700 1 --2- 172.21.15.60:0/164111676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c1c06af80 0x7f3c1c19a970 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:37.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.504+0000 7f3c240ef700 1 -- 172.21.15.60:0/164111676 >> 172.21.15.60:0/164111676 conn(0x7f3c1c076bc0 msgr2=0x7f3c1c10cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:37.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.504+0000 7f3c240ef700 1 -- 172.21.15.60:0/164111676 shutdown_connections 2024-02-26T04:55:37.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:37.504+0000 7f3c240ef700 1 -- 172.21.15.60:0/164111676 wait complete. 2024-02-26T04:55:37.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:37 smithi191 ceph-mon[36352]: pgmap v298: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-02-26T04:55:37.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:37 smithi156 ceph-mon[35805]: pgmap v298: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-02-26T04:55:38.191 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-26T04:55:38.191 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T04:55:38.191 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-02-26T04:55:38.191 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-02-26T04:55:38.191 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-02-26T04:55:38.191 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-26T04:55:38.191 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-26T04:55:38.191 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T04:55:38.191 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-02-26T04:55:38.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:38.192 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-02-26T04:55:38.209 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T04:55:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T04:55:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T04:55:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-02-26T04:55:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-02-26T04:55:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-02-26T04:55:38.209 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T04:55:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-02-26T04:55:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-02-26T04:55:38.210 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T04:55:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-02-26T04:55:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-02-26T04:55:38.210 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T04:55:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-02-26T04:55:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-02-26T04:55:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-26T04:55:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42128sec preferred_lft 42128sec 2024-02-26T04:55:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-02-26T04:55:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-02-26T04:55:38.210 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T04:55:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-02-26T04:55:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-02-26T04:55:38.211 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-26T04:55:38.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:38.211 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T04:55:38.211 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T04:55:38.211 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add name ceph-brx type bridge 2024-02-26T04:55:38.211 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr flush dev ceph-brx 2024-02-26T04:55:38.211 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set ceph-brx up 2024-02-26T04:55:38.211 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-26T04:55:38.212 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T04:55:38.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:38.397 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T04:55:38.424 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-02-26T04:55:38.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:38.425 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-02-26T04:55:38.492 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-02-26T04:55:38.492 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T04:55:38.492 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-26T04:55:38.492 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-26T04:55:38.492 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T04:55:38.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:38.493 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T04:55:38.493 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T04:55:38.493 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-26T04:55:38.493 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-26T04:55:38.493 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-26T04:55:38.493 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T04:55:38.671 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:38 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/164111676' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T04:55:38.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:38 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/164111676' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T04:55:38.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:38.786 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-02-26T04:55:38.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:38.801 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2024-02-26T04:55:38.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:38.857 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T04:55:38.858 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T04:55:38.858 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-26T04:55:38.858 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-02-26T04:55:38.858 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T04:55:38.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:38 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/164111676' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T04:55:38.941 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-02-26T04:55:38.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:38.942 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T04:55:38.942 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T04:55:38.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-02-26T04:55:38.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-26T04:55:38.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-02-26T04:55:38.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-02-26T04:55:38.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-02-26T04:55:38.942 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T04:55:39.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:39.080 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T04:55:39.080 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T04:55:39.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-02-26T04:55:39.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.0 master ceph-brx 2024-02-26T04:55:39.081 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T04:55:39.123 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-02-26T04:55:39.123 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T04:55:39.123 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-02-26T04:55:39.177 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-02-26T04:55:39.178 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T04:55:39.178 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-02-26T04:55:39.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-02-26T04:55:39.299 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-02-26T04:55:39.355 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-02-26T04:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-02-26T04:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-02-26T04:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-02-26T04:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-02-26T04:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-02-26T04:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-02-26T04:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-02-26T04:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-02-26T04:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-02-26T04:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-02-26T04:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2024-02-26T04:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-02-26T04:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-02-26T04:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-02-26T04:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-02-26T04:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-02-26T04:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-26T04:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-02-26T04:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2024-02-26T04:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-02-26T04:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-26T04:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2024-02-26T04:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-02-26T04:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-02-26T04:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay 2024-02-26T04:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/e0efddf9945db9853e1e89331c92bcc7b1af46ba3c6ab3bd8bfb4dea07b7bc08/merged 2024-02-26T04:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/1b32acda21e63e0da8ff56b38845d66363627f01d233b2a8926714597be33dad/merged 2024-02-26T04:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/0 2024-02-26T04:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-02-26T04:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/07d1a8295be63596bd290754cf7cf6f5c51b9f883f113ae8ce12a9692ed2dc01/merged 2024-02-26T04:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/78661de1f1b0be0ffeae9f16c104d861eaa9b2db36a8e9d819775394452a8b4e/merged 2024-02-26T04:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/3e5689cf9cdb1c223e67676e4b8e4d14744af445524acb58810e5801b76d70d9/merged 2024-02-26T04:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/8b228a9b521da68ed4289c34f0f584046abf27498716fe88923a1ff8cd4970d7/merged 2024-02-26T04:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/ba06d146553d586ca82df119d859da06e1cdfbf75de54a584c9b4ff02d42c505/merged 2024-02-26T04:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/3b7c0be1a7f59e92bfe042bf4151a9906ef843a0bdbb9ceb34247c19faaec839/merged 2024-02-26T04:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/3a1a7a493ff22700afea15df54347105c37161c028883f9e4f57ef155824c100/merged 2024-02-26T04:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/8ae4abe93ca15a974186b285551c10d9733788019c7fa42dd9c678bc146d6502/merged 2024-02-26T04:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-02-26T04:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-26T04:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-26T04:55:39.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:39.359 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-02-26T04:55:39.410 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-26T04:55:39.410 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-02-26T04:55:39.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-02-26T04:55:39.480 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-02-26T04:55:39.535 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-02-26T04:55:39.535 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-02-26T04:55:39.535 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-02-26T04:55:39.536 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-02-26T04:55:39.536 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-02-26T04:55:39.536 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-02-26T04:55:39.536 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-02-26T04:55:39.536 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-02-26T04:55:39.536 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-02-26T04:55:39.536 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-02-26T04:55:39.536 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-02-26T04:55:39.536 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2024-02-26T04:55:39.536 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-02-26T04:55:39.536 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-02-26T04:55:39.536 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-02-26T04:55:39.536 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/e0efddf9945db9853e1e89331c92bcc7b1af46ba3c6ab3bd8bfb4dea07b7bc08/merged 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/1b32acda21e63e0da8ff56b38845d66363627f01d233b2a8926714597be33dad/merged 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/0 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-02-26T04:55:39.537 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/07d1a8295be63596bd290754cf7cf6f5c51b9f883f113ae8ce12a9692ed2dc01/merged 2024-02-26T04:55:39.538 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/78661de1f1b0be0ffeae9f16c104d861eaa9b2db36a8e9d819775394452a8b4e/merged 2024-02-26T04:55:39.538 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/3e5689cf9cdb1c223e67676e4b8e4d14744af445524acb58810e5801b76d70d9/merged 2024-02-26T04:55:39.538 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/8b228a9b521da68ed4289c34f0f584046abf27498716fe88923a1ff8cd4970d7/merged 2024-02-26T04:55:39.538 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/ba06d146553d586ca82df119d859da06e1cdfbf75de54a584c9b4ff02d42c505/merged 2024-02-26T04:55:39.538 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/3b7c0be1a7f59e92bfe042bf4151a9906ef843a0bdbb9ceb34247c19faaec839/merged 2024-02-26T04:55:39.538 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/3a1a7a493ff22700afea15df54347105c37161c028883f9e4f57ef155824c100/merged 2024-02-26T04:55:39.538 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/containers/storage/overlay/8ae4abe93ca15a974186b285551c10d9733788019c7fa42dd9c678bc146d6502/merged 2024-02-26T04:55:39.538 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-02-26T04:55:39.538 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-26T04:55:39.538 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-26T04:55:39.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:39.539 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-02-26T04:55:39.566 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:2024-02-26T04:55:39.563+0000 7f88a944c480 -1 init, newargv = 0x564aeae5d090 newargc=15 2024-02-26T04:55:39.566 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:ceph-fuse[73742]: starting ceph client 2024-02-26T04:55:39.571 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:ceph-fuse[73742]: starting fuse 2024-02-26T04:55:39.580 INFO:teuthology.orchestra.run.smithi060.stdout:119 2024-02-26T04:55:39.580 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-02-26T04:55:39.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-02-26T04:55:39.581 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-02-26T04:55:39.581 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-02-26T04:55:39.581 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-02-26T04:55:39.581 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-02-26T04:55:39.581 DEBUG:teuthology.orchestra.run.smithi060:> 2024-02-26T04:55:39.581 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-02-26T04:55:39.581 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-02-26T04:55:39.581 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-02-26T04:55:39.581 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-02-26T04:55:39.581 DEBUG:teuthology.orchestra.run.smithi060:> 2024-02-26T04:55:39.581 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-02-26T04:55:39.581 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-02-26T04:55:39.581 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-02-26T04:55:39.582 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-02-26T04:55:39.582 DEBUG:teuthology.orchestra.run.smithi060:> 2024-02-26T04:55:39.582 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-02-26T04:55:39.582 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-26T04:55:39.582 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-26T04:55:39.582 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-26T04:55:39.582 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-02-26T04:55:39.582 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-02-26T04:55:39.582 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-02-26T04:55:39.582 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-26T04:55:39.582 DEBUG:teuthology.orchestra.run.smithi060:> 2024-02-26T04:55:39.582 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.admin")) 2024-02-26T04:55:39.582 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-02-26T04:55:39.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:39 smithi060 ceph-mon[32382]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-02-26T04:55:39.703 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.admin.73742.asok 2024-02-26T04:55:39.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-26T04:55:39.713 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73742.asok 2024-02-26T04:55:39.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:39.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73742.asok status 2024-02-26T04:55:39.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:39 smithi191 ceph-mon[36352]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-02-26T04:55:39.850 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:55:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-02-26T04:55:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "25296a8f176256aea5ed8b5850b25167dce0ae33", 2024-02-26T04:55:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1514-g25296a8f (25296a8f176256aea5ed8b5850b25167dce0ae33) squid (dev)", 2024-02-26T04:55:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "admin", 2024-02-26T04:55:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-02-26T04:55:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-02-26T04:55:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "73742", 2024-02-26T04:55:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-02-26T04:55:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:55:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-02-26T04:55:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-02-26T04:55:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 14760, 2024-02-26T04:55:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-02-26T04:55:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-02-26T04:55:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-02-26T04:55:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 14760 2024-02-26T04:55:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:55:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-02-26T04:55:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-02-26T04:55:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-02-26T04:55:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2936388754 2024-02-26T04:55:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:55:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:55:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-02-26T04:55:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-02-26T04:55:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-02-26T04:55:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2936388754 2024-02-26T04:55:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:55:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.14760 192.168.0.1:0/2936388754", 2024-02-26T04:55:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/2936388754", 2024-02-26T04:55:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-02-26T04:55:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 16, 2024-02-26T04:55:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 89, 2024-02-26T04:55:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-02-26T04:55:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-02-26T04:55:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-02-26T04:55:39.853 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:55:39.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:39.859 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-02-26T04:55:39.859 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-02-26T04:55:39.873 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-02-26T04:55:39.874 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-02-26T04:55:39.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:39.874 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-02-26T04:55:39.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:39 smithi156 ceph-mon[35805]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-02-26T04:55:39.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:39.942 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-02-26T04:55:40.024 INFO:teuthology.orchestra.run.smithi060.stdout:# file: . 2024-02-26T04:55:40.024 INFO:teuthology.orchestra.run.smithi060.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-02-26T04:55:40.024 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:40.025 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:55:40.025 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T04:55:40.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:55:40.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.221+0000 7f3f68e60640 1 -- 172.21.15.60:0/278800531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f64101cd0 msgr2=0x7f3f64108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:40.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.221+0000 7f3f68e60640 1 --2- 172.21.15.60:0/278800531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f64101cd0 0x7f3f64108560 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009ca0 tx=0x7f3f5c02c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.221+0000 7f3f68e60640 1 -- 172.21.15.60:0/278800531 shutdown_connections 2024-02-26T04:55:40.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.221+0000 7f3f68e60640 1 --2- 172.21.15.60:0/278800531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f64101cd0 0x7f3f64108560 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.221+0000 7f3f68e60640 1 --2- 172.21.15.60:0/278800531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f64101310 0x7f3f64101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.221+0000 7f3f68e60640 1 --2- 172.21.15.60:0/278800531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f64100120 0x7f3f64100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.221+0000 7f3f68e60640 1 -- 172.21.15.60:0/278800531 >> 172.21.15.60:0/278800531 conn(0x7f3f640fb6b0 msgr2=0x7f3f640fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:40.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.221+0000 7f3f68e60640 1 -- 172.21.15.60:0/278800531 shutdown_connections 2024-02-26T04:55:40.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.221+0000 7f3f68e60640 1 -- 172.21.15.60:0/278800531 wait complete. 2024-02-26T04:55:40.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.222+0000 7f3f68e60640 1 Processor -- start 2024-02-26T04:55:40.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.222+0000 7f3f68e60640 1 -- start start 2024-02-26T04:55:40.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.222+0000 7f3f68e60640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f64100120 0x7f3f641971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:40.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.222+0000 7f3f68e60640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f64101310 0x7f3f64197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:40.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.222+0000 7f3f68e60640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f64101cd0 0x7f3f6419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:40.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.222+0000 7f3f68e60640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6419ecc0 con 0x7f3f64100120 2024-02-26T04:55:40.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.222+0000 7f3f68e60640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6419ee30 con 0x7f3f64101cd0 2024-02-26T04:55:40.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.222+0000 7f3f68e60640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6419eff0 con 0x7f3f64101310 2024-02-26T04:55:40.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.222+0000 7f3f6259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f64100120 0x7f3f641971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:40.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.222+0000 7f3f6259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f64100120 0x7f3f641971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46310/0 (socket says 172.21.15.60:46310) 2024-02-26T04:55:40.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.222+0000 7f3f6259c640 1 -- 172.21.15.60:0/496443509 learned_addr learned my addr 172.21.15.60:0/496443509 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.222+0000 7f3f5bfff640 1 --2- 172.21.15.60:0/496443509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f64101310 0x7f3f64197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.222+0000 7f3f62d9d640 1 --2- 172.21.15.60:0/496443509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f64101cd0 0x7f3f6419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.223+0000 7f3f6259c640 1 -- 172.21.15.60:0/496443509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f64101310 msgr2=0x7f3f64197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.223+0000 7f3f6259c640 1 --2- 172.21.15.60:0/496443509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f64101310 0x7f3f64197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.223+0000 7f3f6259c640 1 -- 172.21.15.60:0/496443509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f64101cd0 msgr2=0x7f3f6419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.223+0000 7f3f6259c640 1 --2- 172.21.15.60:0/496443509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f64101cd0 0x7f3f6419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.223+0000 7f3f6259c640 1 -- 172.21.15.60:0/496443509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c009900 con 0x7f3f64100120 2024-02-26T04:55:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.223+0000 7f3f6259c640 1 --2- 172.21.15.60:0/496443509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f64100120 0x7f3f641971d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400d950 tx=0x7f3f5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.223+0000 7f3f5b7fe640 1 -- 172.21.15.60:0/496443509 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54009c30 con 0x7f3f64100120 2024-02-26T04:55:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.223+0000 7f3f68e60640 1 -- 172.21.15.60:0/496443509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6419f2d0 con 0x7f3f64100120 2024-02-26T04:55:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.223+0000 7f3f5b7fe640 1 -- 172.21.15.60:0/496443509 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f54009d90 con 0x7f3f64100120 2024-02-26T04:55:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.223+0000 7f3f5b7fe640 1 -- 172.21.15.60:0/496443509 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5400f720 con 0x7f3f64100120 2024-02-26T04:55:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.223+0000 7f3f68e60640 1 -- 172.21.15.60:0/496443509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6419f7e0 con 0x7f3f64100120 2024-02-26T04:55:40.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.224+0000 7f3f68e60640 1 -- 172.21.15.60:0/496443509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6410bac0 con 0x7f3f64100120 2024-02-26T04:55:40.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.225+0000 7f3f5b7fe640 1 -- 172.21.15.60:0/496443509 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3f54010a60 con 0x7f3f64100120 2024-02-26T04:55:40.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.226+0000 7f3f5b7fe640 1 --2- 172.21.15.60:0/496443509 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3f500a20c0 0x7f3f500a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:40.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.226+0000 7f3f5b7fe640 1 -- 172.21.15.60:0/496443509 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f3f540cdbd0 con 0x7f3f64100120 2024-02-26T04:55:40.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.226+0000 7f3f5bfff640 1 --2- 172.21.15.60:0/496443509 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3f500a20c0 0x7f3f500a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:40.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.227+0000 7f3f5bfff640 1 --2- 172.21.15.60:0/496443509 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3f500a20c0 0x7f3f500a4580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3f641986e0 tx=0x7f3f4c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.230+0000 7f3f5b7fe640 1 -- 172.21.15.60:0/496443509 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3f54096960 con 0x7f3f64100120 2024-02-26T04:55:40.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.414+0000 7f3f68e60640 1 -- 172.21.15.60:0/496443509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f6410bc50 con 0x7f3f64100120 2024-02-26T04:55:40.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.416+0000 7f3f5b7fe640 1 -- 172.21.15.60:0/496443509 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24126 (secure 0 0 0) 0x7f3f54017090 con 0x7f3f64100120 2024-02-26T04:55:40.418 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:40.418 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:55:32.581481+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:55:40.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.417+0000 7f3f68e60640 1 -- 172.21.15.60:0/496443509 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3f500a20c0 msgr2=0x7f3f500a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:40.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.417+0000 7f3f68e60640 1 --2- 172.21.15.60:0/496443509 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3f500a20c0 0x7f3f500a4580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3f641986e0 tx=0x7f3f4c00a2d0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.417+0000 7f3f68e60640 1 -- 172.21.15.60:0/496443509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f64100120 msgr2=0x7f3f641971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:40.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.417+0000 7f3f68e60640 1 --2- 172.21.15.60:0/496443509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f64100120 0x7f3f641971d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400d950 tx=0x7f3f5400de10 comp rx=0 tx=0).stop 2024-02-26T04:55:40.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.418+0000 7f3f68e60640 1 -- 172.21.15.60:0/496443509 shutdown_connections 2024-02-26T04:55:40.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.418+0000 7f3f68e60640 1 --2- 172.21.15.60:0/496443509 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3f500a20c0 0x7f3f500a4580 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.418+0000 7f3f68e60640 1 --2- 172.21.15.60:0/496443509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f64101cd0 0x7f3f6419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.418+0000 7f3f68e60640 1 --2- 172.21.15.60:0/496443509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f64101310 0x7f3f64197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.418+0000 7f3f68e60640 1 --2- 172.21.15.60:0/496443509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f64100120 0x7f3f641971d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.418+0000 7f3f68e60640 1 -- 172.21.15.60:0/496443509 >> 172.21.15.60:0/496443509 conn(0x7f3f640fb6b0 msgr2=0x7f3f640697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:40.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.418+0000 7f3f68e60640 1 -- 172.21.15.60:0/496443509 shutdown_connections 2024-02-26T04:55:40.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.418+0000 7f3f68e60640 1 -- 172.21.15.60:0/496443509 wait complete. 2024-02-26T04:55:40.431 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T04:50:44.572921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T04:55:25.530720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T04:55:26.741604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T04:55:30.453569+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T04:55:40.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T04:55:40.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.538+0000 7f902571e640 1 -- 172.21.15.60:0/1447310315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90201045f0 msgr2=0x7f90201069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:40.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.538+0000 7f902571e640 1 --2- 172.21.15.60:0/1447310315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90201045f0 0x7f90201069e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9008009900 tx=0x7f900802bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:40.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 -- 172.21.15.60:0/1447310315 shutdown_connections 2024-02-26T04:55:40.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 --2- 172.21.15.60:0/1447310315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9020106f20 0x7f9020109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 --2- 172.21.15.60:0/1447310315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90201045f0 0x7f90201069e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 --2- 172.21.15.60:0/1447310315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9020101cc0 0x7f90201040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 -- 172.21.15.60:0/1447310315 >> 172.21.15.60:0/1447310315 conn(0x7f90200fb610 msgr2=0x7f90200fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:40.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 -- 172.21.15.60:0/1447310315 shutdown_connections 2024-02-26T04:55:40.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 -- 172.21.15.60:0/1447310315 wait complete. 2024-02-26T04:55:40.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 Processor -- start 2024-02-26T04:55:40.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 -- start start 2024-02-26T04:55:40.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9020101cc0 0x7f9020101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:40.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90201045f0 0x7f90200ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:40.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9020106f20 0x7f90201001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:40.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9020100910 con 0x7f9020101cc0 2024-02-26T04:55:40.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9020100a80 con 0x7f9020106f20 2024-02-26T04:55:40.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.539+0000 7f902571e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90201a1290 con 0x7f90201045f0 2024-02-26T04:55:40.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f901effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9020101cc0 0x7f9020101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f901effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9020101cc0 0x7f9020101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46320/0 (socket says 172.21.15.60:46320) 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f901effd640 1 -- 172.21.15.60:0/2370270956 learned_addr learned my addr 172.21.15.60:0/2370270956 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f901f7fe640 1 --2- 172.21.15.60:0/2370270956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9020106f20 0x7f90201001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f90167fc640 1 --2- 172.21.15.60:0/2370270956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90201045f0 0x7f90200ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f901effd640 1 -- 172.21.15.60:0/2370270956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90201045f0 msgr2=0x7f90200ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f901effd640 1 --2- 172.21.15.60:0/2370270956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90201045f0 0x7f90200ffc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f901effd640 1 -- 172.21.15.60:0/2370270956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9020106f20 msgr2=0x7f90201001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f901effd640 1 --2- 172.21.15.60:0/2370270956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9020106f20 0x7f90201001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f901effd640 1 -- 172.21.15.60:0/2370270956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9008009560 con 0x7f9020101cc0 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f901effd640 1 --2- 172.21.15.60:0/2370270956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9020101cc0 0x7f9020101630 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f901000c950 tx=0x7f901000ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f901cff9640 1 -- 172.21.15.60:0/2370270956 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010007c70 con 0x7f9020101cc0 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f902571e640 1 -- 172.21.15.60:0/2370270956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90201a1540 con 0x7f9020101cc0 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f901cff9640 1 -- 172.21.15.60:0/2370270956 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9010007dd0 con 0x7f9020101cc0 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f901cff9640 1 -- 172.21.15.60:0/2370270956 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90100188c0 con 0x7f9020101cc0 2024-02-26T04:55:40.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.540+0000 7f902571e640 1 -- 172.21.15.60:0/2370270956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90201a1aa0 con 0x7f9020101cc0 2024-02-26T04:55:40.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.541+0000 7f902571e640 1 -- 172.21.15.60:0/2370270956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90200fd100 con 0x7f9020101cc0 2024-02-26T04:55:40.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.543+0000 7f901cff9640 1 -- 172.21.15.60:0/2370270956 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9010018a20 con 0x7f9020101cc0 2024-02-26T04:55:40.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.544+0000 7f901cff9640 1 --2- 172.21.15.60:0/2370270956 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f900c0a20c0 0x7f900c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:40.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.544+0000 7f90167fc640 1 --2- 172.21.15.60:0/2370270956 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f900c0a20c0 0x7f900c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:40.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.544+0000 7f901cff9640 1 -- 172.21.15.60:0/2370270956 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f90100ce060 con 0x7f9020101cc0 2024-02-26T04:55:40.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.544+0000 7f90167fc640 1 --2- 172.21.15.60:0/2370270956 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f900c0a20c0 0x7f900c0a4580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9008036040 tx=0x7f9008005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.552+0000 7f901cff9640 1 -- 172.21.15.60:0/2370270956 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9010096df0 con 0x7f9020101cc0 2024-02-26T04:55:40.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.735+0000 7f902571e640 1 -- 172.21.15.60:0/2370270956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f902002cdd0 con 0x7f9020101cc0 2024-02-26T04:55:40.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.735+0000 7f901cff9640 1 -- 172.21.15.60:0/2370270956 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f901009a4b0 con 0x7f9020101cc0 2024-02-26T04:55:40.738 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T04:55:40.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.737+0000 7f902571e640 1 -- 172.21.15.60:0/2370270956 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f900c0a20c0 msgr2=0x7f900c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:40.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.737+0000 7f902571e640 1 --2- 172.21.15.60:0/2370270956 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f900c0a20c0 0x7f900c0a4580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9008036040 tx=0x7f9008005fd0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.737+0000 7f902571e640 1 -- 172.21.15.60:0/2370270956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9020101cc0 msgr2=0x7f9020101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:40.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.737+0000 7f902571e640 1 --2- 172.21.15.60:0/2370270956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9020101cc0 0x7f9020101630 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f901000c950 tx=0x7f901000ce10 comp rx=0 tx=0).stop 2024-02-26T04:55:40.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.737+0000 7f902571e640 1 -- 172.21.15.60:0/2370270956 shutdown_connections 2024-02-26T04:55:40.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.737+0000 7f902571e640 1 --2- 172.21.15.60:0/2370270956 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f900c0a20c0 0x7f900c0a4580 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.737+0000 7f902571e640 1 --2- 172.21.15.60:0/2370270956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9020106f20 0x7f90201001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.737+0000 7f902571e640 1 --2- 172.21.15.60:0/2370270956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90201045f0 0x7f90200ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.737+0000 7f902571e640 1 --2- 172.21.15.60:0/2370270956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9020101cc0 0x7f9020101630 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.737+0000 7f902571e640 1 -- 172.21.15.60:0/2370270956 >> 172.21.15.60:0/2370270956 conn(0x7f90200fb610 msgr2=0x7f9020102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:40.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.737+0000 7f902571e640 1 -- 172.21.15.60:0/2370270956 shutdown_connections 2024-02-26T04:55:40.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.737+0000 7f902571e640 1 -- 172.21.15.60:0/2370270956 wait complete. 2024-02-26T04:55:40.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:40 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/496443509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:40.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T04:55:40.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:40 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/496443509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:40.829 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:40 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/496443509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:40.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.857+0000 7f42f77aa640 1 -- 172.21.15.60:0/1721924534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f0106f40 msgr2=0x7f42f0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:40.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.857+0000 7f42f77aa640 1 --2- 172.21.15.60:0/1721924534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f0106f40 0x7f42f0109330 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009ca0 tx=0x7f42e802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.857+0000 7f42f77aa640 1 -- 172.21.15.60:0/1721924534 shutdown_connections 2024-02-26T04:55:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.857+0000 7f42f77aa640 1 --2- 172.21.15.60:0/1721924534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f0106f40 0x7f42f0109330 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.857+0000 7f42f77aa640 1 --2- 172.21.15.60:0/1721924534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42f0104610 0x7f42f0106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.857+0000 7f42f77aa640 1 --2- 172.21.15.60:0/1721924534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42f0101ce0 0x7f42f01040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.857+0000 7f42f77aa640 1 -- 172.21.15.60:0/1721924534 >> 172.21.15.60:0/1721924534 conn(0x7f42f00fb630 msgr2=0x7f42f00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.857+0000 7f42f77aa640 1 -- 172.21.15.60:0/1721924534 shutdown_connections 2024-02-26T04:55:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.857+0000 7f42f77aa640 1 -- 172.21.15.60:0/1721924534 wait complete. 2024-02-26T04:55:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.857+0000 7f42f77aa640 1 Processor -- start 2024-02-26T04:55:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.857+0000 7f42f77aa640 1 -- start start 2024-02-26T04:55:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f77aa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42f0101ce0 0x7f42f0197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f77aa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f0104610 0x7f42f01976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f77aa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42f0106f40 0x7f42f019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f77aa640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f019ec70 con 0x7f42f0104610 2024-02-26T04:55:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f77aa640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f019ede0 con 0x7f42f0106f40 2024-02-26T04:55:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f77aa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f019efa0 con 0x7f42f0101ce0 2024-02-26T04:55:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f4d45640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f0104610 0x7f42f01976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f4d45640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f0104610 0x7f42f01976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46346/0 (socket says 172.21.15.60:46346) 2024-02-26T04:55:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f4d45640 1 -- 172.21.15.60:0/2092771383 learned_addr learned my addr 172.21.15.60:0/2092771383 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f5d47640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42f0106f40 0x7f42f019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f5546640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42f0101ce0 0x7f42f0197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f4d45640 1 -- 172.21.15.60:0/2092771383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42f0101ce0 msgr2=0x7f42f0197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f4d45640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42f0101ce0 0x7f42f0197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f4d45640 1 -- 172.21.15.60:0/2092771383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42f0106f40 msgr2=0x7f42f019e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f4d45640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42f0106f40 0x7f42f019e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f4d45640 1 -- 172.21.15.60:0/2092771383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e8009900 con 0x7f42f0104610 2024-02-26T04:55:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f5546640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42f0101ce0 0x7f42f0197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f4d45640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f0104610 0x7f42f01976c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f42e0005fd0 tx=0x7f42e000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42e67fc640 1 -- 172.21.15.60:0/2092771383 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0009c30 con 0x7f42f0104610 2024-02-26T04:55:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42f77aa640 1 -- 172.21.15.60:0/2092771383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f019f280 con 0x7f42f0104610 2024-02-26T04:55:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42e67fc640 1 -- 172.21.15.60:0/2092771383 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e0009d90 con 0x7f42f0104610 2024-02-26T04:55:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.858+0000 7f42e67fc640 1 -- 172.21.15.60:0/2092771383 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0010460 con 0x7f42f0104610 2024-02-26T04:55:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.859+0000 7f42f77aa640 1 -- 172.21.15.60:0/2092771383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f019f790 con 0x7f42f0104610 2024-02-26T04:55:40.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.859+0000 7f42f77aa640 1 -- 172.21.15.60:0/2092771383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f00fd120 con 0x7f42f0104610 2024-02-26T04:55:40.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.861+0000 7f42e67fc640 1 -- 172.21.15.60:0/2092771383 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f42e0003f20 con 0x7f42f0104610 2024-02-26T04:55:40.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.861+0000 7f42e67fc640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f42dc0a20c0 0x7f42dc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:40.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.862+0000 7f42f5546640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f42dc0a20c0 0x7f42dc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:40.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.862+0000 7f42e67fc640 1 -- 172.21.15.60:0/2092771383 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f42e00ce2e0 con 0x7f42f0104610 2024-02-26T04:55:40.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.862+0000 7f42f5546640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f42dc0a20c0 0x7f42dc0a4580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00abe0 tx=0x7f42ec00a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:40.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:40.865+0000 7f42e67fc640 1 -- 172.21.15.60:0/2092771383 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42e0097070 con 0x7f42f0104610 2024-02-26T04:55:41.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.046+0000 7f42f77aa640 1 -- 172.21.15.60:0/2092771383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42f010cd50 con 0x7f42f0104610 2024-02-26T04:55:41.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.047+0000 7f42e67fc640 1 -- 172.21.15.60:0/2092771383 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f42e00170a0 con 0x7f42f0104610 2024-02-26T04:55:41.049 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:55:41.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.049+0000 7f42f77aa640 1 -- 172.21.15.60:0/2092771383 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f42dc0a20c0 msgr2=0x7f42dc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.049+0000 7f42f77aa640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f42dc0a20c0 0x7f42dc0a4580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00abe0 tx=0x7f42ec00a5c0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.049+0000 7f42f77aa640 1 -- 172.21.15.60:0/2092771383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f0104610 msgr2=0x7f42f01976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.049+0000 7f42f77aa640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f0104610 0x7f42f01976c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f42e0005fd0 tx=0x7f42e000dde0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.049+0000 7f42f77aa640 1 -- 172.21.15.60:0/2092771383 shutdown_connections 2024-02-26T04:55:41.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.049+0000 7f42f77aa640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f42dc0a20c0 0x7f42dc0a4580 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.049+0000 7f42f77aa640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42f0106f40 0x7f42f019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.049+0000 7f42f77aa640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f0104610 0x7f42f01976c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.049+0000 7f42f77aa640 1 --2- 172.21.15.60:0/2092771383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42f0101ce0 0x7f42f0197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.049+0000 7f42f77aa640 1 -- 172.21.15.60:0/2092771383 >> 172.21.15.60:0/2092771383 conn(0x7f42f00fb630 msgr2=0x7f42f0102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:41.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.050+0000 7f42f77aa640 1 -- 172.21.15.60:0/2092771383 shutdown_connections 2024-02-26T04:55:41.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.050+0000 7f42f77aa640 1 -- 172.21.15.60:0/2092771383 wait complete. 2024-02-26T04:55:41.062 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T04:55:41.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.168+0000 7ff30e3f7640 1 -- 172.21.15.60:0/525733199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3080ffa70 msgr2=0x7ff3080ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.168+0000 7ff30e3f7640 1 --2- 172.21.15.60:0/525733199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3080ffa70 0x7ff3080ffe90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009900 tx=0x7ff2f802bcd0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.168+0000 7ff30e3f7640 1 -- 172.21.15.60:0/525733199 shutdown_connections 2024-02-26T04:55:41.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.168+0000 7ff30e3f7640 1 --2- 172.21.15.60:0/525733199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff308100d70 0x7ff3081011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.168+0000 7ff30e3f7640 1 --2- 172.21.15.60:0/525733199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3081003d0 0x7ff308100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.168+0000 7ff30e3f7640 1 --2- 172.21.15.60:0/525733199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3080ffa70 0x7ff3080ffe90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.168+0000 7ff30e3f7640 1 -- 172.21.15.60:0/525733199 >> 172.21.15.60:0/525733199 conn(0x7ff3080fb610 msgr2=0x7ff3080fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:41.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.168+0000 7ff30e3f7640 1 -- 172.21.15.60:0/525733199 shutdown_connections 2024-02-26T04:55:41.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.168+0000 7ff30e3f7640 1 -- 172.21.15.60:0/525733199 wait complete. 2024-02-26T04:55:41.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.168+0000 7ff30e3f7640 1 Processor -- start 2024-02-26T04:55:41.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.168+0000 7ff30e3f7640 1 -- start start 2024-02-26T04:55:41.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.168+0000 7ff30e3f7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3080ffa70 0x7ff3081994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:41.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff30e3f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3081003d0 0x7ff3081999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:41.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff30e3f7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff308100d70 0x7ff3081a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:41.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff30e3f7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3081a0f90 con 0x7ff3081003d0 2024-02-26T04:55:41.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff30e3f7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3081a1100 con 0x7ff3080ffa70 2024-02-26T04:55:41.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff30e3f7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3081a12c0 con 0x7ff308100d70 2024-02-26T04:55:41.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff3077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3081003d0 0x7ff3081999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:41.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff3077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3081003d0 0x7ff3081999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46366/0 (socket says 172.21.15.60:46366) 2024-02-26T04:55:41.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff3077fe640 1 -- 172.21.15.60:0/995176566 learned_addr learned my addr 172.21.15.60:0/995176566 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:41.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff307fff640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3080ffa70 0x7ff3081994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:41.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff3077fe640 1 -- 172.21.15.60:0/995176566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff308100d70 msgr2=0x7ff3081a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff30c994640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff308100d70 0x7ff3081a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:41.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff3077fe640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff308100d70 0x7ff3081a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff3077fe640 1 -- 172.21.15.60:0/995176566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3080ffa70 msgr2=0x7ff3081994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff3077fe640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3080ffa70 0x7ff3081994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff3077fe640 1 -- 172.21.15.60:0/995176566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f8009560 con 0x7ff3081003d0 2024-02-26T04:55:41.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff307fff640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3080ffa70 0x7ff3081994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:41.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff3077fe640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3081003d0 0x7ff3081999e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000d770 tx=0x7ff2f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:41.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff3057fa640 1 -- 172.21.15.60:0/995176566 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0009c30 con 0x7ff3081003d0 2024-02-26T04:55:41.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff30e3f7640 1 -- 172.21.15.60:0/995176566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3081a15a0 con 0x7ff3081003d0 2024-02-26T04:55:41.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff3057fa640 1 -- 172.21.15.60:0/995176566 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2f000de80 con 0x7ff3081003d0 2024-02-26T04:55:41.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff3057fa640 1 -- 172.21.15.60:0/995176566 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0004e80 con 0x7ff3081003d0 2024-02-26T04:55:41.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.169+0000 7ff30e3f7640 1 -- 172.21.15.60:0/995176566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3081a1ab0 con 0x7ff3081003d0 2024-02-26T04:55:41.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.170+0000 7ff30e3f7640 1 -- 172.21.15.60:0/995176566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30810aa00 con 0x7ff3081003d0 2024-02-26T04:55:41.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.172+0000 7ff3057fa640 1 -- 172.21.15.60:0/995176566 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff2f0005000 con 0x7ff3081003d0 2024-02-26T04:55:41.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.173+0000 7ff3057fa640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff2f40a20c0 0x7ff2f40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:41.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.173+0000 7ff307fff640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff2f40a20c0 0x7ff2f40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:41.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.173+0000 7ff3057fa640 1 -- 172.21.15.60:0/995176566 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff2f00cddd0 con 0x7ff3081003d0 2024-02-26T04:55:41.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.173+0000 7ff307fff640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff2f40a20c0 0x7ff2f40a4580 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8006010 tx=0x7ff2f802c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:41.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.176+0000 7ff3057fa640 1 -- 172.21.15.60:0/995176566 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff2f0096b60 con 0x7ff3081003d0 2024-02-26T04:55:41.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.357+0000 7ff30e3f7640 1 -- 172.21.15.60:0/995176566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff30810ef90 con 0x7ff3081003d0 2024-02-26T04:55:41.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.357+0000 7ff3057fa640 1 -- 172.21.15.60:0/995176566 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff2f0017020 con 0x7ff3081003d0 2024-02-26T04:55:41.360 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T04:55:41.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.359+0000 7ff30e3f7640 1 -- 172.21.15.60:0/995176566 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff2f40a20c0 msgr2=0x7ff2f40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.359+0000 7ff30e3f7640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff2f40a20c0 0x7ff2f40a4580 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8006010 tx=0x7ff2f802c880 comp rx=0 tx=0).stop 2024-02-26T04:55:41.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.359+0000 7ff30e3f7640 1 -- 172.21.15.60:0/995176566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3081003d0 msgr2=0x7ff3081999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.359+0000 7ff30e3f7640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3081003d0 0x7ff3081999e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000d770 tx=0x7ff2f000dc30 comp rx=0 tx=0).stop 2024-02-26T04:55:41.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.359+0000 7ff30e3f7640 1 -- 172.21.15.60:0/995176566 shutdown_connections 2024-02-26T04:55:41.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.359+0000 7ff30e3f7640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff2f40a20c0 0x7ff2f40a4580 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.359+0000 7ff30e3f7640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff308100d70 0x7ff3081a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.359+0000 7ff30e3f7640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3081003d0 0x7ff3081999e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.359+0000 7ff30e3f7640 1 --2- 172.21.15.60:0/995176566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3080ffa70 0x7ff3081994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.359+0000 7ff30e3f7640 1 -- 172.21.15.60:0/995176566 >> 172.21.15.60:0/995176566 conn(0x7ff3080fb610 msgr2=0x7ff3080fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:41.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.359+0000 7ff30e3f7640 1 -- 172.21.15.60:0/995176566 shutdown_connections 2024-02-26T04:55:41.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.359+0000 7ff30e3f7640 1 -- 172.21.15.60:0/995176566 wait complete. 2024-02-26T04:55:41.372 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T04:55:41.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.476+0000 7efee5036640 1 -- 172.21.15.60:0/3882172096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0101c30 msgr2=0x7efee01084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.476+0000 7efee5036640 1 --2- 172.21.15.60:0/3882172096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0101c30 0x7efee01084c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efed8009ca0 tx=0x7efed802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.476+0000 7efee5036640 1 -- 172.21.15.60:0/3882172096 shutdown_connections 2024-02-26T04:55:41.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.476+0000 7efee5036640 1 --2- 172.21.15.60:0/3882172096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0101c30 0x7efee01084c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.476+0000 7efee5036640 1 --2- 172.21.15.60:0/3882172096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efee0101270 0x7efee01016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.476+0000 7efee5036640 1 --2- 172.21.15.60:0/3882172096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efee0100080 0x7efee01004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.476+0000 7efee5036640 1 -- 172.21.15.60:0/3882172096 >> 172.21.15.60:0/3882172096 conn(0x7efee00fb610 msgr2=0x7efee00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.476+0000 7efee5036640 1 -- 172.21.15.60:0/3882172096 shutdown_connections 2024-02-26T04:55:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.477+0000 7efee5036640 1 -- 172.21.15.60:0/3882172096 wait complete. 2024-02-26T04:55:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.477+0000 7efee5036640 1 Processor -- start 2024-02-26T04:55:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.477+0000 7efee5036640 1 -- start start 2024-02-26T04:55:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.477+0000 7efee5036640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efee0100080 0x7efee010a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.477+0000 7efee5036640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0101270 0x7efee0108f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.477+0000 7efee5036640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efee0101c30 0x7efee01094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.477+0000 7efee5036640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee0109c20 con 0x7efee0101270 2024-02-26T04:55:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.477+0000 7efee5036640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee0109d90 con 0x7efee0101c30 2024-02-26T04:55:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.477+0000 7efee5036640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee01a1230 con 0x7efee0100080 2024-02-26T04:55:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.477+0000 7efed7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0101270 0x7efee0108f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efed7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0101270 0x7efee0108f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46372/0 (socket says 172.21.15.60:46372) 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efed7fff640 1 -- 172.21.15.60:0/2464949377 learned_addr learned my addr 172.21.15.60:0/2464949377 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efeded9d640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efee0100080 0x7efee010a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efedf59e640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efee0101c30 0x7efee01094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efed7fff640 1 -- 172.21.15.60:0/2464949377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efee0100080 msgr2=0x7efee010a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efed7fff640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efee0100080 0x7efee010a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efed7fff640 1 -- 172.21.15.60:0/2464949377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efee0101c30 msgr2=0x7efee01094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efed7fff640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efee0101c30 0x7efee01094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efed7fff640 1 -- 172.21.15.60:0/2464949377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed8009900 con 0x7efee0101270 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efeded9d640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efee0100080 0x7efee010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efed7fff640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0101270 0x7efee0108f80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efec800d980 tx=0x7efec800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efedcd99640 1 -- 172.21.15.60:0/2464949377 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8009c30 con 0x7efee0101270 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efee5036640 1 -- 172.21.15.60:0/2464949377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee01a14e0 con 0x7efee0101270 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efedcd99640 1 -- 172.21.15.60:0/2464949377 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efec800b660 con 0x7efee0101270 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efedcd99640 1 -- 172.21.15.60:0/2464949377 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec800f6b0 con 0x7efee0101270 2024-02-26T04:55:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.478+0000 7efee5036640 1 -- 172.21.15.60:0/2464949377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee01a19f0 con 0x7efee0101270 2024-02-26T04:55:41.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.479+0000 7efee5036640 1 -- 172.21.15.60:0/2464949377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee01073a0 con 0x7efee0101270 2024-02-26T04:55:41.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.480+0000 7efedcd99640 1 -- 172.21.15.60:0/2464949377 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7efec8010460 con 0x7efee0101270 2024-02-26T04:55:41.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.481+0000 7efedcd99640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efecc0a20c0 0x7efecc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:41.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.481+0000 7efeded9d640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efecc0a20c0 0x7efecc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:41.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.481+0000 7efedcd99640 1 -- 172.21.15.60:0/2464949377 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efec80cdbd0 con 0x7efee0101270 2024-02-26T04:55:41.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.481+0000 7efeded9d640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efecc0a20c0 0x7efecc0a4580 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efed00058c0 tx=0x7efed0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.488+0000 7efedcd99640 1 -- 172.21.15.60:0/2464949377 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efec8096960 con 0x7efee0101270 2024-02-26T04:55:41.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.669+0000 7efee5036640 1 -- 172.21.15.60:0/2464949377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efee010cd20 con 0x7efee0101270 2024-02-26T04:55:41.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.670+0000 7efedcd99640 1 -- 172.21.15.60:0/2464949377 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7efec80200a0 con 0x7efee0101270 2024-02-26T04:55:41.673 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:55:41.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.672+0000 7efee5036640 1 -- 172.21.15.60:0/2464949377 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efecc0a20c0 msgr2=0x7efecc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.672+0000 7efee5036640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efecc0a20c0 0x7efecc0a4580 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efed00058c0 tx=0x7efed0005850 comp rx=0 tx=0).stop 2024-02-26T04:55:41.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.672+0000 7efee5036640 1 -- 172.21.15.60:0/2464949377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0101270 msgr2=0x7efee0108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.672+0000 7efee5036640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0101270 0x7efee0108f80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efec800d980 tx=0x7efec800de40 comp rx=0 tx=0).stop 2024-02-26T04:55:41.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.672+0000 7efee5036640 1 -- 172.21.15.60:0/2464949377 shutdown_connections 2024-02-26T04:55:41.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.672+0000 7efee5036640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efecc0a20c0 0x7efecc0a4580 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.672+0000 7efee5036640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efee0101c30 0x7efee01094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.672+0000 7efee5036640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0101270 0x7efee0108f80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.672+0000 7efee5036640 1 --2- 172.21.15.60:0/2464949377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efee0100080 0x7efee010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.672+0000 7efee5036640 1 -- 172.21.15.60:0/2464949377 >> 172.21.15.60:0/2464949377 conn(0x7efee00fb610 msgr2=0x7efee0104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:41.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.672+0000 7efee5036640 1 -- 172.21.15.60:0/2464949377 shutdown_connections 2024-02-26T04:55:41.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.672+0000 7efee5036640 1 -- 172.21.15.60:0/2464949377 wait complete. 2024-02-26T04:55:41.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:41 smithi060 ceph-mon[32382]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 126 B/s wr, 0 op/s 2024-02-26T04:55:41.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:41 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2370270956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:55:41.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:41 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2092771383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:55:41.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:41 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/995176566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:55:41.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T04:55:41.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:41 smithi191 ceph-mon[36352]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 126 B/s wr, 0 op/s 2024-02-26T04:55:41.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:41 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2370270956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:55:41.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:41 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2092771383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:55:41.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:41 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/995176566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:55:41.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.788+0000 7f9ba3d11640 1 -- 172.21.15.60:0/2827309872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b9c0ff460 msgr2=0x7f9b9c0ff880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.788+0000 7f9ba3d11640 1 --2- 172.21.15.60:0/2827309872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b9c0ff460 0x7f9b9c0ff880 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009900 tx=0x7f9b9802bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.788+0000 7f9ba3d11640 1 -- 172.21.15.60:0/2827309872 shutdown_connections 2024-02-26T04:55:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.788+0000 7f9ba3d11640 1 --2- 172.21.15.60:0/2827309872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b9c102df0 0x7f9b9c109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.788+0000 7f9ba3d11640 1 --2- 172.21.15.60:0/2827309872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b9c102430 0x7f9b9c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.788+0000 7f9ba3d11640 1 --2- 172.21.15.60:0/2827309872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b9c0ff460 0x7f9b9c0ff880 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.788+0000 7f9ba3d11640 1 -- 172.21.15.60:0/2827309872 >> 172.21.15.60:0/2827309872 conn(0x7f9b9c0fad90 msgr2=0x7f9b9c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.788+0000 7f9ba3d11640 1 -- 172.21.15.60:0/2827309872 shutdown_connections 2024-02-26T04:55:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.788+0000 7f9ba3d11640 1 -- 172.21.15.60:0/2827309872 wait complete. 2024-02-26T04:55:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba3d11640 1 Processor -- start 2024-02-26T04:55:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba3d11640 1 -- start start 2024-02-26T04:55:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba3d11640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b9c0ff460 0x7f9b9c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba3d11640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b9c102430 0x7f9b9c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba3d11640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b9c102df0 0x7f9b9c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba3d11640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c19edd0 con 0x7f9b9c102430 2024-02-26T04:55:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba3d11640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c19ef40 con 0x7f9b9c102df0 2024-02-26T04:55:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba3d11640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c19f100 con 0x7f9b9c0ff460 2024-02-26T04:55:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba12ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b9c102430 0x7f9b9c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba12ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b9c102430 0x7f9b9c197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46384/0 (socket says 172.21.15.60:46384) 2024-02-26T04:55:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba12ac640 1 -- 172.21.15.60:0/524257073 learned_addr learned my addr 172.21.15.60:0/524257073 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba1aad640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b9c0ff460 0x7f9b9c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba12ac640 1 -- 172.21.15.60:0/524257073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b9c0ff460 msgr2=0x7f9b9c1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba12ac640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b9c0ff460 0x7f9b9c1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba12ac640 1 -- 172.21.15.60:0/524257073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b9c102df0 msgr2=0x7f9b9c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba22ae640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b9c102df0 0x7f9b9c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.789+0000 7f9ba12ac640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b9c102df0 0x7f9b9c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.790+0000 7f9ba12ac640 1 -- 172.21.15.60:0/524257073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b98009560 con 0x7f9b9c102430 2024-02-26T04:55:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.790+0000 7f9ba1aad640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b9c0ff460 0x7f9b9c1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.790+0000 7f9ba22ae640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b9c102df0 0x7f9b9c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.790+0000 7f9ba12ac640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b9c102430 0x7f9b9c197820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00b4a0 tx=0x7f9b8c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.790+0000 7f9b92ffd640 1 -- 172.21.15.60:0/524257073 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c010610 con 0x7f9b9c102430 2024-02-26T04:55:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.790+0000 7f9b92ffd640 1 -- 172.21.15.60:0/524257073 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b8c010c50 con 0x7f9b9c102430 2024-02-26T04:55:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.790+0000 7f9ba3d11640 1 -- 172.21.15.60:0/524257073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c19f3e0 con 0x7f9b9c102430 2024-02-26T04:55:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.790+0000 7f9b92ffd640 1 -- 172.21.15.60:0/524257073 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c017e20 con 0x7f9b9c102430 2024-02-26T04:55:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.790+0000 7f9ba3d11640 1 -- 172.21.15.60:0/524257073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c19f970 con 0x7f9b9c102430 2024-02-26T04:55:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.791+0000 7f9ba3d11640 1 -- 172.21.15.60:0/524257073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c108580 con 0x7f9b9c102430 2024-02-26T04:55:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.792+0000 7f9b92ffd640 1 -- 172.21.15.60:0/524257073 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9b8c017480 con 0x7f9b9c102430 2024-02-26T04:55:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.793+0000 7f9b92ffd640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9b880a20c0 0x7f9b880a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.793+0000 7f9b92ffd640 1 -- 172.21.15.60:0/524257073 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9b8c0cd8f0 con 0x7f9b9c102430 2024-02-26T04:55:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.793+0000 7f9ba1aad640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9b880a20c0 0x7f9b880a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.793+0000 7f9ba1aad640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9b880a20c0 0x7f9b880a4580 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9b98036040 tx=0x7f9b98000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.796+0000 7f9b92ffd640 1 -- 172.21.15.60:0/524257073 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9b8c096680 con 0x7f9b9c102430 2024-02-26T04:55:41.842 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:41 smithi156 ceph-mon[35805]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 126 B/s wr, 0 op/s 2024-02-26T04:55:41.842 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:41 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2370270956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:55:41.842 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:41 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2092771383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:55:41.842 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:41 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/995176566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:55:41.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.990+0000 7f9ba3d11640 1 -- 172.21.15.60:0/524257073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9b9c10cc00 con 0x7f9b9c102430 2024-02-26T04:55:41.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.990+0000 7f9b92ffd640 1 -- 172.21.15.60:0/524257073 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+588 (secure 0 0 0) 0x7f9b8c099d40 con 0x7f9b9c102430 2024-02-26T04:55:41.993 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/556269024 2024-02-27T04:48:18.223906+0000 2024-02-26T04:55:41.993 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1221449590 2024-02-27T04:48:18.223906+0000 2024-02-26T04:55:41.993 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/114766175 2024-02-27T04:47:50.799728+0000 2024-02-26T04:55:41.993 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/2375965956 2024-02-27T04:47:50.799728+0000 2024-02-26T04:55:41.993 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1037269853 2024-02-27T04:48:18.223906+0000 2024-02-26T04:55:41.993 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/3694881479 2024-02-27T04:48:18.223906+0000 2024-02-26T04:55:41.993 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/114766175 2024-02-27T04:47:50.799728+0000 2024-02-26T04:55:41.994 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/558398124 2024-02-27T04:47:50.799728+0000 2024-02-26T04:55:41.994 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/3694881479 2024-02-27T04:48:18.223906+0000 2024-02-26T04:55:41.994 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1162202856 2024-02-27T04:47:50.799728+0000 2024-02-26T04:55:41.994 INFO:teuthology.orchestra.run.smithi060.stderr:listed 10 entries 2024-02-26T04:55:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.994+0000 7f9ba3d11640 1 -- 172.21.15.60:0/524257073 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9b880a20c0 msgr2=0x7f9b880a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.994+0000 7f9ba3d11640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9b880a20c0 0x7f9b880a4580 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9b98036040 tx=0x7f9b98000bc0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.994+0000 7f9ba3d11640 1 -- 172.21.15.60:0/524257073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b9c102430 msgr2=0x7f9b9c197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.994+0000 7f9ba3d11640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b9c102430 0x7f9b9c197820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00b4a0 tx=0x7f9b8c00b960 comp rx=0 tx=0).stop 2024-02-26T04:55:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.994+0000 7f9ba3d11640 1 -- 172.21.15.60:0/524257073 shutdown_connections 2024-02-26T04:55:41.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.994+0000 7f9ba3d11640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9b880a20c0 0x7f9b880a4580 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.994+0000 7f9ba3d11640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b9c102df0 0x7f9b9c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.994+0000 7f9ba3d11640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b9c102430 0x7f9b9c197820 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.994+0000 7f9ba3d11640 1 --2- 172.21.15.60:0/524257073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b9c0ff460 0x7f9b9c1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:41.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.994+0000 7f9ba3d11640 1 -- 172.21.15.60:0/524257073 >> 172.21.15.60:0/524257073 conn(0x7f9b9c0fad90 msgr2=0x7f9b9c105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:41.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.994+0000 7f9ba3d11640 1 -- 172.21.15.60:0/524257073 shutdown_connections 2024-02-26T04:55:41.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:41.994+0000 7f9ba3d11640 1 -- 172.21.15.60:0/524257073 wait complete. 2024-02-26T04:55:42.011 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:55:42.011 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T04:55:42.027 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:55:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.167+0000 7f227b18c640 1 -- 172.21.15.60:0/3011910464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22741012b0 msgr2=0x7f2274101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.167+0000 7f227b18c640 1 --2- 172.21.15.60:0/3011910464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22741012b0 0x7f2274101730 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2268009900 tx=0x7f226802bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.167+0000 7f227b18c640 1 -- 172.21.15.60:0/3011910464 shutdown_connections 2024-02-26T04:55:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.167+0000 7f227b18c640 1 --2- 172.21.15.60:0/3011910464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2274101c70 0x7f2274108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.167+0000 7f227b18c640 1 --2- 172.21.15.60:0/3011910464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22741012b0 0x7f2274101730 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.167+0000 7f227b18c640 1 --2- 172.21.15.60:0/3011910464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22741000c0 0x7f22741004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.167+0000 7f227b18c640 1 -- 172.21.15.60:0/3011910464 >> 172.21.15.60:0/3011910464 conn(0x7f22740fb630 msgr2=0x7f22740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.167+0000 7f227b18c640 1 -- 172.21.15.60:0/3011910464 shutdown_connections 2024-02-26T04:55:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.167+0000 7f227b18c640 1 -- 172.21.15.60:0/3011910464 wait complete. 2024-02-26T04:55:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.168+0000 7f227b18c640 1 Processor -- start 2024-02-26T04:55:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.168+0000 7f227b18c640 1 -- start start 2024-02-26T04:55:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.168+0000 7f227b18c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22741000c0 0x7f2274104050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.168+0000 7f227b18c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22741012b0 0x7f2274102690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.168+0000 7f227b18c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2274101c70 0x7f2274102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.168+0000 7f227b18c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2274103330 con 0x7f22741012b0 2024-02-26T04:55:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.168+0000 7f227b18c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741034a0 con 0x7f2274101c70 2024-02-26T04:55:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.168+0000 7f227b18c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741a1300 con 0x7f22741000c0 2024-02-26T04:55:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.168+0000 7f2273fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22741012b0 0x7f2274102690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.168+0000 7f2273fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22741012b0 0x7f2274102690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46394/0 (socket says 172.21.15.60:46394) 2024-02-26T04:55:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.168+0000 7f2273fff640 1 -- 172.21.15.60:0/2661230875 learned_addr learned my addr 172.21.15.60:0/2661230875 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.168+0000 7f2278f28640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22741000c0 0x7f2274104050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.168+0000 7f2279729640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2274101c70 0x7f2274102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.169+0000 7f2273fff640 1 -- 172.21.15.60:0/2661230875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22741000c0 msgr2=0x7f2274104050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.169+0000 7f2273fff640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22741000c0 0x7f2274104050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.169+0000 7f2273fff640 1 -- 172.21.15.60:0/2661230875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2274101c70 msgr2=0x7f2274102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.169+0000 7f2273fff640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2274101c70 0x7f2274102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.169+0000 7f2273fff640 1 -- 172.21.15.60:0/2661230875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2268009560 con 0x7f22741012b0 2024-02-26T04:55:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.169+0000 7f2278f28640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22741000c0 0x7f2274104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.169+0000 7f2279729640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2274101c70 0x7f2274102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.169+0000 7f2273fff640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22741012b0 0x7f2274102690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2268036040 tx=0x7f22680040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.169+0000 7f2271ffb640 1 -- 172.21.15.60:0/2661230875 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22680044c0 con 0x7f22741012b0 2024-02-26T04:55:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.169+0000 7f2271ffb640 1 -- 172.21.15.60:0/2661230875 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22680052d0 con 0x7f22741012b0 2024-02-26T04:55:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.169+0000 7f227b18c640 1 -- 172.21.15.60:0/2661230875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22741a1550 con 0x7f22741012b0 2024-02-26T04:55:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.169+0000 7f2271ffb640 1 -- 172.21.15.60:0/2661230875 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268003e10 con 0x7f22741012b0 2024-02-26T04:55:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.169+0000 7f227b18c640 1 -- 172.21.15.60:0/2661230875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22741a1ab0 con 0x7f22741012b0 2024-02-26T04:55:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.170+0000 7f227b18c640 1 -- 172.21.15.60:0/2661230875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227410bad0 con 0x7f22741012b0 2024-02-26T04:55:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.171+0000 7f2271ffb640 1 -- 172.21.15.60:0/2661230875 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2268005440 con 0x7f22741012b0 2024-02-26T04:55:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.172+0000 7f2271ffb640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f225c0a2140 0x7f225c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.172+0000 7f2278f28640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f225c0a2140 0x7f225c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.172+0000 7f2271ffb640 1 -- 172.21.15.60:0/2661230875 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f22680f0870 con 0x7f22741012b0 2024-02-26T04:55:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.172+0000 7f2278f28640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f225c0a2140 0x7f225c0a4600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2274101110 tx=0x7f226c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:42.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.175+0000 7f2271ffb640 1 -- 172.21.15.60:0/2661230875 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f22680b9600 con 0x7f22741012b0 2024-02-26T04:55:42.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.358+0000 7f227b18c640 1 -- 172.21.15.60:0/2661230875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f227410bc60 con 0x7f22741012b0 2024-02-26T04:55:42.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.359+0000 7f2271ffb640 1 -- 172.21.15.60:0/2661230875 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24126 (secure 0 0 0) 0x7f22680bccc0 con 0x7f22741012b0 2024-02-26T04:55:42.362 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:42.362 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:55:32.581481+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:55:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.361+0000 7f227b18c640 1 -- 172.21.15.60:0/2661230875 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f225c0a2140 msgr2=0x7f225c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.361+0000 7f227b18c640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f225c0a2140 0x7f225c0a4600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2274101110 tx=0x7f226c009500 comp rx=0 tx=0).stop 2024-02-26T04:55:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.361+0000 7f227b18c640 1 -- 172.21.15.60:0/2661230875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22741012b0 msgr2=0x7f2274102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.361+0000 7f227b18c640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22741012b0 0x7f2274102690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2268036040 tx=0x7f22680040c0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.361+0000 7f227b18c640 1 -- 172.21.15.60:0/2661230875 shutdown_connections 2024-02-26T04:55:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.361+0000 7f227b18c640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f225c0a2140 0x7f225c0a4600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.361+0000 7f227b18c640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2274101c70 0x7f2274102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.361+0000 7f227b18c640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22741012b0 0x7f2274102690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.361+0000 7f227b18c640 1 --2- 172.21.15.60:0/2661230875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22741000c0 0x7f2274104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.361+0000 7f227b18c640 1 -- 172.21.15.60:0/2661230875 >> 172.21.15.60:0/2661230875 conn(0x7f22740fb630 msgr2=0x7f22740fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:42.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.362+0000 7f227b18c640 1 -- 172.21.15.60:0/2661230875 shutdown_connections 2024-02-26T04:55:42.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.362+0000 7f227b18c640 1 -- 172.21.15.60:0/2661230875 wait complete. 2024-02-26T04:55:42.375 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T04:50:44.572921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T04:55:25.530720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T04:55:26.741604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T04:55:30.453569+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T04:55:42.375 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T04:55:42.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.478+0000 7f5315820640 1 -- 172.21.15.60:0/1266450834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53101011a0 msgr2=0x7f53101015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.478+0000 7f5315820640 1 --2- 172.21.15.60:0/1266450834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53101011a0 0x7f53101015c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5308009900 tx=0x7f530802bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:42.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.478+0000 7f5315820640 1 -- 172.21.15.60:0/1266450834 shutdown_connections 2024-02-26T04:55:42.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.478+0000 7f5315820640 1 --2- 172.21.15.60:0/1266450834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5310102d50 0x7f53101095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.478+0000 7f5315820640 1 --2- 172.21.15.60:0/1266450834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5310102390 0x7f5310102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.478+0000 7f5315820640 1 --2- 172.21.15.60:0/1266450834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53101011a0 0x7f53101015c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.478+0000 7f5315820640 1 -- 172.21.15.60:0/1266450834 >> 172.21.15.60:0/1266450834 conn(0x7f53100fc710 msgr2=0x7f53100feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:42.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.478+0000 7f5315820640 1 -- 172.21.15.60:0/1266450834 shutdown_connections 2024-02-26T04:55:42.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.478+0000 7f5315820640 1 -- 172.21.15.60:0/1266450834 wait complete. 2024-02-26T04:55:42.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.478+0000 7f5315820640 1 Processor -- start 2024-02-26T04:55:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.478+0000 7f5315820640 1 -- start start 2024-02-26T04:55:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f5315820640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53101011a0 0x7f5310199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f5315820640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5310102390 0x7f5310199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f5315820640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5310102d50 0x7f53101a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f5315820640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53101a0f10 con 0x7f5310102390 2024-02-26T04:55:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f5315820640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53101a1080 con 0x7f53101011a0 2024-02-26T04:55:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f5315820640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53101a1240 con 0x7f5310102d50 2024-02-26T04:55:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f530e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5310102390 0x7f5310199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f530e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5310102390 0x7f5310199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46414/0 (socket says 172.21.15.60:46414) 2024-02-26T04:55:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f530e7fc640 1 -- 172.21.15.60:0/2436661035 learned_addr learned my addr 172.21.15.60:0/2436661035 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f530effd640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53101011a0 0x7f5310199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f530f7fe640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5310102d50 0x7f53101a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f530e7fc640 1 -- 172.21.15.60:0/2436661035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5310102d50 msgr2=0x7f53101a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f530e7fc640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5310102d50 0x7f53101a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f530e7fc640 1 -- 172.21.15.60:0/2436661035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53101011a0 msgr2=0x7f5310199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f530e7fc640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53101011a0 0x7f5310199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f530e7fc640 1 -- 172.21.15.60:0/2436661035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5308009560 con 0x7f5310102390 2024-02-26T04:55:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f530f7fe640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5310102d50 0x7f53101a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f530e7fc640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5310102390 0x7f5310199960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f530400d950 tx=0x7f530400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f531481e640 1 -- 172.21.15.60:0/2436661035 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5304009a30 con 0x7f5310102390 2024-02-26T04:55:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f531481e640 1 -- 172.21.15.60:0/2436661035 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5304009b90 con 0x7f5310102390 2024-02-26T04:55:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f531481e640 1 -- 172.21.15.60:0/2436661035 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5304004a00 con 0x7f5310102390 2024-02-26T04:55:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f5315820640 1 -- 172.21.15.60:0/2436661035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53101a1520 con 0x7f5310102390 2024-02-26T04:55:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.479+0000 7f5315820640 1 -- 172.21.15.60:0/2436661035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53101a1a80 con 0x7f5310102390 2024-02-26T04:55:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.480+0000 7f5315820640 1 -- 172.21.15.60:0/2436661035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531010cbb0 con 0x7f5310102390 2024-02-26T04:55:42.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.482+0000 7f531481e640 1 -- 172.21.15.60:0/2436661035 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5304009d00 con 0x7f5310102390 2024-02-26T04:55:42.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.482+0000 7f531481e640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f52f80a2140 0x7f52f80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:42.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.482+0000 7f530effd640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f52f80a2140 0x7f52f80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:42.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.483+0000 7f531481e640 1 -- 172.21.15.60:0/2436661035 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f53040cd910 con 0x7f5310102390 2024-02-26T04:55:42.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.483+0000 7f530effd640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f52f80a2140 0x7f52f80a4600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5308005ab0 tx=0x7f5308005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:42.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.487+0000 7f531481e640 1 -- 172.21.15.60:0/2436661035 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f53040966a0 con 0x7f5310102390 2024-02-26T04:55:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.669+0000 7f5315820640 1 -- 172.21.15.60:0/2436661035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53101a1d00 con 0x7f5310102390 2024-02-26T04:55:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.670+0000 7f531481e640 1 -- 172.21.15.60:0/2436661035 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5304099d60 con 0x7f5310102390 2024-02-26T04:55:42.672 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T04:55:42.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.672+0000 7f5315820640 1 -- 172.21.15.60:0/2436661035 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f52f80a2140 msgr2=0x7f52f80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.672+0000 7f5315820640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f52f80a2140 0x7f52f80a4600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5308005ab0 tx=0x7f5308005fd0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.672+0000 7f5315820640 1 -- 172.21.15.60:0/2436661035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5310102390 msgr2=0x7f5310199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.672+0000 7f5315820640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5310102390 0x7f5310199960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f530400d950 tx=0x7f530400de10 comp rx=0 tx=0).stop 2024-02-26T04:55:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.672+0000 7f5315820640 1 -- 172.21.15.60:0/2436661035 shutdown_connections 2024-02-26T04:55:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.672+0000 7f5315820640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f52f80a2140 0x7f52f80a4600 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.672+0000 7f5315820640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5310102d50 0x7f53101a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.672+0000 7f5315820640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5310102390 0x7f5310199960 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.672+0000 7f5315820640 1 --2- 172.21.15.60:0/2436661035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53101011a0 0x7f5310199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.672+0000 7f5315820640 1 -- 172.21.15.60:0/2436661035 >> 172.21.15.60:0/2436661035 conn(0x7f53100fc710 msgr2=0x7f53100733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.672+0000 7f5315820640 1 -- 172.21.15.60:0/2436661035 shutdown_connections 2024-02-26T04:55:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.672+0000 7f5315820640 1 -- 172.21.15.60:0/2436661035 wait complete. 2024-02-26T04:55:42.684 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:42 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2464949377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:55:42.684 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:42 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/524257073' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T04:55:42.684 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:42 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2661230875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:42.685 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T04:55:42.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:42 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2464949377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:55:42.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:42 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/524257073' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T04:55:42.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:42 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2661230875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:42.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.788+0000 7f5ba24de640 1 -- 172.21.15.60:0/2911906131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b9c101c30 msgr2=0x7f5b9c1084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.788+0000 7f5ba24de640 1 --2- 172.21.15.60:0/2911906131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b9c101c30 0x7f5b9c1084c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009ca0 tx=0x7f5b9402c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.788+0000 7f5ba24de640 1 -- 172.21.15.60:0/2911906131 shutdown_connections 2024-02-26T04:55:42.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.788+0000 7f5ba24de640 1 --2- 172.21.15.60:0/2911906131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b9c101c30 0x7f5b9c1084c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.788+0000 7f5ba24de640 1 --2- 172.21.15.60:0/2911906131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b9c101270 0x7f5b9c1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.788+0000 7f5ba24de640 1 --2- 172.21.15.60:0/2911906131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b9c100080 0x7f5b9c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.788+0000 7f5ba24de640 1 -- 172.21.15.60:0/2911906131 >> 172.21.15.60:0/2911906131 conn(0x7f5b9c0fb610 msgr2=0x7f5b9c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:42.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.788+0000 7f5ba24de640 1 -- 172.21.15.60:0/2911906131 shutdown_connections 2024-02-26T04:55:42.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.788+0000 7f5ba24de640 1 -- 172.21.15.60:0/2911906131 wait complete. 2024-02-26T04:55:42.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.788+0000 7f5ba24de640 1 Processor -- start 2024-02-26T04:55:42.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.788+0000 7f5ba24de640 1 -- start start 2024-02-26T04:55:42.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba24de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b9c100080 0x7f5b9c1040e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba24de640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b9c101270 0x7f5b9c102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba24de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b9c101c30 0x7f5b9c102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba24de640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c1033c0 con 0x7f5b9c101c30 2024-02-26T04:55:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba24de640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c103530 con 0x7f5b9c101270 2024-02-26T04:55:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba24de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c1a1260 con 0x7f5b9c100080 2024-02-26T04:55:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba0a7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b9c101c30 0x7f5b9c102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba0a7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b9c101c30 0x7f5b9c102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46432/0 (socket says 172.21.15.60:46432) 2024-02-26T04:55:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba0a7b640 1 -- 172.21.15.60:0/2252993102 learned_addr learned my addr 172.21.15.60:0/2252993102 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5b9b7fe640 1 --2- 172.21.15.60:0/2252993102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b9c101270 0x7f5b9c102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5b9bfff640 1 --2- 172.21.15.60:0/2252993102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b9c100080 0x7f5b9c1040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba0a7b640 1 -- 172.21.15.60:0/2252993102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b9c100080 msgr2=0x7f5b9c1040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba0a7b640 1 --2- 172.21.15.60:0/2252993102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b9c100080 0x7f5b9c1040e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba0a7b640 1 -- 172.21.15.60:0/2252993102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b9c101270 msgr2=0x7f5b9c102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba0a7b640 1 --2- 172.21.15.60:0/2252993102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b9c101270 0x7f5b9c102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba0a7b640 1 -- 172.21.15.60:0/2252993102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b94009900 con 0x7f5b9c101c30 2024-02-26T04:55:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5ba0a7b640 1 --2- 172.21.15.60:0/2252993102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b9c101c30 0x7f5b9c102c60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5b94033000 tx=0x7f5b94034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.789+0000 7f5b997fa640 1 -- 172.21.15.60:0/2252993102 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940329b0 con 0x7f5b9c101c30 2024-02-26T04:55:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.790+0000 7f5b997fa640 1 -- 172.21.15.60:0/2252993102 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b94003e50 con 0x7f5b9c101c30 2024-02-26T04:55:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.790+0000 7f5ba24de640 1 -- 172.21.15.60:0/2252993102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9c1a14b0 con 0x7f5b9c101c30 2024-02-26T04:55:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.790+0000 7f5b997fa640 1 -- 172.21.15.60:0/2252993102 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b94040450 con 0x7f5b9c101c30 2024-02-26T04:55:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.790+0000 7f5ba24de640 1 -- 172.21.15.60:0/2252993102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9c1a1960 con 0x7f5b9c101c30 2024-02-26T04:55:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.790+0000 7f5ba24de640 1 -- 172.21.15.60:0/2252993102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9c10ba90 con 0x7f5b9c101c30 2024-02-26T04:55:42.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.792+0000 7f5b997fa640 1 -- 172.21.15.60:0/2252993102 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5b9400a040 con 0x7f5b9c101c30 2024-02-26T04:55:42.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.793+0000 7f5b997fa640 1 --2- 172.21.15.60:0/2252993102 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5b840a20c0 0x7f5b840a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:42.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.793+0000 7f5b9bfff640 1 --2- 172.21.15.60:0/2252993102 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5b840a20c0 0x7f5b840a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:42.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.793+0000 7f5b997fa640 1 -- 172.21.15.60:0/2252993102 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f5b940f11a0 con 0x7f5b9c101c30 2024-02-26T04:55:42.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.793+0000 7f5b9bfff640 1 --2- 172.21.15.60:0/2252993102 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5b840a20c0 0x7f5b840a4580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c005fd0 tx=0x7f5b8c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:42.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.796+0000 7f5b997fa640 1 -- 172.21.15.60:0/2252993102 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b940b9ed0 con 0x7f5b9c101c30 2024-02-26T04:55:42.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:42 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2464949377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:55:42.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:42 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/524257073' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T04:55:42.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:42 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2661230875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:42.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.980+0000 7f5ba24de640 1 -- 172.21.15.60:0/2252993102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b9c10bc20 con 0x7f5b9c101c30 2024-02-26T04:55:42.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.980+0000 7f5b997fa640 1 -- 172.21.15.60:0/2252993102 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5b940bd590 con 0x7f5b9c101c30 2024-02-26T04:55:42.983 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:55:42.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.982+0000 7f5ba24de640 1 -- 172.21.15.60:0/2252993102 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5b840a20c0 msgr2=0x7f5b840a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.982+0000 7f5ba24de640 1 --2- 172.21.15.60:0/2252993102 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5b840a20c0 0x7f5b840a4580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c005fd0 tx=0x7f5b8c005850 comp rx=0 tx=0).stop 2024-02-26T04:55:42.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.982+0000 7f5ba24de640 1 -- 172.21.15.60:0/2252993102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b9c101c30 msgr2=0x7f5b9c102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:42.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.982+0000 7f5ba24de640 1 --2- 172.21.15.60:0/2252993102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b9c101c30 0x7f5b9c102c60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5b94033000 tx=0x7f5b94034a90 comp rx=0 tx=0).stop 2024-02-26T04:55:42.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.983+0000 7f5ba24de640 1 -- 172.21.15.60:0/2252993102 shutdown_connections 2024-02-26T04:55:42.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.983+0000 7f5ba24de640 1 --2- 172.21.15.60:0/2252993102 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5b840a20c0 0x7f5b840a4580 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.983+0000 7f5ba24de640 1 --2- 172.21.15.60:0/2252993102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b9c101c30 0x7f5b9c102c60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.983+0000 7f5ba24de640 1 --2- 172.21.15.60:0/2252993102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b9c101270 0x7f5b9c102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.983+0000 7f5ba24de640 1 --2- 172.21.15.60:0/2252993102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b9c100080 0x7f5b9c1040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:42.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.983+0000 7f5ba24de640 1 -- 172.21.15.60:0/2252993102 >> 172.21.15.60:0/2252993102 conn(0x7f5b9c0fb610 msgr2=0x7f5b9c0682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:42.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.983+0000 7f5ba24de640 1 -- 172.21.15.60:0/2252993102 shutdown_connections 2024-02-26T04:55:42.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:42.983+0000 7f5ba24de640 1 -- 172.21.15.60:0/2252993102 wait complete. 2024-02-26T04:55:42.996 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T04:55:43.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 -- 172.21.15.60:0/3491390951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac5c101270 msgr2=0x7fac5c1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 --2- 172.21.15.60:0/3491390951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac5c101270 0x7fac5c1016f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009900 tx=0x7fac4c02bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 -- 172.21.15.60:0/3491390951 shutdown_connections 2024-02-26T04:55:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 --2- 172.21.15.60:0/3491390951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac5c101c30 0x7fac5c1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 --2- 172.21.15.60:0/3491390951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac5c101270 0x7fac5c1016f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 --2- 172.21.15.60:0/3491390951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac5c100080 0x7fac5c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 -- 172.21.15.60:0/3491390951 >> 172.21.15.60:0/3491390951 conn(0x7fac5c0fb610 msgr2=0x7fac5c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 -- 172.21.15.60:0/3491390951 shutdown_connections 2024-02-26T04:55:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 -- 172.21.15.60:0/3491390951 wait complete. 2024-02-26T04:55:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 Processor -- start 2024-02-26T04:55:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 -- start start 2024-02-26T04:55:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac5c100080 0x7fac5c104010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac5c101270 0x7fac5c102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac5c101c30 0x7fac5c102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c1032f0 con 0x7fac5c100080 2024-02-26T04:55:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c103460 con 0x7fac5c101c30 2024-02-26T04:55:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.099+0000 7fac63836640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c1a12c0 con 0x7fac5c101270 2024-02-26T04:55:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac615d2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac5c100080 0x7fac5c104010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac615d2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac5c100080 0x7fac5c104010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46456/0 (socket says 172.21.15.60:46456) 2024-02-26T04:55:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac615d2640 1 -- 172.21.15.60:0/370145661 learned_addr learned my addr 172.21.15.60:0/370145661 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac61dd3640 1 --2- 172.21.15.60:0/370145661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac5c101c30 0x7fac5c102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac615d2640 1 -- 172.21.15.60:0/370145661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac5c101270 msgr2=0x7fac5c102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac615d2640 1 --2- 172.21.15.60:0/370145661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac5c101270 0x7fac5c102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac615d2640 1 -- 172.21.15.60:0/370145661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac5c101c30 msgr2=0x7fac5c102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac615d2640 1 --2- 172.21.15.60:0/370145661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac5c101c30 0x7fac5c102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac615d2640 1 -- 172.21.15.60:0/370145661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c009560 con 0x7fac5c100080 2024-02-26T04:55:43.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac615d2640 1 --2- 172.21.15.60:0/370145661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac5c100080 0x7fac5c104010 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fac5800e9f0 tx=0x7fac5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:43.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac527fc640 1 -- 172.21.15.60:0/370145661 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac580096e0 con 0x7fac5c100080 2024-02-26T04:55:43.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac527fc640 1 -- 172.21.15.60:0/370145661 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac5800cd80 con 0x7fac5c100080 2024-02-26T04:55:43.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac527fc640 1 -- 172.21.15.60:0/370145661 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac58010430 con 0x7fac5c100080 2024-02-26T04:55:43.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac63836640 1 -- 172.21.15.60:0/370145661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5c1a1570 con 0x7fac5c100080 2024-02-26T04:55:43.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.100+0000 7fac63836640 1 -- 172.21.15.60:0/370145661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac5c1a1a30 con 0x7fac5c100080 2024-02-26T04:55:43.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.101+0000 7fac63836640 1 -- 172.21.15.60:0/370145661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac5c10ba90 con 0x7fac5c100080 2024-02-26T04:55:43.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.102+0000 7fac527fc640 1 -- 172.21.15.60:0/370145661 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fac58003680 con 0x7fac5c100080 2024-02-26T04:55:43.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.103+0000 7fac527fc640 1 --2- 172.21.15.60:0/370145661 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fac480a20c0 0x7fac480a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:43.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.103+0000 7fac60dd1640 1 --2- 172.21.15.60:0/370145661 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fac480a20c0 0x7fac480a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:43.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.103+0000 7fac527fc640 1 -- 172.21.15.60:0/370145661 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fac58014070 con 0x7fac5c100080 2024-02-26T04:55:43.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.103+0000 7fac60dd1640 1 --2- 172.21.15.60:0/370145661 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fac480a20c0 0x7fac480a4580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac4c02c6a0 tx=0x7fac4c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:43.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.107+0000 7fac527fc640 1 -- 172.21.15.60:0/370145661 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac58097050 con 0x7fac5c100080 2024-02-26T04:55:43.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.291+0000 7fac63836640 1 -- 172.21.15.60:0/370145661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac5c1a1cb0 con 0x7fac5c100080 2024-02-26T04:55:43.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.291+0000 7fac527fc640 1 -- 172.21.15.60:0/370145661 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac5809a710 con 0x7fac5c100080 2024-02-26T04:55:43.294 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T04:55:43.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.293+0000 7fac63836640 1 -- 172.21.15.60:0/370145661 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fac480a20c0 msgr2=0x7fac480a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.293+0000 7fac63836640 1 --2- 172.21.15.60:0/370145661 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fac480a20c0 0x7fac480a4580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac4c02c6a0 tx=0x7fac4c000bc0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.293+0000 7fac63836640 1 -- 172.21.15.60:0/370145661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac5c100080 msgr2=0x7fac5c104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.293+0000 7fac63836640 1 --2- 172.21.15.60:0/370145661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac5c100080 0x7fac5c104010 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fac5800e9f0 tx=0x7fac5800eeb0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.293+0000 7fac63836640 1 -- 172.21.15.60:0/370145661 shutdown_connections 2024-02-26T04:55:43.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.293+0000 7fac63836640 1 --2- 172.21.15.60:0/370145661 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fac480a20c0 0x7fac480a4580 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.293+0000 7fac63836640 1 --2- 172.21.15.60:0/370145661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac5c101c30 0x7fac5c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.293+0000 7fac63836640 1 --2- 172.21.15.60:0/370145661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac5c101270 0x7fac5c102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.293+0000 7fac63836640 1 --2- 172.21.15.60:0/370145661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac5c100080 0x7fac5c104010 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.293+0000 7fac63836640 1 -- 172.21.15.60:0/370145661 >> 172.21.15.60:0/370145661 conn(0x7fac5c0fb610 msgr2=0x7fac5c0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:43.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.293+0000 7fac63836640 1 -- 172.21.15.60:0/370145661 shutdown_connections 2024-02-26T04:55:43.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.293+0000 7fac63836640 1 -- 172.21.15.60:0/370145661 wait complete. 2024-02-26T04:55:43.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T04:55:43.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.409+0000 7f936f1cc640 1 -- 172.21.15.60:0/2643200923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93681011a0 msgr2=0x7f93681015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.409+0000 7f936f1cc640 1 --2- 172.21.15.60:0/2643200923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93681011a0 0x7f93681015c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9360009900 tx=0x7f936002bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:43.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.409+0000 7f936f1cc640 1 -- 172.21.15.60:0/2643200923 shutdown_connections 2024-02-26T04:55:43.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.409+0000 7f936f1cc640 1 --2- 172.21.15.60:0/2643200923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9368102d50 0x7f93681095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.409+0000 7f936f1cc640 1 --2- 172.21.15.60:0/2643200923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9368102390 0x7f9368102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.409+0000 7f936f1cc640 1 --2- 172.21.15.60:0/2643200923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93681011a0 0x7f93681015c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.409+0000 7f936f1cc640 1 -- 172.21.15.60:0/2643200923 >> 172.21.15.60:0/2643200923 conn(0x7f93680fc710 msgr2=0x7f93680feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:43.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.409+0000 7f936f1cc640 1 -- 172.21.15.60:0/2643200923 shutdown_connections 2024-02-26T04:55:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.409+0000 7f936f1cc640 1 -- 172.21.15.60:0/2643200923 wait complete. 2024-02-26T04:55:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.409+0000 7f936f1cc640 1 Processor -- start 2024-02-26T04:55:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.409+0000 7f936f1cc640 1 -- start start 2024-02-26T04:55:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f936f1cc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93681011a0 0x7f93681971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f936f1cc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9368102390 0x7f93681976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f936f1cc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9368102d50 0x7f936819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f936f1cc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936819ec90 con 0x7f9368102390 2024-02-26T04:55:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f936f1cc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936819ee00 con 0x7f93681011a0 2024-02-26T04:55:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f936f1cc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936819efc0 con 0x7f9368102d50 2024-02-26T04:55:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f9367fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9368102390 0x7f93681976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f9367fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9368102390 0x7f93681976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46474/0 (socket says 172.21.15.60:46474) 2024-02-26T04:55:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f9367fff640 1 -- 172.21.15.60:0/1632918557 learned_addr learned my addr 172.21.15.60:0/1632918557 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f936cf68640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93681011a0 0x7f93681971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f936d769640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9368102d50 0x7f936819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f9367fff640 1 -- 172.21.15.60:0/1632918557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9368102d50 msgr2=0x7f936819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f9367fff640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9368102d50 0x7f936819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f9367fff640 1 -- 172.21.15.60:0/1632918557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93681011a0 msgr2=0x7f93681971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f9367fff640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93681011a0 0x7f93681971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f9367fff640 1 -- 172.21.15.60:0/1632918557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9360009560 con 0x7f9368102390 2024-02-26T04:55:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f936cf68640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93681011a0 0x7f93681971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:55:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f9367fff640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9368102390 0x7f93681976e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f935800cce0 tx=0x7f935800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f9365ffb640 1 -- 172.21.15.60:0/1632918557 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935800f650 con 0x7f9368102390 2024-02-26T04:55:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f9365ffb640 1 -- 172.21.15.60:0/1632918557 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f935800ce90 con 0x7f9368102390 2024-02-26T04:55:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f936f1cc640 1 -- 172.21.15.60:0/1632918557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936819f2a0 con 0x7f9368102390 2024-02-26T04:55:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.410+0000 7f9365ffb640 1 -- 172.21.15.60:0/1632918557 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9358018bf0 con 0x7f9368102390 2024-02-26T04:55:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.411+0000 7f936f1cc640 1 -- 172.21.15.60:0/1632918557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936819f7b0 con 0x7f9368102390 2024-02-26T04:55:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.412+0000 7f936f1cc640 1 -- 172.21.15.60:0/1632918557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93681084c0 con 0x7f9368102390 2024-02-26T04:55:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.412+0000 7f9365ffb640 1 -- 172.21.15.60:0/1632918557 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9358018d50 con 0x7f9368102390 2024-02-26T04:55:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.413+0000 7f9365ffb640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f93540a20c0 0x7f93540a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.413+0000 7f936cf68640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f93540a20c0 0x7f93540a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:43.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.413+0000 7f9365ffb640 1 -- 172.21.15.60:0/1632918557 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f93580ce400 con 0x7f9368102390 2024-02-26T04:55:43.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.414+0000 7f936cf68640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f93540a20c0 0x7f93540a4580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9360005ab0 tx=0x7f9360005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:43.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.418+0000 7f9365ffb640 1 -- 172.21.15.60:0/1632918557 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9358097190 con 0x7f9368102390 2024-02-26T04:55:43.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.600+0000 7f936f1cc640 1 -- 172.21.15.60:0/1632918557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f936802cda0 con 0x7f9368102390 2024-02-26T04:55:43.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.600+0000 7f9365ffb640 1 -- 172.21.15.60:0/1632918557 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f935809a850 con 0x7f9368102390 2024-02-26T04:55:43.602 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:55:43.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.602+0000 7f936f1cc640 1 -- 172.21.15.60:0/1632918557 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f93540a20c0 msgr2=0x7f93540a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.602+0000 7f936f1cc640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f93540a20c0 0x7f93540a4580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9360005ab0 tx=0x7f9360005fd0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.602+0000 7f936f1cc640 1 -- 172.21.15.60:0/1632918557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9368102390 msgr2=0x7f93681976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.602+0000 7f936f1cc640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9368102390 0x7f93681976e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f935800cce0 tx=0x7f935800a620 comp rx=0 tx=0).stop 2024-02-26T04:55:43.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.602+0000 7f936f1cc640 1 -- 172.21.15.60:0/1632918557 shutdown_connections 2024-02-26T04:55:43.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.602+0000 7f936f1cc640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f93540a20c0 0x7f93540a4580 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.602+0000 7f936f1cc640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9368102d50 0x7f936819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.602+0000 7f936f1cc640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9368102390 0x7f93681976e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.602+0000 7f936f1cc640 1 --2- 172.21.15.60:0/1632918557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93681011a0 0x7f93681971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.602+0000 7f936f1cc640 1 -- 172.21.15.60:0/1632918557 >> 172.21.15.60:0/1632918557 conn(0x7f93680fc710 msgr2=0x7f9368105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:43.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.602+0000 7f936f1cc640 1 -- 172.21.15.60:0/1632918557 shutdown_connections 2024-02-26T04:55:43.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.602+0000 7f936f1cc640 1 -- 172.21.15.60:0/1632918557 wait complete. 2024-02-26T04:55:43.614 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:43 smithi060 ceph-mon[32382]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2024-02-26T04:55:43.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:43 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2436661035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:55:43.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:43 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2252993102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:55:43.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:43 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/370145661' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:55:43.616 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T04:55:43.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.717+0000 7ffaf3ade640 1 -- 172.21.15.60:0/2443270365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec101280 msgr2=0x7ffaec101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.717+0000 7ffaf3ade640 1 --2- 172.21.15.60:0/2443270365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec101280 0x7ffaec101700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009900 tx=0x7ffae002bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.717+0000 7ffaf3ade640 1 -- 172.21.15.60:0/2443270365 shutdown_connections 2024-02-26T04:55:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.717+0000 7ffaf3ade640 1 --2- 172.21.15.60:0/2443270365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffaec101c40 0x7ffaec1084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.717+0000 7ffaf3ade640 1 --2- 172.21.15.60:0/2443270365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec101280 0x7ffaec101700 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.717+0000 7ffaf3ade640 1 --2- 172.21.15.60:0/2443270365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaec100090 0x7ffaec1004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.717+0000 7ffaf3ade640 1 -- 172.21.15.60:0/2443270365 >> 172.21.15.60:0/2443270365 conn(0x7ffaec0fb620 msgr2=0x7ffaec0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.717+0000 7ffaf3ade640 1 -- 172.21.15.60:0/2443270365 shutdown_connections 2024-02-26T04:55:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.717+0000 7ffaf3ade640 1 -- 172.21.15.60:0/2443270365 wait complete. 2024-02-26T04:55:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.717+0000 7ffaf3ade640 1 Processor -- start 2024-02-26T04:55:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.717+0000 7ffaf3ade640 1 -- start start 2024-02-26T04:55:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf3ade640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaec100090 0x7ffaec197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf3ade640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffaec101280 0x7ffaec197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf3ade640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec101c40 0x7ffaec19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf3ade640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec19ed00 con 0x7ffaec101c40 2024-02-26T04:55:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf3ade640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec19ee70 con 0x7ffaec100090 2024-02-26T04:55:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf3ade640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec19f030 con 0x7ffaec101280 2024-02-26T04:55:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf207b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec101c40 0x7ffaec19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf207b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec101c40 0x7ffaec19e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46498/0 (socket says 172.21.15.60:46498) 2024-02-26T04:55:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf207b640 1 -- 172.21.15.60:0/3815921517 learned_addr learned my addr 172.21.15.60:0/3815921517 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf187a640 1 --2- 172.21.15.60:0/3815921517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaec100090 0x7ffaec197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf1079640 1 --2- 172.21.15.60:0/3815921517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffaec101280 0x7ffaec197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf207b640 1 -- 172.21.15.60:0/3815921517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffaec101280 msgr2=0x7ffaec197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf207b640 1 --2- 172.21.15.60:0/3815921517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffaec101280 0x7ffaec197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf207b640 1 -- 172.21.15.60:0/3815921517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaec100090 msgr2=0x7ffaec197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf207b640 1 --2- 172.21.15.60:0/3815921517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaec100090 0x7ffaec197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf207b640 1 -- 172.21.15.60:0/3815921517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae0009560 con 0x7ffaec101c40 2024-02-26T04:55:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf207b640 1 --2- 172.21.15.60:0/3815921517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec101c40 0x7ffaec19e7c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffae400d750 tx=0x7ffae400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaeaffd640 1 -- 172.21.15.60:0/3815921517 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae4009bd0 con 0x7ffaec101c40 2024-02-26T04:55:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaeaffd640 1 -- 172.21.15.60:0/3815921517 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffae400de60 con 0x7ffaec101c40 2024-02-26T04:55:43.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf3ade640 1 -- 172.21.15.60:0/3815921517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaec19f310 con 0x7ffaec101c40 2024-02-26T04:55:43.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaeaffd640 1 -- 172.21.15.60:0/3815921517 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae4004e70 con 0x7ffaec101c40 2024-02-26T04:55:43.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.718+0000 7ffaf3ade640 1 -- 172.21.15.60:0/3815921517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaec19f840 con 0x7ffaec101c40 2024-02-26T04:55:43.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.719+0000 7ffaf3ade640 1 -- 172.21.15.60:0/3815921517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaec10baa0 con 0x7ffaec101c40 2024-02-26T04:55:43.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.721+0000 7ffaeaffd640 1 -- 172.21.15.60:0/3815921517 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ffae4005000 con 0x7ffaec101c40 2024-02-26T04:55:43.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.721+0000 7ffaeaffd640 1 --2- 172.21.15.60:0/3815921517 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffad40a2140 0x7ffad40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:43.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.722+0000 7ffaf187a640 1 --2- 172.21.15.60:0/3815921517 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffad40a2140 0x7ffad40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:43.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.722+0000 7ffaeaffd640 1 -- 172.21.15.60:0/3815921517 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ffae40cdb90 con 0x7ffaec101c40 2024-02-26T04:55:43.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.722+0000 7ffaf187a640 1 --2- 172.21.15.60:0/3815921517 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffad40a2140 0x7ffad40a4600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffaec1010e0 tx=0x7ffadc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:43.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.725+0000 7ffaeaffd640 1 -- 172.21.15.60:0/3815921517 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffae409a520 con 0x7ffaec101c40 2024-02-26T04:55:43.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:43 smithi191 ceph-mon[36352]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2024-02-26T04:55:43.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:43 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2436661035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:55:43.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:43 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2252993102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:55:43.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:43 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/370145661' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:55:43.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.918+0000 7ffaf3ade640 1 -- 172.21.15.60:0/3815921517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffaec10bc30 con 0x7ffaec101c40 2024-02-26T04:55:43.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.919+0000 7ffaeaffd640 1 -- 172.21.15.60:0/3815921517 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+588 (secure 0 0 0) 0x7ffae4017070 con 0x7ffaec101c40 2024-02-26T04:55:43.921 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/556269024 2024-02-27T04:48:18.223906+0000 2024-02-26T04:55:43.921 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1221449590 2024-02-27T04:48:18.223906+0000 2024-02-26T04:55:43.921 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/114766175 2024-02-27T04:47:50.799728+0000 2024-02-26T04:55:43.922 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/2375965956 2024-02-27T04:47:50.799728+0000 2024-02-26T04:55:43.922 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1037269853 2024-02-27T04:48:18.223906+0000 2024-02-26T04:55:43.922 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/3694881479 2024-02-27T04:48:18.223906+0000 2024-02-26T04:55:43.922 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/114766175 2024-02-27T04:47:50.799728+0000 2024-02-26T04:55:43.922 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/558398124 2024-02-27T04:47:50.799728+0000 2024-02-26T04:55:43.922 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/3694881479 2024-02-27T04:48:18.223906+0000 2024-02-26T04:55:43.922 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1162202856 2024-02-27T04:47:50.799728+0000 2024-02-26T04:55:43.922 INFO:teuthology.orchestra.run.smithi060.stderr:listed 10 entries 2024-02-26T04:55:43.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.921+0000 7ffaf3ade640 1 -- 172.21.15.60:0/3815921517 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffad40a2140 msgr2=0x7ffad40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.921+0000 7ffaf3ade640 1 --2- 172.21.15.60:0/3815921517 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffad40a2140 0x7ffad40a4600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffaec1010e0 tx=0x7ffadc009500 comp rx=0 tx=0).stop 2024-02-26T04:55:43.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.921+0000 7ffaf3ade640 1 -- 172.21.15.60:0/3815921517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec101c40 msgr2=0x7ffaec19e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:43.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.921+0000 7ffaf3ade640 1 --2- 172.21.15.60:0/3815921517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec101c40 0x7ffaec19e7c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffae400d750 tx=0x7ffae400dc10 comp rx=0 tx=0).stop 2024-02-26T04:55:43.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.921+0000 7ffaf3ade640 1 -- 172.21.15.60:0/3815921517 shutdown_connections 2024-02-26T04:55:43.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.921+0000 7ffaf3ade640 1 --2- 172.21.15.60:0/3815921517 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffad40a2140 0x7ffad40a4600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.921+0000 7ffaf3ade640 1 --2- 172.21.15.60:0/3815921517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec101c40 0x7ffaec19e7c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.921+0000 7ffaf3ade640 1 --2- 172.21.15.60:0/3815921517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffaec101280 0x7ffaec197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.921+0000 7ffaf3ade640 1 --2- 172.21.15.60:0/3815921517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaec100090 0x7ffaec197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:43.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.921+0000 7ffaf3ade640 1 -- 172.21.15.60:0/3815921517 >> 172.21.15.60:0/3815921517 conn(0x7ffaec0fb620 msgr2=0x7ffaec0fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:43.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.921+0000 7ffaf3ade640 1 -- 172.21.15.60:0/3815921517 shutdown_connections 2024-02-26T04:55:43.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:43.921+0000 7ffaf3ade640 1 -- 172.21.15.60:0/3815921517 wait complete. 2024-02-26T04:55:43.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:43 smithi156 ceph-mon[35805]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2024-02-26T04:55:43.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:43 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2436661035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:55:43.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:43 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2252993102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:55:43.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:43 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/370145661' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:55:43.934 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-02-26T04:55:43.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:43.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-02-26T04:55:43.959 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:ceph-fuse[73742]: fuse finished with error 0 and tester_r 0 2024-02-26T04:55:43.961 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T04:55:44.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:44 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1632918557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:55:44.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:44 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3815921517' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T04:55:44.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:44 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1632918557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:55:44.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:44 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3815921517' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T04:55:44.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:44 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1632918557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:55:44.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:44 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3815921517' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T04:55:45.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:45 smithi191 ceph-mon[36352]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 92 B/s wr, 0 op/s 2024-02-26T04:55:45.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:45 smithi156 ceph-mon[35805]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 92 B/s wr, 0 op/s 2024-02-26T04:55:45.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:45 smithi060 ceph-mon[32382]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 92 B/s wr, 0 op/s 2024-02-26T04:55:47.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:47 smithi191 ceph-mon[36352]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-26T04:55:47.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:47 smithi156 ceph-mon[35805]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-26T04:55:47.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:47 smithi060 ceph-mon[32382]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-26T04:55:49.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:49 smithi156 ceph-mon[35805]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-26T04:55:49.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:49 smithi060 ceph-mon[32382]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-26T04:55:50.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:49 smithi191 ceph-mon[36352]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-26T04:55:50.063 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T04:55:50.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:55:50.063 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-02-26T04:55:50.089 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:55:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.243+0000 7f8fe3620640 1 -- 172.21.15.60:0/1284160548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fdc1000c0 msgr2=0x7f8fdc1004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.243+0000 7f8fe3620640 1 --2- 172.21.15.60:0/1284160548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fdc1000c0 0x7f8fdc1004e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009900 tx=0x7f8fd802bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.243+0000 7f8fe3620640 1 -- 172.21.15.60:0/1284160548 shutdown_connections 2024-02-26T04:55:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.243+0000 7f8fe3620640 1 --2- 172.21.15.60:0/1284160548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fdc101c70 0x7f8fdc108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.243+0000 7f8fe3620640 1 --2- 172.21.15.60:0/1284160548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fdc1012b0 0x7f8fdc101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.243+0000 7f8fe3620640 1 --2- 172.21.15.60:0/1284160548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fdc1000c0 0x7f8fdc1004e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.243+0000 7f8fe3620640 1 -- 172.21.15.60:0/1284160548 >> 172.21.15.60:0/1284160548 conn(0x7f8fdc0fb630 msgr2=0x7f8fdc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.243+0000 7f8fe3620640 1 -- 172.21.15.60:0/1284160548 shutdown_connections 2024-02-26T04:55:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.243+0000 7f8fe3620640 1 -- 172.21.15.60:0/1284160548 wait complete. 2024-02-26T04:55:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe3620640 1 Processor -- start 2024-02-26T04:55:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe3620640 1 -- start start 2024-02-26T04:55:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe3620640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fdc1000c0 0x7f8fdc10a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe3620640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fdc1012b0 0x7f8fdc108fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe3620640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fdc101c70 0x7f8fdc109500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe3620640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc109c60 con 0x7f8fdc1012b0 2024-02-26T04:55:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe3620640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc109dd0 con 0x7f8fdc101c70 2024-02-26T04:55:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe3620640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1a1280 con 0x7f8fdc1000c0 2024-02-26T04:55:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe0bbb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fdc1012b0 0x7f8fdc108fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe0bbb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fdc1012b0 0x7f8fdc108fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50890/0 (socket says 172.21.15.60:50890) 2024-02-26T04:55:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe0bbb640 1 -- 172.21.15.60:0/92805493 learned_addr learned my addr 172.21.15.60:0/92805493 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe13bc640 1 --2- 172.21.15.60:0/92805493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fdc1000c0 0x7f8fdc10a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe1bbd640 1 --2- 172.21.15.60:0/92805493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fdc101c70 0x7f8fdc109500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe0bbb640 1 -- 172.21.15.60:0/92805493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fdc1000c0 msgr2=0x7f8fdc10a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.244+0000 7f8fe0bbb640 1 --2- 172.21.15.60:0/92805493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fdc1000c0 0x7f8fdc10a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.245+0000 7f8fe0bbb640 1 -- 172.21.15.60:0/92805493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fdc101c70 msgr2=0x7f8fdc109500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.245+0000 7f8fe0bbb640 1 --2- 172.21.15.60:0/92805493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fdc101c70 0x7f8fdc109500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.245+0000 7f8fe0bbb640 1 -- 172.21.15.60:0/92805493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd8009560 con 0x7f8fdc1012b0 2024-02-26T04:55:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.245+0000 7f8fe0bbb640 1 --2- 172.21.15.60:0/92805493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fdc1012b0 0x7f8fdc108fc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00d7a0 tx=0x7f8fcc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.245+0000 7f8fd27fc640 1 -- 172.21.15.60:0/92805493 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc009c30 con 0x7f8fdc1012b0 2024-02-26T04:55:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.245+0000 7f8fd27fc640 1 -- 172.21.15.60:0/92805493 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fcc00deb0 con 0x7f8fdc1012b0 2024-02-26T04:55:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.245+0000 7f8fe3620640 1 -- 172.21.15.60:0/92805493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fdc1a1530 con 0x7f8fdc1012b0 2024-02-26T04:55:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.245+0000 7f8fd27fc640 1 -- 172.21.15.60:0/92805493 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc004ea0 con 0x7f8fdc1012b0 2024-02-26T04:55:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.245+0000 7f8fe3620640 1 -- 172.21.15.60:0/92805493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc1a1a40 con 0x7f8fdc1012b0 2024-02-26T04:55:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.246+0000 7f8fe3620640 1 -- 172.21.15.60:0/92805493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fdc1073e0 con 0x7f8fdc1012b0 2024-02-26T04:55:50.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.247+0000 7f8fd27fc640 1 -- 172.21.15.60:0/92805493 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8fcc005020 con 0x7f8fdc1012b0 2024-02-26T04:55:50.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.248+0000 7f8fd27fc640 1 --2- 172.21.15.60:0/92805493 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8fc80a2140 0x7f8fc80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:50.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.248+0000 7f8fd27fc640 1 -- 172.21.15.60:0/92805493 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8fcc0cd830 con 0x7f8fdc1012b0 2024-02-26T04:55:50.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.248+0000 7f8fe13bc640 1 --2- 172.21.15.60:0/92805493 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8fc80a2140 0x7f8fc80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:50.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.248+0000 7f8fe13bc640 1 --2- 172.21.15.60:0/92805493 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8fc80a2140 0x7f8fc80a4600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8005ab0 tx=0x7f8fd8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:50.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.252+0000 7f8fd27fc640 1 -- 172.21.15.60:0/92805493 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8fcc096610 con 0x7f8fdc1012b0 2024-02-26T04:55:50.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.436+0000 7f8fe3620640 1 -- 172.21.15.60:0/92805493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fdc02cdd0 con 0x7f8fdc1012b0 2024-02-26T04:55:50.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.438+0000 7f8fd27fc640 1 -- 172.21.15.60:0/92805493 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24126 (secure 0 0 0) 0x7f8fcc099cd0 con 0x7f8fdc1012b0 2024-02-26T04:55:50.441 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:50.441 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:55:32.581481+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:55:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.440+0000 7f8fe3620640 1 -- 172.21.15.60:0/92805493 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8fc80a2140 msgr2=0x7f8fc80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.440+0000 7f8fe3620640 1 --2- 172.21.15.60:0/92805493 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8fc80a2140 0x7f8fc80a4600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8005ab0 tx=0x7f8fd8005fd0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.440+0000 7f8fe3620640 1 -- 172.21.15.60:0/92805493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fdc1012b0 msgr2=0x7f8fdc108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.440+0000 7f8fe3620640 1 --2- 172.21.15.60:0/92805493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fdc1012b0 0x7f8fdc108fc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00d7a0 tx=0x7f8fcc00dc60 comp rx=0 tx=0).stop 2024-02-26T04:55:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.440+0000 7f8fe3620640 1 -- 172.21.15.60:0/92805493 shutdown_connections 2024-02-26T04:55:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.440+0000 7f8fe3620640 1 --2- 172.21.15.60:0/92805493 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8fc80a2140 0x7f8fc80a4600 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.440+0000 7f8fe3620640 1 --2- 172.21.15.60:0/92805493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8fdc101c70 0x7f8fdc109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.440+0000 7f8fe3620640 1 --2- 172.21.15.60:0/92805493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fdc1012b0 0x7f8fdc108fc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.440+0000 7f8fe3620640 1 --2- 172.21.15.60:0/92805493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fdc1000c0 0x7f8fdc10a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.440+0000 7f8fe3620640 1 -- 172.21.15.60:0/92805493 >> 172.21.15.60:0/92805493 conn(0x7f8fdc0fb630 msgr2=0x7f8fdc1045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.440+0000 7f8fe3620640 1 -- 172.21.15.60:0/92805493 shutdown_connections 2024-02-26T04:55:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.440+0000 7f8fe3620640 1 -- 172.21.15.60:0/92805493 wait complete. 2024-02-26T04:55:50.454 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:55:50.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.555+0000 7efe92155640 1 -- 172.21.15.60:0/4181927240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c106e90 msgr2=0x7efe8c1092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:50.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.555+0000 7efe92155640 1 --2- 172.21.15.60:0/4181927240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c106e90 0x7efe8c1092e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe84009ca0 tx=0x7efe8402c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.555+0000 7efe92155640 1 -- 172.21.15.60:0/4181927240 shutdown_connections 2024-02-26T04:55:50.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.555+0000 7efe92155640 1 --2- 172.21.15.60:0/4181927240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c106e90 0x7efe8c1092e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.555+0000 7efe92155640 1 --2- 172.21.15.60:0/4181927240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe8c104560 0x7efe8c106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.555+0000 7efe92155640 1 --2- 172.21.15.60:0/4181927240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe8c101c30 0x7efe8c104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.555+0000 7efe92155640 1 -- 172.21.15.60:0/4181927240 >> 172.21.15.60:0/4181927240 conn(0x7efe8c0fb660 msgr2=0x7efe8c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.555+0000 7efe92155640 1 -- 172.21.15.60:0/4181927240 shutdown_connections 2024-02-26T04:55:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.555+0000 7efe92155640 1 -- 172.21.15.60:0/4181927240 wait complete. 2024-02-26T04:55:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.556+0000 7efe92155640 1 Processor -- start 2024-02-26T04:55:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.556+0000 7efe92155640 1 -- start start 2024-02-26T04:55:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.556+0000 7efe92155640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe8c101c30 0x7efe8c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.556+0000 7efe92155640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c104560 0x7efe8c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.556+0000 7efe92155640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe8c106e90 0x7efe8c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.556+0000 7efe92155640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c19ed60 con 0x7efe8c104560 2024-02-26T04:55:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.556+0000 7efe92155640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c19eed0 con 0x7efe8c101c30 2024-02-26T04:55:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.556+0000 7efe92155640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c19f090 con 0x7efe8c106e90 2024-02-26T04:55:50.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.556+0000 7efe8affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c104560 0x7efe8c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe8affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c104560 0x7efe8c1977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50908/0 (socket says 172.21.15.60:50908) 2024-02-26T04:55:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe8affd640 1 -- 172.21.15.60:0/1141480910 learned_addr learned my addr 172.21.15.60:0/1141480910 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe8b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe8c101c30 0x7efe8c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe8bfff640 1 --2- 172.21.15.60:0/1141480910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe8c106e90 0x7efe8c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe8affd640 1 -- 172.21.15.60:0/1141480910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe8c106e90 msgr2=0x7efe8c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe8affd640 1 --2- 172.21.15.60:0/1141480910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe8c106e90 0x7efe8c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe8affd640 1 -- 172.21.15.60:0/1141480910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe8c101c30 msgr2=0x7efe8c197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe8affd640 1 --2- 172.21.15.60:0/1141480910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe8c101c30 0x7efe8c197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe8affd640 1 -- 172.21.15.60:0/1141480910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe84009900 con 0x7efe8c104560 2024-02-26T04:55:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe8bfff640 1 --2- 172.21.15.60:0/1141480910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe8c106e90 0x7efe8c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe8affd640 1 --2- 172.21.15.60:0/1141480910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c104560 0x7efe8c1977b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efe8000b450 tx=0x7efe8000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe88ff9640 1 -- 172.21.15.60:0/1141480910 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe80011890 con 0x7efe8c104560 2024-02-26T04:55:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe88ff9640 1 -- 172.21.15.60:0/1141480910 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe800103e0 con 0x7efe8c104560 2024-02-26T04:55:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe92155640 1 -- 172.21.15.60:0/1141480910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8c19f370 con 0x7efe8c104560 2024-02-26T04:55:50.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe88ff9640 1 -- 172.21.15.60:0/1141480910 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8000f580 con 0x7efe8c104560 2024-02-26T04:55:50.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.557+0000 7efe92155640 1 -- 172.21.15.60:0/1141480910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8c19f880 con 0x7efe8c104560 2024-02-26T04:55:50.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.558+0000 7efe92155640 1 -- 172.21.15.60:0/1141480910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe8c0fd150 con 0x7efe8c104560 2024-02-26T04:55:50.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.559+0000 7efe88ff9640 1 -- 172.21.15.60:0/1141480910 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7efe800108e0 con 0x7efe8c104560 2024-02-26T04:55:50.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.560+0000 7efe88ff9640 1 --2- 172.21.15.60:0/1141480910 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efe740a20c0 0x7efe740a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:50.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.560+0000 7efe8b7fe640 1 --2- 172.21.15.60:0/1141480910 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efe740a20c0 0x7efe740a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:50.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.560+0000 7efe88ff9640 1 -- 172.21.15.60:0/1141480910 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efe800cdc40 con 0x7efe8c104560 2024-02-26T04:55:50.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.561+0000 7efe8b7fe640 1 --2- 172.21.15.60:0/1141480910 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efe740a20c0 0x7efe740a4580 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efe7c005d00 tx=0x7efe7c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:50.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.565+0000 7efe88ff9640 1 -- 172.21.15.60:0/1141480910 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe800969d0 con 0x7efe8c104560 2024-02-26T04:55:50.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:50 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/92805493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:50.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.747+0000 7efe92155640 1 -- 172.21.15.60:0/1141480910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe8c10cc20 con 0x7efe8c104560 2024-02-26T04:55:50.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.749+0000 7efe88ff9640 1 -- 172.21.15.60:0/1141480910 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24126 (secure 0 0 0) 0x7efe8009a090 con 0x7efe8c104560 2024-02-26T04:55:50.751 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:50.751 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:55:32.581481+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:55:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.751+0000 7efe92155640 1 -- 172.21.15.60:0/1141480910 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efe740a20c0 msgr2=0x7efe740a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.751+0000 7efe92155640 1 --2- 172.21.15.60:0/1141480910 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efe740a20c0 0x7efe740a4580 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efe7c005d00 tx=0x7efe7c00b410 comp rx=0 tx=0).stop 2024-02-26T04:55:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.751+0000 7efe92155640 1 -- 172.21.15.60:0/1141480910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c104560 msgr2=0x7efe8c1977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.751+0000 7efe92155640 1 --2- 172.21.15.60:0/1141480910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c104560 0x7efe8c1977b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efe8000b450 tx=0x7efe8000b910 comp rx=0 tx=0).stop 2024-02-26T04:55:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.751+0000 7efe92155640 1 -- 172.21.15.60:0/1141480910 shutdown_connections 2024-02-26T04:55:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.751+0000 7efe92155640 1 --2- 172.21.15.60:0/1141480910 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efe740a20c0 0x7efe740a4580 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.751+0000 7efe92155640 1 --2- 172.21.15.60:0/1141480910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe8c106e90 0x7efe8c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.751+0000 7efe92155640 1 --2- 172.21.15.60:0/1141480910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c104560 0x7efe8c1977b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.751+0000 7efe92155640 1 --2- 172.21.15.60:0/1141480910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe8c101c30 0x7efe8c197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.751+0000 7efe92155640 1 -- 172.21.15.60:0/1141480910 >> 172.21.15.60:0/1141480910 conn(0x7efe8c0fb660 msgr2=0x7efe8c102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:50.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.751+0000 7efe92155640 1 -- 172.21.15.60:0/1141480910 shutdown_connections 2024-02-26T04:55:50.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.751+0000 7efe92155640 1 -- 172.21.15.60:0/1141480910 wait complete. 2024-02-26T04:55:50.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-26T04:55:50.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.867+0000 7f9797888640 1 -- 172.21.15.60:0/3680087728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9790101280 msgr2=0x7f9790101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:50.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.867+0000 7f9797888640 1 --2- 172.21.15.60:0/3680087728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9790101280 0x7f9790101700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9780009900 tx=0x7f978002bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:50.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.867+0000 7f9797888640 1 -- 172.21.15.60:0/3680087728 shutdown_connections 2024-02-26T04:55:50.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.867+0000 7f9797888640 1 --2- 172.21.15.60:0/3680087728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9790101c40 0x7f97901084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.867+0000 7f9797888640 1 --2- 172.21.15.60:0/3680087728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9790101280 0x7f9790101700 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.867+0000 7f9797888640 1 --2- 172.21.15.60:0/3680087728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9790100090 0x7f97901004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.867+0000 7f9797888640 1 -- 172.21.15.60:0/3680087728 >> 172.21.15.60:0/3680087728 conn(0x7f97900fb620 msgr2=0x7f97900fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:50.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.867+0000 7f9797888640 1 -- 172.21.15.60:0/3680087728 shutdown_connections 2024-02-26T04:55:50.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.867+0000 7f9797888640 1 -- 172.21.15.60:0/3680087728 wait complete. 2024-02-26T04:55:50.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9797888640 1 Processor -- start 2024-02-26T04:55:50.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9797888640 1 -- start start 2024-02-26T04:55:50.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9797888640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9790100090 0x7f97901972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:50.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9797888640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9790101280 0x7f9790197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:50.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9797888640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9790101c40 0x7f979019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:50.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9797888640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979019edb0 con 0x7f9790101c40 2024-02-26T04:55:50.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9797888640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979019ef20 con 0x7f9790100090 2024-02-26T04:55:50.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9797888640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979019f0e0 con 0x7f9790101280 2024-02-26T04:55:50.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9795e25640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9790101c40 0x7f979019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:50.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9795e25640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9790101c40 0x7f979019e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50928/0 (socket says 172.21.15.60:50928) 2024-02-26T04:55:50.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9795e25640 1 -- 172.21.15.60:0/3445375417 learned_addr learned my addr 172.21.15.60:0/3445375417 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:50.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9794e23640 1 --2- 172.21.15.60:0/3445375417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9790101280 0x7f9790197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:50.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9795624640 1 --2- 172.21.15.60:0/3445375417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9790100090 0x7f97901972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:50.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9795e25640 1 -- 172.21.15.60:0/3445375417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9790101280 msgr2=0x7f9790197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:50.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9795e25640 1 --2- 172.21.15.60:0/3445375417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9790101280 0x7f9790197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9795e25640 1 -- 172.21.15.60:0/3445375417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9790100090 msgr2=0x7f97901972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:50.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9795e25640 1 --2- 172.21.15.60:0/3445375417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9790100090 0x7f97901972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:50.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9795e25640 1 -- 172.21.15.60:0/3445375417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9780009560 con 0x7f9790101c40 2024-02-26T04:55:50.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.868+0000 7f9795e25640 1 --2- 172.21.15.60:0/3445375417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9790101c40 0x7f979019e870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f978800cad0 tx=0x7f978800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:50.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.869+0000 7f97867fc640 1 -- 172.21.15.60:0/3445375417 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978800ae20 con 0x7f9790101c40 2024-02-26T04:55:50.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.869+0000 7f97867fc640 1 -- 172.21.15.60:0/3445375417 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9788005020 con 0x7f9790101c40 2024-02-26T04:55:50.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.869+0000 7f9797888640 1 -- 172.21.15.60:0/3445375417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979019f3c0 con 0x7f9790101c40 2024-02-26T04:55:50.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.869+0000 7f97867fc640 1 -- 172.21.15.60:0/3445375417 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978800f6b0 con 0x7f9790101c40 2024-02-26T04:55:50.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.869+0000 7f9797888640 1 -- 172.21.15.60:0/3445375417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979019f850 con 0x7f9790101c40 2024-02-26T04:55:50.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.870+0000 7f9797888640 1 -- 172.21.15.60:0/3445375417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979010baf0 con 0x7f9790101c40 2024-02-26T04:55:50.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.871+0000 7f97867fc640 1 -- 172.21.15.60:0/3445375417 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9788005520 con 0x7f9790101c40 2024-02-26T04:55:50.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.871+0000 7f97867fc640 1 --2- 172.21.15.60:0/3445375417 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f977c0a20c0 0x7f977c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:50.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.871+0000 7f9795624640 1 --2- 172.21.15.60:0/3445375417 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f977c0a20c0 0x7f977c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:50.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.871+0000 7f97867fc640 1 -- 172.21.15.60:0/3445375417 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f97880ce2e0 con 0x7f9790101c40 2024-02-26T04:55:50.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.872+0000 7f9795624640 1 --2- 172.21.15.60:0/3445375417 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f977c0a20c0 0x7f977c0a4580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f978c0058c0 tx=0x7f978c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:50.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:50.875+0000 7f97867fc640 1 -- 172.21.15.60:0/3445375417 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9788097070 con 0x7f9790101c40 2024-02-26T04:55:50.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:50 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/92805493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:51.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:50 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/92805493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:51.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.097+0000 7f9797888640 1 -- 172.21.15.60:0/3445375417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f979010bc80 con 0x7f9790101c40 2024-02-26T04:55:51.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.520+0000 7f97867fc640 1 -- 172.21.15.60:0/3445375417 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f978801d020 con 0x7f9790101c40 2024-02-26T04:55:51.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.523+0000 7f9797888640 1 -- 172.21.15.60:0/3445375417 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f977c0a20c0 msgr2=0x7f977c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:51.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.523+0000 7f9797888640 1 --2- 172.21.15.60:0/3445375417 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f977c0a20c0 0x7f977c0a4580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f978c0058c0 tx=0x7f978c00a4c0 comp rx=0 tx=0).stop 2024-02-26T04:55:51.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.523+0000 7f9797888640 1 -- 172.21.15.60:0/3445375417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9790101c40 msgr2=0x7f979019e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:51.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.523+0000 7f9797888640 1 --2- 172.21.15.60:0/3445375417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9790101c40 0x7f979019e870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f978800cad0 tx=0x7f978800cf90 comp rx=0 tx=0).stop 2024-02-26T04:55:51.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.523+0000 7f9797888640 1 -- 172.21.15.60:0/3445375417 shutdown_connections 2024-02-26T04:55:51.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.523+0000 7f9797888640 1 --2- 172.21.15.60:0/3445375417 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f977c0a20c0 0x7f977c0a4580 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:51.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.523+0000 7f9797888640 1 --2- 172.21.15.60:0/3445375417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9790101c40 0x7f979019e870 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:51.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.523+0000 7f9797888640 1 --2- 172.21.15.60:0/3445375417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9790101280 0x7f9790197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:51.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.523+0000 7f9797888640 1 --2- 172.21.15.60:0/3445375417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9790100090 0x7f97901972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:51.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.523+0000 7f9797888640 1 -- 172.21.15.60:0/3445375417 >> 172.21.15.60:0/3445375417 conn(0x7f97900fb620 msgr2=0x7f97900fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:51.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.523+0000 7f9797888640 1 -- 172.21.15.60:0/3445375417 shutdown_connections 2024-02-26T04:55:51.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.523+0000 7f9797888640 1 -- 172.21.15.60:0/3445375417 wait complete. 2024-02-26T04:55:51.537 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-02-26T04:55:51.537 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-02-26T04:55:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.641+0000 7f8c2f78c640 1 -- 172.21.15.60:0/2890401507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28101280 msgr2=0x7f8c28101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.641+0000 7f8c2f78c640 1 --2- 172.21.15.60:0/2890401507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28101280 0x7f8c28101700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009900 tx=0x7f8c1802bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.641+0000 7f8c2f78c640 1 -- 172.21.15.60:0/2890401507 shutdown_connections 2024-02-26T04:55:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.641+0000 7f8c2f78c640 1 --2- 172.21.15.60:0/2890401507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101c40 0x7f8c281084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.641+0000 7f8c2f78c640 1 --2- 172.21.15.60:0/2890401507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28101280 0x7f8c28101700 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.641+0000 7f8c2f78c640 1 --2- 172.21.15.60:0/2890401507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c28100090 0x7f8c281004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.641+0000 7f8c2f78c640 1 -- 172.21.15.60:0/2890401507 >> 172.21.15.60:0/2890401507 conn(0x7f8c280fb620 msgr2=0x7f8c280fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.642+0000 7f8c2f78c640 1 -- 172.21.15.60:0/2890401507 shutdown_connections 2024-02-26T04:55:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.642+0000 7f8c2f78c640 1 -- 172.21.15.60:0/2890401507 wait complete. 2024-02-26T04:55:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.642+0000 7f8c2f78c640 1 Processor -- start 2024-02-26T04:55:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.642+0000 7f8c2f78c640 1 -- start start 2024-02-26T04:55:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.642+0000 7f8c2f78c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28100090 0x7f8c28197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.642+0000 7f8c2f78c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101280 0x7f8c281977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.642+0000 7f8c2f78c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c28101c40 0x7f8c2819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.642+0000 7f8c2f78c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2819ed80 con 0x7f8c28100090 2024-02-26T04:55:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.642+0000 7f8c2f78c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2819eef0 con 0x7f8c28101280 2024-02-26T04:55:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.642+0000 7f8c2f78c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2819f0b0 con 0x7f8c28101c40 2024-02-26T04:55:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.642+0000 7f8c2d528640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28100090 0x7f8c28197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.642+0000 7f8c2d528640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28100090 0x7f8c28197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50938/0 (socket says 172.21.15.60:50938) 2024-02-26T04:55:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.642+0000 7f8c2d528640 1 -- 172.21.15.60:0/1461378377 learned_addr learned my addr 172.21.15.60:0/1461378377 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c2dd29640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c28101c40 0x7f8c2819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c2cd27640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101280 0x7f8c281977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c2d528640 1 -- 172.21.15.60:0/1461378377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c28101c40 msgr2=0x7f8c2819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c2d528640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c28101c40 0x7f8c2819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c2d528640 1 -- 172.21.15.60:0/1461378377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101280 msgr2=0x7f8c281977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c2d528640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101280 0x7f8c281977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c2d528640 1 -- 172.21.15.60:0/1461378377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c18009560 con 0x7f8c28100090 2024-02-26T04:55:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c2dd29640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c28101c40 0x7f8c2819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c2cd27640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101280 0x7f8c281977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c2d528640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28100090 0x7f8c28197290 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400d950 tx=0x7f8c2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c1e7fc640 1 -- 172.21.15.60:0/1461378377 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24009c30 con 0x7f8c28100090 2024-02-26T04:55:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c1e7fc640 1 -- 172.21.15.60:0/1461378377 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c24009d90 con 0x7f8c28100090 2024-02-26T04:55:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c1e7fc640 1 -- 172.21.15.60:0/1461378377 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2400f720 con 0x7f8c28100090 2024-02-26T04:55:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c2f78c640 1 -- 172.21.15.60:0/1461378377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c2819f390 con 0x7f8c28100090 2024-02-26T04:55:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.643+0000 7f8c2f78c640 1 -- 172.21.15.60:0/1461378377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2819f850 con 0x7f8c28100090 2024-02-26T04:55:51.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.644+0000 7f8c2f78c640 1 -- 172.21.15.60:0/1461378377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2810baa0 con 0x7f8c28100090 2024-02-26T04:55:51.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.646+0000 7f8c1e7fc640 1 -- 172.21.15.60:0/1461378377 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8c24010a60 con 0x7f8c28100090 2024-02-26T04:55:51.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.646+0000 7f8c1e7fc640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8c140a20c0 0x7f8c140a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:51.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.647+0000 7f8c2cd27640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8c140a20c0 0x7f8c140a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:51.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.647+0000 7f8c1e7fc640 1 -- 172.21.15.60:0/1461378377 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8c240cdce0 con 0x7f8c28100090 2024-02-26T04:55:51.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.647+0000 7f8c2cd27640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8c140a20c0 0x7f8c140a4580 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8c18036040 tx=0x7f8c18000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:51.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.650+0000 7f8c1e7fc640 1 -- 172.21.15.60:0/1461378377 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c24096a10 con 0x7f8c28100090 2024-02-26T04:55:51.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:51.891+0000 7f8c2f78c640 1 -- 172.21.15.60:0/1461378377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f8c2819fa70 con 0x7f8c28100090 2024-02-26T04:55:51.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:51 smithi156 ceph-mon[35805]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-26T04:55:51.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:51 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1141480910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:51.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:51 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3445375417' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T04:55:51.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:51 smithi060 ceph-mon[32382]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-26T04:55:51.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:51 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1141480910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:51.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:51 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3445375417' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T04:55:52.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:51 smithi191 ceph-mon[36352]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-26T04:55:52.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:51 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1141480910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:52.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:51 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3445375417' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-26T04:55:52.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.526+0000 7f8c1e7fc640 1 -- 172.21.15.60:0/1461378377 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f8c240170b0 con 0x7f8c28100090 2024-02-26T04:55:52.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.529+0000 7f8c2f78c640 1 -- 172.21.15.60:0/1461378377 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8c140a20c0 msgr2=0x7f8c140a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:52.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.529+0000 7f8c2f78c640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8c140a20c0 0x7f8c140a4580 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8c18036040 tx=0x7f8c18000bc0 comp rx=0 tx=0).stop 2024-02-26T04:55:52.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.529+0000 7f8c2f78c640 1 -- 172.21.15.60:0/1461378377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28100090 msgr2=0x7f8c28197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:52.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.529+0000 7f8c2f78c640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28100090 0x7f8c28197290 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400d950 tx=0x7f8c2400de10 comp rx=0 tx=0).stop 2024-02-26T04:55:52.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.529+0000 7f8c2f78c640 1 -- 172.21.15.60:0/1461378377 shutdown_connections 2024-02-26T04:55:52.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.529+0000 7f8c2f78c640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8c140a20c0 0x7f8c140a4580 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:52.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.529+0000 7f8c2f78c640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c28101c40 0x7f8c2819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:52.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.529+0000 7f8c2f78c640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101280 0x7f8c281977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:52.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.529+0000 7f8c2f78c640 1 --2- 172.21.15.60:0/1461378377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28100090 0x7f8c28197290 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:52.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.529+0000 7f8c2f78c640 1 -- 172.21.15.60:0/1461378377 >> 172.21.15.60:0/1461378377 conn(0x7f8c280fb620 msgr2=0x7f8c28069750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:52.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.529+0000 7f8c2f78c640 1 -- 172.21.15.60:0/1461378377 shutdown_connections 2024-02-26T04:55:52.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.529+0000 7f8c2f78c640 1 -- 172.21.15.60:0/1461378377 wait complete. 2024-02-26T04:55:52.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-02-26T04:55:52.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.657+0000 7efeddf08640 1 -- 172.21.15.60:0/1955283911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efed8101630 msgr2=0x7efed8101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:52.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.657+0000 7efeddf08640 1 --2- 172.21.15.60:0/1955283911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efed8101630 0x7efed8101a50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efec8009930 tx=0x7efec802bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:52.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.657+0000 7efeddf08640 1 -- 172.21.15.60:0/1955283911 shutdown_connections 2024-02-26T04:55:52.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.657+0000 7efeddf08640 1 --2- 172.21.15.60:0/1955283911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efed8100d20 0x7efed81085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:52.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.657+0000 7efeddf08640 1 --2- 172.21.15.60:0/1955283911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed8100360 0x7efed81007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:52.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.657+0000 7efeddf08640 1 --2- 172.21.15.60:0/1955283911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efed8101630 0x7efed8101a50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:52.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.657+0000 7efeddf08640 1 -- 172.21.15.60:0/1955283911 >> 172.21.15.60:0/1955283911 conn(0x7efed80fadb0 msgr2=0x7efed80fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:52.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.657+0000 7efeddf08640 1 -- 172.21.15.60:0/1955283911 shutdown_connections 2024-02-26T04:55:52.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.657+0000 7efeddf08640 1 -- 172.21.15.60:0/1955283911 wait complete. 2024-02-26T04:55:52.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.658+0000 7efeddf08640 1 Processor -- start 2024-02-26T04:55:52.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.658+0000 7efeddf08640 1 -- start start 2024-02-26T04:55:52.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.658+0000 7efeddf08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efed8100360 0x7efed8104110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:52.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.658+0000 7efeddf08640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efed8100d20 0x7efed8102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:52.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.658+0000 7efeddf08640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed8101630 0x7efed8102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:52.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.658+0000 7efeddf08640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed81033f0 con 0x7efed8100360 2024-02-26T04:55:52.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.658+0000 7efeddf08640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed8103560 con 0x7efed8100d20 2024-02-26T04:55:52.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.658+0000 7efeddf08640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed81a12e0 con 0x7efed8101630 2024-02-26T04:55:52.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.658+0000 7efed77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efed8100360 0x7efed8104110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:52.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.658+0000 7efed77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efed8100360 0x7efed8104110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50956/0 (socket says 172.21.15.60:50956) 2024-02-26T04:55:52.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.658+0000 7efed77fe640 1 -- 172.21.15.60:0/2191051244 learned_addr learned my addr 172.21.15.60:0/2191051244 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:52.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.659+0000 7efed7fff640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed8101630 0x7efed8102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:52.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.659+0000 7efed6ffd640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efed8100d20 0x7efed8102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:52.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.659+0000 7efed77fe640 1 -- 172.21.15.60:0/2191051244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed8101630 msgr2=0x7efed8102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:52.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.659+0000 7efed77fe640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed8101630 0x7efed8102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:52.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.659+0000 7efed77fe640 1 -- 172.21.15.60:0/2191051244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efed8100d20 msgr2=0x7efed8102750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:52.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.659+0000 7efed77fe640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efed8100d20 0x7efed8102750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:52.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.659+0000 7efed77fe640 1 -- 172.21.15.60:0/2191051244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec8009560 con 0x7efed8100360 2024-02-26T04:55:52.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.659+0000 7efed6ffd640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efed8100d20 0x7efed8102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:52.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.659+0000 7efed77fe640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efed8100360 0x7efed8104110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efec8009230 tx=0x7efec8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:52.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.659+0000 7efed4ff9640 1 -- 172.21.15.60:0/2191051244 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8004df0 con 0x7efed8100360 2024-02-26T04:55:52.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.659+0000 7efed4ff9640 1 -- 172.21.15.60:0/2191051244 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efec8005430 con 0x7efed8100360 2024-02-26T04:55:52.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.659+0000 7efeddf08640 1 -- 172.21.15.60:0/2191051244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed81a1530 con 0x7efed8100360 2024-02-26T04:55:52.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.659+0000 7efed4ff9640 1 -- 172.21.15.60:0/2191051244 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec803e450 con 0x7efed8100360 2024-02-26T04:55:52.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.659+0000 7efeddf08640 1 -- 172.21.15.60:0/2191051244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed81a1a40 con 0x7efed8100360 2024-02-26T04:55:52.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.660+0000 7efeddf08640 1 -- 172.21.15.60:0/2191051244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed810bad0 con 0x7efed8100360 2024-02-26T04:55:52.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.661+0000 7efed4ff9640 1 -- 172.21.15.60:0/2191051244 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7efec80035f0 con 0x7efed8100360 2024-02-26T04:55:52.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.662+0000 7efed4ff9640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efec00a20c0 0x7efec00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:52.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.662+0000 7efed6ffd640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efec00a20c0 0x7efec00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:52.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.662+0000 7efed4ff9640 1 -- 172.21.15.60:0/2191051244 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efec80f0260 con 0x7efed8100360 2024-02-26T04:55:52.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.662+0000 7efed6ffd640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efec00a20c0 0x7efec00a4580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efed8103e80 tx=0x7efecc005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:52.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.666+0000 7efed4ff9640 1 -- 172.21.15.60:0/2191051244 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efec80b8f90 con 0x7efed8100360 2024-02-26T04:55:52.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:52.888+0000 7efeddf08640 1 -- 172.21.15.60:0/2191051244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7efed810bcb0 con 0x7efed8100360 2024-02-26T04:55:52.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:52 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3445375417' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-26T04:55:52.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:52 smithi156 ceph-mon[35805]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T04:55:52.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:52 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1461378377' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T04:55:52.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:52 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3445375417' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-26T04:55:52.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:52 smithi060 ceph-mon[32382]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T04:55:52.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:52 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1461378377' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T04:55:53.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:52 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3445375417' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-26T04:55:53.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:52 smithi191 ceph-mon[36352]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T04:55:53.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:52 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1461378377' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-26T04:55:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.533+0000 7efed4ff9640 1 -- 172.21.15.60:0/2191051244 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7efec80bc650 con 0x7efed8100360 2024-02-26T04:55:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.536+0000 7efeddf08640 1 -- 172.21.15.60:0/2191051244 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efec00a20c0 msgr2=0x7efec00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.536+0000 7efeddf08640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efec00a20c0 0x7efec00a4580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efed8103e80 tx=0x7efecc005d30 comp rx=0 tx=0).stop 2024-02-26T04:55:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.536+0000 7efeddf08640 1 -- 172.21.15.60:0/2191051244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efed8100360 msgr2=0x7efed8104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.536+0000 7efeddf08640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efed8100360 0x7efed8104110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efec8009230 tx=0x7efec8004270 comp rx=0 tx=0).stop 2024-02-26T04:55:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.536+0000 7efeddf08640 1 -- 172.21.15.60:0/2191051244 shutdown_connections 2024-02-26T04:55:53.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.536+0000 7efeddf08640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efec00a20c0 0x7efec00a4580 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:53.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.536+0000 7efeddf08640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed8101630 0x7efed8102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:53.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.536+0000 7efeddf08640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efed8100d20 0x7efed8102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:53.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.536+0000 7efeddf08640 1 --2- 172.21.15.60:0/2191051244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efed8100360 0x7efed8104110 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:53.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.536+0000 7efeddf08640 1 -- 172.21.15.60:0/2191051244 >> 172.21.15.60:0/2191051244 conn(0x7efed80fadb0 msgr2=0x7efed80fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:53.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.536+0000 7efeddf08640 1 -- 172.21.15.60:0/2191051244 shutdown_connections 2024-02-26T04:55:53.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.536+0000 7efeddf08640 1 -- 172.21.15.60:0/2191051244 wait complete. 2024-02-26T04:55:53.556 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-02-26T04:55:53.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.666+0000 7faa65d1e640 1 -- 172.21.15.60:0/518132426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa601000a0 msgr2=0x7faa601004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.666+0000 7faa65d1e640 1 --2- 172.21.15.60:0/518132426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa601000a0 0x7faa601004c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faa50009900 tx=0x7faa5002bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.666+0000 7faa65d1e640 1 -- 172.21.15.60:0/518132426 shutdown_connections 2024-02-26T04:55:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.666+0000 7faa65d1e640 1 --2- 172.21.15.60:0/518132426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa60101c50 0x7faa601084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.666+0000 7faa65d1e640 1 --2- 172.21.15.60:0/518132426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa60101290 0x7faa60101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.666+0000 7faa65d1e640 1 --2- 172.21.15.60:0/518132426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa601000a0 0x7faa601004c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.666+0000 7faa65d1e640 1 -- 172.21.15.60:0/518132426 >> 172.21.15.60:0/518132426 conn(0x7faa600fb610 msgr2=0x7faa600fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.666+0000 7faa65d1e640 1 -- 172.21.15.60:0/518132426 shutdown_connections 2024-02-26T04:55:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.666+0000 7faa65d1e640 1 -- 172.21.15.60:0/518132426 wait complete. 2024-02-26T04:55:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.666+0000 7faa65d1e640 1 Processor -- start 2024-02-26T04:55:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa65d1e640 1 -- start start 2024-02-26T04:55:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa65d1e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa601000a0 0x7faa6006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa65d1e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa60101290 0x7faa6019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa65d1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa60101c50 0x7faa6019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa65d1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6019eda0 con 0x7faa60101c50 2024-02-26T04:55:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa65d1e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6019ef10 con 0x7faa601000a0 2024-02-26T04:55:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa65d1e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6019f0d0 con 0x7faa60101290 2024-02-26T04:55:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa5ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa60101c50 0x7faa6019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa5ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa60101c50 0x7faa6019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50966/0 (socket says 172.21.15.60:50966) 2024-02-26T04:55:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa5ffff640 1 -- 172.21.15.60:0/2525899410 learned_addr learned my addr 172.21.15.60:0/2525899410 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa5effd640 1 --2- 172.21.15.60:0/2525899410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa60101290 0x7faa6019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa5f7fe640 1 --2- 172.21.15.60:0/2525899410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa601000a0 0x7faa6006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa5ffff640 1 -- 172.21.15.60:0/2525899410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa60101290 msgr2=0x7faa6019bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa5ffff640 1 --2- 172.21.15.60:0/2525899410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa60101290 0x7faa6019bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa5ffff640 1 -- 172.21.15.60:0/2525899410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa601000a0 msgr2=0x7faa6006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa5ffff640 1 --2- 172.21.15.60:0/2525899410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa601000a0 0x7faa6006a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa5ffff640 1 -- 172.21.15.60:0/2525899410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa50009560 con 0x7faa60101c50 2024-02-26T04:55:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.667+0000 7faa5ffff640 1 --2- 172.21.15.60:0/2525899410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa60101c50 0x7faa6019e5c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faa5800cad0 tx=0x7faa5800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:53.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.668+0000 7faa5cff9640 1 -- 172.21.15.60:0/2525899410 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5800ae20 con 0x7faa60101c50 2024-02-26T04:55:53.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.668+0000 7faa65d1e640 1 -- 172.21.15.60:0/2525899410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6019f350 con 0x7faa60101c50 2024-02-26T04:55:53.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.668+0000 7faa5cff9640 1 -- 172.21.15.60:0/2525899410 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa58005020 con 0x7faa60101c50 2024-02-26T04:55:53.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.668+0000 7faa5cff9640 1 -- 172.21.15.60:0/2525899410 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5800f6b0 con 0x7faa60101c50 2024-02-26T04:55:53.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.668+0000 7faa65d1e640 1 -- 172.21.15.60:0/2525899410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6019f810 con 0x7faa60101c50 2024-02-26T04:55:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.669+0000 7faa65d1e640 1 -- 172.21.15.60:0/2525899410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa601073e0 con 0x7faa60101c50 2024-02-26T04:55:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.670+0000 7faa5cff9640 1 -- 172.21.15.60:0/2525899410 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7faa58005520 con 0x7faa60101c50 2024-02-26T04:55:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.670+0000 7faa5cff9640 1 --2- 172.21.15.60:0/2525899410 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7faa480a20c0 0x7faa480a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.670+0000 7faa5f7fe640 1 --2- 172.21.15.60:0/2525899410 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7faa480a20c0 0x7faa480a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.670+0000 7faa5cff9640 1 -- 172.21.15.60:0/2525899410 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7faa580cdef0 con 0x7faa60101c50 2024-02-26T04:55:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.671+0000 7faa5f7fe640 1 --2- 172.21.15.60:0/2525899410 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7faa480a20c0 0x7faa480a4580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faa50006010 tx=0x7faa50000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:53.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.675+0000 7faa5cff9640 1 -- 172.21.15.60:0/2525899410 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa58096c80 con 0x7faa60101c50 2024-02-26T04:55:53.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:53.896+0000 7faa65d1e640 1 -- 172.21.15.60:0/2525899410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7faa6002cea0 con 0x7faa60101c50 2024-02-26T04:55:53.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:53 smithi156 ceph-mon[35805]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T04:55:53.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:53 smithi156 ceph-mon[35805]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-26T04:55:53.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:53 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1461378377' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-26T04:55:53.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:53 smithi156 ceph-mon[35805]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-26T04:55:53.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:53 smithi156 ceph-mon[35805]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T04:55:53.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:53 smithi156 ceph-mon[35805]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-02-26T04:55:53.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:53 smithi156 ceph-mon[35805]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-02-26T04:55:53.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:53 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2191051244' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T04:55:53.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:53 smithi060 ceph-mon[32382]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T04:55:53.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:53 smithi060 ceph-mon[32382]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-26T04:55:53.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:53 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1461378377' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-26T04:55:53.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:53 smithi060 ceph-mon[32382]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-26T04:55:53.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:53 smithi060 ceph-mon[32382]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T04:55:53.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:53 smithi060 ceph-mon[32382]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-02-26T04:55:53.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:53 smithi060 ceph-mon[32382]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-02-26T04:55:53.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:53 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2191051244' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T04:55:54.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:53 smithi191 ceph-mon[36352]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-26T04:55:54.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:53 smithi191 ceph-mon[36352]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-26T04:55:54.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:53 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1461378377' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-26T04:55:54.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:53 smithi191 ceph-mon[36352]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-26T04:55:54.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:53 smithi191 ceph-mon[36352]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-26T04:55:54.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:53 smithi191 ceph-mon[36352]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-02-26T04:55:54.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:53 smithi191 ceph-mon[36352]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-02-26T04:55:54.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:53 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2191051244' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-26T04:55:54.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.141+0000 7faa5cff9640 1 -- 172.21.15.60:0/2525899410 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7faa5809a340 con 0x7faa60101c50 2024-02-26T04:55:54.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.143+0000 7faa65d1e640 1 -- 172.21.15.60:0/2525899410 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7faa480a20c0 msgr2=0x7faa480a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.143+0000 7faa65d1e640 1 --2- 172.21.15.60:0/2525899410 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7faa480a20c0 0x7faa480a4580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faa50006010 tx=0x7faa50000bc0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.143+0000 7faa65d1e640 1 -- 172.21.15.60:0/2525899410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa60101c50 msgr2=0x7faa6019e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.143+0000 7faa65d1e640 1 --2- 172.21.15.60:0/2525899410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa60101c50 0x7faa6019e5c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faa5800cad0 tx=0x7faa5800cf90 comp rx=0 tx=0).stop 2024-02-26T04:55:54.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.143+0000 7faa65d1e640 1 -- 172.21.15.60:0/2525899410 shutdown_connections 2024-02-26T04:55:54.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.143+0000 7faa65d1e640 1 --2- 172.21.15.60:0/2525899410 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7faa480a20c0 0x7faa480a4580 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.143+0000 7faa65d1e640 1 --2- 172.21.15.60:0/2525899410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa60101c50 0x7faa6019e5c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.143+0000 7faa65d1e640 1 --2- 172.21.15.60:0/2525899410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa60101290 0x7faa6019bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.143+0000 7faa65d1e640 1 --2- 172.21.15.60:0/2525899410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa601000a0 0x7faa6006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.143+0000 7faa65d1e640 1 -- 172.21.15.60:0/2525899410 >> 172.21.15.60:0/2525899410 conn(0x7faa600fb610 msgr2=0x7faa601045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:54.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.143+0000 7faa65d1e640 1 -- 172.21.15.60:0/2525899410 shutdown_connections 2024-02-26T04:55:54.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.143+0000 7faa65d1e640 1 -- 172.21.15.60:0/2525899410 wait complete. 2024-02-26T04:55:54.157 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-02-26T04:55:54.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-02-26T04:55:54.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.262+0000 7f3ab4b64640 1 -- 172.21.15.60:0/3766327458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab01000a0 msgr2=0x7f3ab01004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.262+0000 7f3ab4b64640 1 --2- 172.21.15.60:0/3766327458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab01000a0 0x7f3ab01004c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009900 tx=0x7f3aa002bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:54.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.262+0000 7f3ab4b64640 1 -- 172.21.15.60:0/3766327458 shutdown_connections 2024-02-26T04:55:54.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.262+0000 7f3ab4b64640 1 --2- 172.21.15.60:0/3766327458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ab0101c50 0x7f3ab01084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.262+0000 7f3ab4b64640 1 --2- 172.21.15.60:0/3766327458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab0101290 0x7f3ab0101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.262+0000 7f3ab4b64640 1 --2- 172.21.15.60:0/3766327458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab01000a0 0x7f3ab01004c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.262+0000 7f3ab4b64640 1 -- 172.21.15.60:0/3766327458 >> 172.21.15.60:0/3766327458 conn(0x7f3ab00fb630 msgr2=0x7f3ab00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:54.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.262+0000 7f3ab4b64640 1 -- 172.21.15.60:0/3766327458 shutdown_connections 2024-02-26T04:55:54.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.262+0000 7f3ab4b64640 1 -- 172.21.15.60:0/3766327458 wait complete. 2024-02-26T04:55:54.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.262+0000 7f3ab4b64640 1 Processor -- start 2024-02-26T04:55:54.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.262+0000 7f3ab4b64640 1 -- start start 2024-02-26T04:55:54.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3ab4b64640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ab01000a0 0x7f3ab0197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:54.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3ab4b64640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab0101290 0x7f3ab0197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:54.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3ab4b64640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab0101c50 0x7f3ab019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:54.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3ab4b64640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab019ed40 con 0x7f3ab0101c50 2024-02-26T04:55:54.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3ab4b64640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab019eeb0 con 0x7f3ab0101290 2024-02-26T04:55:54.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3ab4b64640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab019f070 con 0x7f3ab01000a0 2024-02-26T04:55:54.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aaed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab0101c50 0x7f3ab019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:54.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aaed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab0101c50 0x7f3ab019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50984/0 (socket says 172.21.15.60:50984) 2024-02-26T04:55:54.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aaed9d640 1 -- 172.21.15.60:0/128871874 learned_addr learned my addr 172.21.15.60:0/128871874 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:54.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aa7fff640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab0101290 0x7f3ab0197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:54.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aae59c640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ab01000a0 0x7f3ab0197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:54.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aaed9d640 1 -- 172.21.15.60:0/128871874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ab01000a0 msgr2=0x7f3ab0197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aaed9d640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ab01000a0 0x7f3ab0197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aaed9d640 1 -- 172.21.15.60:0/128871874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab0101290 msgr2=0x7f3ab0197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aaed9d640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab0101290 0x7f3ab0197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aaed9d640 1 -- 172.21.15.60:0/128871874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa0009560 con 0x7f3ab0101c50 2024-02-26T04:55:54.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aae59c640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ab01000a0 0x7f3ab0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:54.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aaed9d640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab0101c50 0x7f3ab019e800 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800eaa0 tx=0x7f3aa800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:54.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aa77fe640 1 -- 172.21.15.60:0/128871874 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa800cdb0 con 0x7f3ab0101c50 2024-02-26T04:55:54.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aa77fe640 1 -- 172.21.15.60:0/128871874 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3aa8005020 con 0x7f3ab0101c50 2024-02-26T04:55:54.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3ab4b64640 1 -- 172.21.15.60:0/128871874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab019f350 con 0x7f3ab0101c50 2024-02-26T04:55:54.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3aa77fe640 1 -- 172.21.15.60:0/128871874 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80116b0 con 0x7f3ab0101c50 2024-02-26T04:55:54.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.263+0000 7f3ab4b64640 1 -- 172.21.15.60:0/128871874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab019f860 con 0x7f3ab0101c50 2024-02-26T04:55:54.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.264+0000 7f3ab4b64640 1 -- 172.21.15.60:0/128871874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab0107410 con 0x7f3ab0101c50 2024-02-26T04:55:54.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.265+0000 7f3aa77fe640 1 -- 172.21.15.60:0/128871874 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3aa8005520 con 0x7f3ab0101c50 2024-02-26T04:55:54.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.266+0000 7f3aa77fe640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3a9c0a20c0 0x7f3a9c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:54.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.266+0000 7f3aae59c640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3a9c0a20c0 0x7f3a9c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:54.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.266+0000 7f3aa77fe640 1 -- 172.21.15.60:0/128871874 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f3aa80ce160 con 0x7f3ab0101c50 2024-02-26T04:55:54.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.267+0000 7f3aae59c640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3a9c0a20c0 0x7f3a9c0a4580 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0006010 tx=0x7f3aa0005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:54.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.271+0000 7f3aa77fe640 1 -- 172.21.15.60:0/128871874 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3aa8096e90 con 0x7f3ab0101c50 2024-02-26T04:55:54.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.478+0000 7f3ab4b64640 1 -- 172.21.15.60:0/128871874 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ab002ceb0 con 0x7f3a9c0a20c0 2024-02-26T04:55:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.500+0000 7f3aa77fe640 1 -- 172.21.15.60:0/128871874 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3ab002ceb0 con 0x7f3a9c0a20c0 2024-02-26T04:55:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.502+0000 7f3ab4b64640 1 -- 172.21.15.60:0/128871874 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3a9c0a20c0 msgr2=0x7f3a9c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.502+0000 7f3ab4b64640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3a9c0a20c0 0x7f3a9c0a4580 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0006010 tx=0x7f3aa0005be0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.502+0000 7f3ab4b64640 1 -- 172.21.15.60:0/128871874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab0101c50 msgr2=0x7f3ab019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.502+0000 7f3ab4b64640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab0101c50 0x7f3ab019e800 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800eaa0 tx=0x7f3aa800ef60 comp rx=0 tx=0).stop 2024-02-26T04:55:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.502+0000 7f3ab4b64640 1 -- 172.21.15.60:0/128871874 shutdown_connections 2024-02-26T04:55:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.502+0000 7f3ab4b64640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f3a9c0a20c0 0x7f3a9c0a4580 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.502+0000 7f3ab4b64640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab0101c50 0x7f3ab019e800 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.502+0000 7f3ab4b64640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab0101290 0x7f3ab0197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.502+0000 7f3ab4b64640 1 --2- 172.21.15.60:0/128871874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ab01000a0 0x7f3ab0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.502+0000 7f3ab4b64640 1 -- 172.21.15.60:0/128871874 >> 172.21.15.60:0/128871874 conn(0x7f3ab00fb630 msgr2=0x7f3ab01045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.502+0000 7f3ab4b64640 1 -- 172.21.15.60:0/128871874 shutdown_connections 2024-02-26T04:55:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.502+0000 7f3ab4b64640 1 -- 172.21.15.60:0/128871874 wait complete. 2024-02-26T04:55:54.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-02-26T04:55:54.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.623+0000 7f4a24d9f640 1 -- 172.21.15.60:0/1749294098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a20104410 msgr2=0x7f4a20106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.623+0000 7f4a24d9f640 1 --2- 172.21.15.60:0/1749294098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a20104410 0x7f4a20106800 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009900 tx=0x7f4a1402bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:54.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.624+0000 7f4a24d9f640 1 -- 172.21.15.60:0/1749294098 shutdown_connections 2024-02-26T04:55:54.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.624+0000 7f4a24d9f640 1 --2- 172.21.15.60:0/1749294098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a20106d40 0x7f4a20109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.624+0000 7f4a24d9f640 1 --2- 172.21.15.60:0/1749294098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a20104410 0x7f4a20106800 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.624+0000 7f4a24d9f640 1 --2- 172.21.15.60:0/1749294098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a20101ae0 0x7f4a20103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.624+0000 7f4a24d9f640 1 -- 172.21.15.60:0/1749294098 >> 172.21.15.60:0/1749294098 conn(0x7f4a200fb4f0 msgr2=0x7f4a200fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.624+0000 7f4a24d9f640 1 -- 172.21.15.60:0/1749294098 shutdown_connections 2024-02-26T04:55:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.624+0000 7f4a24d9f640 1 -- 172.21.15.60:0/1749294098 wait complete. 2024-02-26T04:55:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.624+0000 7f4a24d9f640 1 Processor -- start 2024-02-26T04:55:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.624+0000 7f4a24d9f640 1 -- start start 2024-02-26T04:55:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a24d9f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a20101ae0 0x7f4a20197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a24d9f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a20104410 0x7f4a201977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a24d9f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a20106d40 0x7f4a2019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a24d9f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2019ed60 con 0x7f4a20101ae0 2024-02-26T04:55:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a24d9f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2019eed0 con 0x7f4a20106d40 2024-02-26T04:55:54.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a24d9f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2019f090 con 0x7f4a20104410 2024-02-26T04:55:54.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a1f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a20101ae0 0x7f4a20197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:54.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a1f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a20101ae0 0x7f4a20197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51006/0 (socket says 172.21.15.60:51006) 2024-02-26T04:55:54.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a1f7fe640 1 -- 172.21.15.60:0/45878528 learned_addr learned my addr 172.21.15.60:0/45878528 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:54.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a1effd640 1 --2- 172.21.15.60:0/45878528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a20104410 0x7f4a201977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:54.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a1ffff640 1 --2- 172.21.15.60:0/45878528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a20106d40 0x7f4a2019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:54.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a1f7fe640 1 -- 172.21.15.60:0/45878528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a20104410 msgr2=0x7f4a201977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a1f7fe640 1 --2- 172.21.15.60:0/45878528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a20104410 0x7f4a201977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a1f7fe640 1 -- 172.21.15.60:0/45878528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a20106d40 msgr2=0x7f4a2019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a1f7fe640 1 --2- 172.21.15.60:0/45878528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a20106d40 0x7f4a2019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a1f7fe640 1 -- 172.21.15.60:0/45878528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a14009560 con 0x7f4a20101ae0 2024-02-26T04:55:54.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a1f7fe640 1 --2- 172.21.15.60:0/45878528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a20101ae0 0x7f4a20197270 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000b4a0 tx=0x7f4a1000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:54.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.625+0000 7f4a1cff9640 1 -- 172.21.15.60:0/45878528 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10010610 con 0x7f4a20101ae0 2024-02-26T04:55:54.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.626+0000 7f4a24d9f640 1 -- 172.21.15.60:0/45878528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2019f370 con 0x7f4a20101ae0 2024-02-26T04:55:54.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.626+0000 7f4a1cff9640 1 -- 172.21.15.60:0/45878528 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a10010c50 con 0x7f4a20101ae0 2024-02-26T04:55:54.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.626+0000 7f4a1cff9640 1 -- 172.21.15.60:0/45878528 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10017e20 con 0x7f4a20101ae0 2024-02-26T04:55:54.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.626+0000 7f4a24d9f640 1 -- 172.21.15.60:0/45878528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a2019f880 con 0x7f4a20101ae0 2024-02-26T04:55:54.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.627+0000 7f4a24d9f640 1 -- 172.21.15.60:0/45878528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a200fcfe0 con 0x7f4a20101ae0 2024-02-26T04:55:54.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.628+0000 7f4a1cff9640 1 -- 172.21.15.60:0/45878528 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f4a10017480 con 0x7f4a20101ae0 2024-02-26T04:55:54.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.628+0000 7f4a1cff9640 1 --2- 172.21.15.60:0/45878528 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4a080a20c0 0x7f4a080a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:54.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.628+0000 7f4a1cff9640 1 -- 172.21.15.60:0/45878528 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4a100cd490 con 0x7f4a20101ae0 2024-02-26T04:55:54.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.628+0000 7f4a1effd640 1 --2- 172.21.15.60:0/45878528 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4a080a20c0 0x7f4a080a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:54.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.629+0000 7f4a1effd640 1 --2- 172.21.15.60:0/45878528 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4a080a20c0 0x7f4a080a4580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009230 tx=0x7f4a14005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:54.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.633+0000 7f4a1cff9640 1 -- 172.21.15.60:0/45878528 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a100962d0 con 0x7f4a20101ae0 2024-02-26T04:55:54.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.842+0000 7f4a24d9f640 1 -- 172.21.15.60:0/45878528 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a200615c0 con 0x7f4a080a20c0 2024-02-26T04:55:54.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.867+0000 7f4a1cff9640 1 -- 172.21.15.60:0/45878528 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4a200615c0 con 0x7f4a080a20c0 2024-02-26T04:55:54.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.868+0000 7f4a24d9f640 1 -- 172.21.15.60:0/45878528 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4a080a20c0 msgr2=0x7f4a080a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.868+0000 7f4a24d9f640 1 --2- 172.21.15.60:0/45878528 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4a080a20c0 0x7f4a080a4580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009230 tx=0x7f4a14005fd0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.868+0000 7f4a24d9f640 1 -- 172.21.15.60:0/45878528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a20101ae0 msgr2=0x7f4a20197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.868+0000 7f4a24d9f640 1 --2- 172.21.15.60:0/45878528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a20101ae0 0x7f4a20197270 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000b4a0 tx=0x7f4a1000b960 comp rx=0 tx=0).stop 2024-02-26T04:55:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.869+0000 7f4a24d9f640 1 -- 172.21.15.60:0/45878528 shutdown_connections 2024-02-26T04:55:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.869+0000 7f4a24d9f640 1 --2- 172.21.15.60:0/45878528 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4a080a20c0 0x7f4a080a4580 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.869+0000 7f4a24d9f640 1 --2- 172.21.15.60:0/45878528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a20106d40 0x7f4a2019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.869+0000 7f4a24d9f640 1 --2- 172.21.15.60:0/45878528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a20104410 0x7f4a201977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.869+0000 7f4a24d9f640 1 --2- 172.21.15.60:0/45878528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a20101ae0 0x7f4a20197270 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.869+0000 7f4a24d9f640 1 -- 172.21.15.60:0/45878528 >> 172.21.15.60:0/45878528 conn(0x7f4a200fb4f0 msgr2=0x7f4a20102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.869+0000 7f4a24d9f640 1 -- 172.21.15.60:0/45878528 shutdown_connections 2024-02-26T04:55:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.869+0000 7f4a24d9f640 1 -- 172.21.15.60:0/45878528 wait complete. 2024-02-26T04:55:54.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2191051244' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-26T04:55:54.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] up:active 2024-02-26T04:55:54.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-mon[32382]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-26T04:55:54.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-mon[32382]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-26T04:55:54.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-mon[32382]: Cluster is now healthy 2024-02-26T04:55:54.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-mon[32382]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-02-26T04:55:54.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-mon[32382]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-02-26T04:55:54.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-mon[32382]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-02-26T04:55:54.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2525899410' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T04:55:54.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2525899410' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-26T04:55:54.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] up:active 2024-02-26T04:55:54.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-mon[32382]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T04:55:54.882 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:55:54.482+0000 7fc812a14700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T04:55:54.882 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:55:54.482+0000 7fc812a14700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T04:55:54.882 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:55:54.482+0000 7fc812a14700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T04:55:54.883 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:55:54.482+0000 7fc812a14700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T04:55:54.883 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 04:55:54 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T04:55:54.482+0000 7fc812a14700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-26T04:55:54.884 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-02-26T04:55:54.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:54 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2191051244' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-26T04:55:54.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:54 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] up:active 2024-02-26T04:55:54.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:54 smithi156 ceph-mon[35805]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-26T04:55:54.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:54 smithi156 ceph-mon[35805]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-26T04:55:54.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:54 smithi156 ceph-mon[35805]: Cluster is now healthy 2024-02-26T04:55:54.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:54 smithi156 ceph-mon[35805]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-02-26T04:55:54.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:54 smithi156 ceph-mon[35805]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-02-26T04:55:54.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:54 smithi156 ceph-mon[35805]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-02-26T04:55:54.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:54 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2525899410' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T04:55:54.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:54 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2525899410' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-26T04:55:54.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:54 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] up:active 2024-02-26T04:55:54.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:54 smithi156 ceph-mon[35805]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T04:55:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.990+0000 7f4c2aa98640 1 -- 172.21.15.60:0/1631962357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24102380 msgr2=0x7f4c24102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.990+0000 7f4c2aa98640 1 --2- 172.21.15.60:0/1631962357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24102380 0x7f4c24102800 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009900 tx=0x7f4c1002bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.990+0000 7f4c2aa98640 1 -- 172.21.15.60:0/1631962357 shutdown_connections 2024-02-26T04:55:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.990+0000 7f4c2aa98640 1 --2- 172.21.15.60:0/1631962357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c24102d40 0x7f4c241095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.990+0000 7f4c2aa98640 1 --2- 172.21.15.60:0/1631962357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24102380 0x7f4c24102800 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.990+0000 7f4c2aa98640 1 --2- 172.21.15.60:0/1631962357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c24101190 0x7f4c241015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.990+0000 7f4c2aa98640 1 -- 172.21.15.60:0/1631962357 >> 172.21.15.60:0/1631962357 conn(0x7f4c240fc720 msgr2=0x7f4c240feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:54.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.990+0000 7f4c2aa98640 1 -- 172.21.15.60:0/1631962357 shutdown_connections 2024-02-26T04:55:54.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.990+0000 7f4c2aa98640 1 -- 172.21.15.60:0/1631962357 wait complete. 2024-02-26T04:55:54.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.990+0000 7f4c2aa98640 1 Processor -- start 2024-02-26T04:55:54.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c2aa98640 1 -- start start 2024-02-26T04:55:54.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c2aa98640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24101190 0x7f4c241994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:54.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c2aa98640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c24102380 0x7f4c241999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:54.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c2aa98640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c24102d40 0x7f4c241a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:54.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c2aa98640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241a0f90 con 0x7f4c24101190 2024-02-26T04:55:54.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c2aa98640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241a1100 con 0x7f4c24102d40 2024-02-26T04:55:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c2aa98640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241a12c0 con 0x7f4c24102380 2024-02-26T04:55:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c28834640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24101190 0x7f4c241994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c28834640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24101190 0x7f4c241994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51024/0 (socket says 172.21.15.60:51024) 2024-02-26T04:55:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c28834640 1 -- 172.21.15.60:0/2276204238 learned_addr learned my addr 172.21.15.60:0/2276204238 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c23fff640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c24102380 0x7f4c241999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c29035640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c24102d40 0x7f4c241a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c28834640 1 -- 172.21.15.60:0/2276204238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c24102380 msgr2=0x7f4c241999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c28834640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c24102380 0x7f4c241999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c28834640 1 -- 172.21.15.60:0/2276204238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c24102d40 msgr2=0x7f4c241a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c28834640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c24102d40 0x7f4c241a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c28834640 1 -- 172.21.15.60:0/2276204238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c10009560 con 0x7f4c24101190 2024-02-26T04:55:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c23fff640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c24102380 0x7f4c241999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c28834640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24101190 0x7f4c241994a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800d950 tx=0x7f4c1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c21ffb640 1 -- 172.21.15.60:0/2276204238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18009c30 con 0x7f4c24101190 2024-02-26T04:55:54.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c2aa98640 1 -- 172.21.15.60:0/2276204238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c241a15a0 con 0x7f4c24101190 2024-02-26T04:55:54.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c21ffb640 1 -- 172.21.15.60:0/2276204238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c18009d90 con 0x7f4c24101190 2024-02-26T04:55:54.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.991+0000 7f4c21ffb640 1 -- 172.21.15.60:0/2276204238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1800f720 con 0x7f4c24101190 2024-02-26T04:55:54.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.992+0000 7f4c2aa98640 1 -- 172.21.15.60:0/2276204238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c241a1ab0 con 0x7f4c24101190 2024-02-26T04:55:54.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.993+0000 7f4c2aa98640 1 -- 172.21.15.60:0/2276204238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2410cba0 con 0x7f4c24101190 2024-02-26T04:55:54.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.994+0000 7f4c21ffb640 1 -- 172.21.15.60:0/2276204238 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f4c18010a60 con 0x7f4c24101190 2024-02-26T04:55:54.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.994+0000 7f4c21ffb640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4c140a20c0 0x7f4c140a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:54.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.994+0000 7f4c21ffb640 1 -- 172.21.15.60:0/2276204238 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4c180cdce0 con 0x7f4c24101190 2024-02-26T04:55:54.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.994+0000 7f4c23fff640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4c140a20c0 0x7f4c140a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:54.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.995+0000 7f4c23fff640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4c140a20c0 0x7f4c140a4580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c2419a9b0 tx=0x7f4c10005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:55.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:54.999+0000 7f4c21ffb640 1 -- 172.21.15.60:0/2276204238 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4c18096a70 con 0x7f4c24101190 2024-02-26T04:55:55.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:54 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2191051244' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-26T04:55:55.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:54 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] up:active 2024-02-26T04:55:55.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:54 smithi191 ceph-mon[36352]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-26T04:55:55.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:54 smithi191 ceph-mon[36352]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-26T04:55:55.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:54 smithi191 ceph-mon[36352]: Cluster is now healthy 2024-02-26T04:55:55.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:54 smithi191 ceph-mon[36352]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-02-26T04:55:55.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:54 smithi191 ceph-mon[36352]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-02-26T04:55:55.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:54 smithi191 ceph-mon[36352]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-02-26T04:55:55.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:54 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2525899410' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-26T04:55:55.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:54 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2525899410' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-26T04:55:55.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:54 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] up:active 2024-02-26T04:55:55.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:54 smithi191 ceph-mon[36352]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T04:55:55.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.210+0000 7f4c2aa98640 1 -- 172.21.15.60:0/2276204238 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c24074ad0 con 0x7f4c140a20c0 2024-02-26T04:55:55.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.214+0000 7f4c21ffb640 1 -- 172.21.15.60:0/2276204238 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f4c24074ad0 con 0x7f4c140a20c0 2024-02-26T04:55:55.217 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/qa/sv_0/a54503c3-323c-43b0-9f7d-87e8fa877d8e 2024-02-26T04:55:55.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.217+0000 7f4c2aa98640 1 -- 172.21.15.60:0/2276204238 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4c140a20c0 msgr2=0x7f4c140a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:55.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.217+0000 7f4c2aa98640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4c140a20c0 0x7f4c140a4580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c2419a9b0 tx=0x7f4c10005be0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.217+0000 7f4c2aa98640 1 -- 172.21.15.60:0/2276204238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24101190 msgr2=0x7f4c241994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:55.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.217+0000 7f4c2aa98640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24101190 0x7f4c241994a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800d950 tx=0x7f4c1800de10 comp rx=0 tx=0).stop 2024-02-26T04:55:55.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.217+0000 7f4c2aa98640 1 -- 172.21.15.60:0/2276204238 shutdown_connections 2024-02-26T04:55:55.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.217+0000 7f4c2aa98640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4c140a20c0 0x7f4c140a4580 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.217+0000 7f4c2aa98640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c24102d40 0x7f4c241a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.217+0000 7f4c2aa98640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c24102380 0x7f4c241999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.217+0000 7f4c2aa98640 1 --2- 172.21.15.60:0/2276204238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24101190 0x7f4c241994a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.217+0000 7f4c2aa98640 1 -- 172.21.15.60:0/2276204238 >> 172.21.15.60:0/2276204238 conn(0x7f4c240fc720 msgr2=0x7f4c240733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:55.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.217+0000 7f4c2aa98640 1 -- 172.21.15.60:0/2276204238 shutdown_connections 2024-02-26T04:55:55.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.217+0000 7f4c2aa98640 1 -- 172.21.15.60:0/2276204238 wait complete. 2024-02-26T04:55:55.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-02-26T04:55:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.335+0000 7f02692b0640 1 -- 172.21.15.60:0/3246227396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0264101c50 msgr2=0x7f02641084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.335+0000 7f02692b0640 1 --2- 172.21.15.60:0/3246227396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0264101c50 0x7f02641084e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f025c009ca0 tx=0x7f025c02c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.335+0000 7f02692b0640 1 -- 172.21.15.60:0/3246227396 shutdown_connections 2024-02-26T04:55:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.335+0000 7f02692b0640 1 --2- 172.21.15.60:0/3246227396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0264101c50 0x7f02641084e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.335+0000 7f02692b0640 1 --2- 172.21.15.60:0/3246227396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0264101290 0x7f0264101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.335+0000 7f02692b0640 1 --2- 172.21.15.60:0/3246227396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02641000a0 0x7f02641004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.335+0000 7f02692b0640 1 -- 172.21.15.60:0/3246227396 >> 172.21.15.60:0/3246227396 conn(0x7f02640fb630 msgr2=0x7f02640fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.336+0000 7f02692b0640 1 -- 172.21.15.60:0/3246227396 shutdown_connections 2024-02-26T04:55:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.336+0000 7f02692b0640 1 -- 172.21.15.60:0/3246227396 wait complete. 2024-02-26T04:55:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.336+0000 7f02692b0640 1 Processor -- start 2024-02-26T04:55:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.336+0000 7f02692b0640 1 -- start start 2024-02-26T04:55:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.336+0000 7f02692b0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02641000a0 0x7f02641971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.336+0000 7f02692b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0264101290 0x7f0264197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.336+0000 7f02692b0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0264101c50 0x7f026419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.336+0000 7f02692b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026419ecd0 con 0x7f0264101290 2024-02-26T04:55:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.336+0000 7f02692b0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026419ee40 con 0x7f0264101c50 2024-02-26T04:55:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.336+0000 7f02692b0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026419f000 con 0x7f02641000a0 2024-02-26T04:55:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f02627fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0264101290 0x7f0264197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f02627fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0264101290 0x7f0264197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51050/0 (socket says 172.21.15.60:51050) 2024-02-26T04:55:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f02627fc640 1 -- 172.21.15.60:0/3969917356 learned_addr learned my addr 172.21.15.60:0/3969917356 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:55.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f02637fe640 1 --2- 172.21.15.60:0/3969917356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0264101c50 0x7f026419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:55.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f0262ffd640 1 --2- 172.21.15.60:0/3969917356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02641000a0 0x7f02641971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:55.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f02627fc640 1 -- 172.21.15.60:0/3969917356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02641000a0 msgr2=0x7f02641971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:55.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f02627fc640 1 --2- 172.21.15.60:0/3969917356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02641000a0 0x7f02641971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f02627fc640 1 -- 172.21.15.60:0/3969917356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0264101c50 msgr2=0x7f026419e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:55.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f02627fc640 1 --2- 172.21.15.60:0/3969917356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0264101c50 0x7f026419e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f02627fc640 1 -- 172.21.15.60:0/3969917356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c009900 con 0x7f0264101290 2024-02-26T04:55:55.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f02627fc640 1 --2- 172.21.15.60:0/3969917356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0264101290 0x7f0264197720 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f025800c920 tx=0x7f025800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:55.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f024bfff640 1 -- 172.21.15.60:0/3969917356 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258007c70 con 0x7f0264101290 2024-02-26T04:55:55.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f02692b0640 1 -- 172.21.15.60:0/3969917356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026419f2e0 con 0x7f0264101290 2024-02-26T04:55:55.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f024bfff640 1 -- 172.21.15.60:0/3969917356 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0258007dd0 con 0x7f0264101290 2024-02-26T04:55:55.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f024bfff640 1 -- 172.21.15.60:0/3969917356 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258018860 con 0x7f0264101290 2024-02-26T04:55:55.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.337+0000 7f02692b0640 1 -- 172.21.15.60:0/3969917356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026419f840 con 0x7f0264101290 2024-02-26T04:55:55.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.338+0000 7f02692b0640 1 -- 172.21.15.60:0/3969917356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026410bab0 con 0x7f0264101290 2024-02-26T04:55:55.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.339+0000 7f024bfff640 1 -- 172.21.15.60:0/3969917356 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f025800fac0 con 0x7f0264101290 2024-02-26T04:55:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.340+0000 7f024bfff640 1 --2- 172.21.15.60:0/3969917356 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f024c0a20c0 0x7f024c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.340+0000 7f0262ffd640 1 --2- 172.21.15.60:0/3969917356 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f024c0a20c0 0x7f024c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.340+0000 7f024bfff640 1 -- 172.21.15.60:0/3969917356 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f02580ce100 con 0x7f0264101290 2024-02-26T04:55:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.340+0000 7f0262ffd640 1 --2- 172.21.15.60:0/3969917356 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f024c0a20c0 0x7f024c0a4580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0254005fd0 tx=0x7f0254005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.345+0000 7f024bfff640 1 -- 172.21.15.60:0/3969917356 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0258096e30 con 0x7f0264101290 2024-02-26T04:55:55.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.552+0000 7f024bfff640 1 -- 172.21.15.60:0/3969917356 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f0258097200 con 0x7f0264101290 2024-02-26T04:55:55.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.560+0000 7f02692b0640 1 -- 172.21.15.60:0/3969917356 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f02640fd700 con 0x7f024c0a20c0 2024-02-26T04:55:55.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.581+0000 7f024bfff640 1 -- 172.21.15.60:0/3969917356 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f02640fd700 con 0x7f024c0a20c0 2024-02-26T04:55:55.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.583+0000 7f02692b0640 1 -- 172.21.15.60:0/3969917356 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f024c0a20c0 msgr2=0x7f024c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:55.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.583+0000 7f02692b0640 1 --2- 172.21.15.60:0/3969917356 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f024c0a20c0 0x7f024c0a4580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0254005fd0 tx=0x7f0254005850 comp rx=0 tx=0).stop 2024-02-26T04:55:55.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.583+0000 7f02692b0640 1 -- 172.21.15.60:0/3969917356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0264101290 msgr2=0x7f0264197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:55.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.583+0000 7f02692b0640 1 --2- 172.21.15.60:0/3969917356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0264101290 0x7f0264197720 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f025800c920 tx=0x7f025800cde0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.583+0000 7f02692b0640 1 -- 172.21.15.60:0/3969917356 shutdown_connections 2024-02-26T04:55:55.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.583+0000 7f02692b0640 1 --2- 172.21.15.60:0/3969917356 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f024c0a20c0 0x7f024c0a4580 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.583+0000 7f02692b0640 1 --2- 172.21.15.60:0/3969917356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0264101c50 0x7f026419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.583+0000 7f02692b0640 1 --2- 172.21.15.60:0/3969917356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0264101290 0x7f0264197720 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.583+0000 7f02692b0640 1 --2- 172.21.15.60:0/3969917356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02641000a0 0x7f02641971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.583+0000 7f02692b0640 1 -- 172.21.15.60:0/3969917356 >> 172.21.15.60:0/3969917356 conn(0x7f02640fb630 msgr2=0x7f02640fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:55.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.584+0000 7f02692b0640 1 -- 172.21.15.60:0/3969917356 shutdown_connections 2024-02-26T04:55:55.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.584+0000 7f02692b0640 1 -- 172.21.15.60:0/3969917356 wait complete. 2024-02-26T04:55:55.597 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-02-26T04:55:55.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.702+0000 7ff4ac288640 1 -- 172.21.15.60:0/2641038164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a41024f0 msgr2=0x7ff4a4102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:55.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.702+0000 7ff4ac288640 1 --2- 172.21.15.60:0/2641038164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a41024f0 0x7ff4a4102970 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff494009900 tx=0x7ff49402bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:55.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.702+0000 7ff4ac288640 1 -- 172.21.15.60:0/2641038164 shutdown_connections 2024-02-26T04:55:55.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.702+0000 7ff4ac288640 1 --2- 172.21.15.60:0/2641038164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4a4102eb0 0x7ff4a4109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.702+0000 7ff4ac288640 1 --2- 172.21.15.60:0/2641038164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a41024f0 0x7ff4a4102970 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.702+0000 7ff4ac288640 1 --2- 172.21.15.60:0/2641038164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4a40ff480 0x7ff4a40ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.702+0000 7ff4ac288640 1 -- 172.21.15.60:0/2641038164 >> 172.21.15.60:0/2641038164 conn(0x7ff4a40fad70 msgr2=0x7ff4a40fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:55.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.703+0000 7ff4ac288640 1 -- 172.21.15.60:0/2641038164 shutdown_connections 2024-02-26T04:55:55.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.703+0000 7ff4ac288640 1 -- 172.21.15.60:0/2641038164 wait complete. 2024-02-26T04:55:55.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.703+0000 7ff4ac288640 1 Processor -- start 2024-02-26T04:55:55.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.703+0000 7ff4ac288640 1 -- start start 2024-02-26T04:55:55.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.703+0000 7ff4ac288640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4a40ff480 0x7ff4a41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.703+0000 7ff4ac288640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4a41024f0 0x7ff4a4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.703+0000 7ff4ac288640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a4102eb0 0x7ff4a419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.703+0000 7ff4ac288640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a419edc0 con 0x7ff4a4102eb0 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.703+0000 7ff4ac288640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a419ef30 con 0x7ff4a41024f0 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.703+0000 7ff4ac288640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a419f0f0 con 0x7ff4a40ff480 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.703+0000 7ff4aa825640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a4102eb0 0x7ff4a419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.703+0000 7ff4aa825640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a4102eb0 0x7ff4a419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51066/0 (socket says 172.21.15.60:51066) 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.703+0000 7ff4aa825640 1 -- 172.21.15.60:0/1704578949 learned_addr learned my addr 172.21.15.60:0/1704578949 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.703+0000 7ff4a9823640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4a41024f0 0x7ff4a4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.704+0000 7ff4aa024640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4a40ff480 0x7ff4a41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.704+0000 7ff4aa825640 1 -- 172.21.15.60:0/1704578949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4a40ff480 msgr2=0x7ff4a41972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.704+0000 7ff4aa825640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4a40ff480 0x7ff4a41972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.704+0000 7ff4aa825640 1 -- 172.21.15.60:0/1704578949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4a41024f0 msgr2=0x7ff4a4197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.704+0000 7ff4aa825640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4a41024f0 0x7ff4a4197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.704+0000 7ff4aa825640 1 -- 172.21.15.60:0/1704578949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff494009560 con 0x7ff4a4102eb0 2024-02-26T04:55:55.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.704+0000 7ff4aa024640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4a40ff480 0x7ff4a41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:55.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.704+0000 7ff4aa825640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a4102eb0 0x7ff4a419e880 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00b970 tx=0x7ff49c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:55.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.704+0000 7ff49b7fe640 1 -- 172.21.15.60:0/1704578949 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00cb40 con 0x7ff4a4102eb0 2024-02-26T04:55:55.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.704+0000 7ff4ac288640 1 -- 172.21.15.60:0/1704578949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a419f3d0 con 0x7ff4a4102eb0 2024-02-26T04:55:55.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.704+0000 7ff49b7fe640 1 -- 172.21.15.60:0/1704578949 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff49c00cca0 con 0x7ff4a4102eb0 2024-02-26T04:55:55.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.704+0000 7ff49b7fe640 1 -- 172.21.15.60:0/1704578949 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0077d0 con 0x7ff4a4102eb0 2024-02-26T04:55:55.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.704+0000 7ff4ac288640 1 -- 172.21.15.60:0/1704578949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a419f8e0 con 0x7ff4a4102eb0 2024-02-26T04:55:55.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.705+0000 7ff4ac288640 1 -- 172.21.15.60:0/1704578949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a4108620 con 0x7ff4a4102eb0 2024-02-26T04:55:55.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.706+0000 7ff49b7fe640 1 -- 172.21.15.60:0/1704578949 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ff49c014440 con 0x7ff4a4102eb0 2024-02-26T04:55:55.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.706+0000 7ff49b7fe640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff4900a2150 0x7ff4900a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:55.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.707+0000 7ff4aa024640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff4900a2150 0x7ff4900a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:55.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.707+0000 7ff49b7fe640 1 -- 172.21.15.60:0/1704578949 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff49c0cd830 con 0x7ff4a4102eb0 2024-02-26T04:55:55.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.707+0000 7ff4aa024640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff4900a2150 0x7ff4900a4610 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4069490 tx=0x7ff4a0005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:55.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.711+0000 7ff49b7fe640 1 -- 172.21.15.60:0/1704578949 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff49c04fd20 con 0x7ff4a4102eb0 2024-02-26T04:55:55.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.927+0000 7ff4ac288640 1 -- 172.21.15.60:0/1704578949 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4a402ceb0 con 0x7ff4900a2150 2024-02-26T04:55:55.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.931+0000 7ff49b7fe640 1 -- 172.21.15.60:0/1704578949 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7ff4a402ceb0 con 0x7ff4900a2150 2024-02-26T04:55:55.934 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/qa/sv_1/b6609776-6af2-4a43-be85-51b16d9004c9 2024-02-26T04:55:55.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.933+0000 7ff4ac288640 1 -- 172.21.15.60:0/1704578949 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff4900a2150 msgr2=0x7ff4900a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:55.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.933+0000 7ff4ac288640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff4900a2150 0x7ff4900a4610 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4069490 tx=0x7ff4a0005e70 comp rx=0 tx=0).stop 2024-02-26T04:55:55.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.933+0000 7ff4ac288640 1 -- 172.21.15.60:0/1704578949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a4102eb0 msgr2=0x7ff4a419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:55.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.933+0000 7ff4ac288640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a4102eb0 0x7ff4a419e880 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00b970 tx=0x7ff49c00be30 comp rx=0 tx=0).stop 2024-02-26T04:55:55.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.933+0000 7ff4ac288640 1 -- 172.21.15.60:0/1704578949 shutdown_connections 2024-02-26T04:55:55.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.933+0000 7ff4ac288640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff4900a2150 0x7ff4900a4610 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.933+0000 7ff4ac288640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4a4102eb0 0x7ff4a419e880 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.933+0000 7ff4ac288640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4a41024f0 0x7ff4a4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.933+0000 7ff4ac288640 1 --2- 172.21.15.60:0/1704578949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4a40ff480 0x7ff4a41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:55.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.933+0000 7ff4ac288640 1 -- 172.21.15.60:0/1704578949 >> 172.21.15.60:0/1704578949 conn(0x7ff4a40fad70 msgr2=0x7ff4a41057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:55.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.933+0000 7ff4ac288640 1 -- 172.21.15.60:0/1704578949 shutdown_connections 2024-02-26T04:55:55.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:55.933+0000 7ff4ac288640 1 -- 172.21.15.60:0/1704578949 wait complete. 2024-02-26T04:55:55.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:55:56.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.047+0000 7fade35f7640 1 -- 172.21.15.60:0/117508775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faddc101ca0 msgr2=0x7faddc104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.047+0000 7fade35f7640 1 --2- 172.21.15.60:0/117508775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faddc101ca0 0x7faddc104090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009900 tx=0x7fadd402bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:56.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.047+0000 7fade35f7640 1 -- 172.21.15.60:0/117508775 shutdown_connections 2024-02-26T04:55:56.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.047+0000 7fade35f7640 1 --2- 172.21.15.60:0/117508775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faddc106f00 0x7faddc1092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.047+0000 7fade35f7640 1 --2- 172.21.15.60:0/117508775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faddc1045d0 0x7faddc1069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.047+0000 7fade35f7640 1 --2- 172.21.15.60:0/117508775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faddc101ca0 0x7faddc104090 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.047+0000 7fade35f7640 1 -- 172.21.15.60:0/117508775 >> 172.21.15.60:0/117508775 conn(0x7faddc0fb610 msgr2=0x7faddc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.047+0000 7fade35f7640 1 -- 172.21.15.60:0/117508775 shutdown_connections 2024-02-26T04:55:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.047+0000 7fade35f7640 1 -- 172.21.15.60:0/117508775 wait complete. 2024-02-26T04:55:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.048+0000 7fade35f7640 1 Processor -- start 2024-02-26T04:55:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.048+0000 7fade35f7640 1 -- start start 2024-02-26T04:55:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.048+0000 7fade35f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faddc101ca0 0x7faddc197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.048+0000 7fade35f7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faddc1045d0 0x7faddc197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.048+0000 7fade35f7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faddc106f00 0x7faddc19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.048+0000 7fade35f7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc19ec30 con 0x7faddc101ca0 2024-02-26T04:55:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.048+0000 7fade35f7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc19eda0 con 0x7faddc1045d0 2024-02-26T04:55:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.048+0000 7fade35f7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc19ef60 con 0x7faddc106f00 2024-02-26T04:55:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.048+0000 7fade1393640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faddc101ca0 0x7faddc197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.048+0000 7fade1393640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faddc101ca0 0x7faddc197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51086/0 (socket says 172.21.15.60:51086) 2024-02-26T04:55:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.048+0000 7fade1393640 1 -- 172.21.15.60:0/2707948398 learned_addr learned my addr 172.21.15.60:0/2707948398 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.048+0000 7fade1393640 1 -- 172.21.15.60:0/2707948398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faddc106f00 msgr2=0x7faddc19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.048+0000 7fade1393640 1 --2- 172.21.15.60:0/2707948398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faddc106f00 0x7faddc19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.049+0000 7fade1393640 1 -- 172.21.15.60:0/2707948398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faddc1045d0 msgr2=0x7faddc197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.049+0000 7fade0b92640 1 --2- 172.21.15.60:0/2707948398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faddc1045d0 0x7faddc197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.049+0000 7fade1393640 1 --2- 172.21.15.60:0/2707948398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faddc1045d0 0x7faddc197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.049+0000 7fade1393640 1 -- 172.21.15.60:0/2707948398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd4009560 con 0x7faddc101ca0 2024-02-26T04:55:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.049+0000 7fade0b92640 1 --2- 172.21.15.60:0/2707948398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faddc1045d0 0x7faddc197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.049+0000 7fade1393640 1 --2- 172.21.15.60:0/2707948398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faddc101ca0 0x7faddc197140 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fadd4005ab0 tx=0x7fadd40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.049+0000 7fadda7fc640 1 -- 172.21.15.60:0/2707948398 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd40044e0 con 0x7faddc101ca0 2024-02-26T04:55:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.049+0000 7fadda7fc640 1 -- 172.21.15.60:0/2707948398 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadd40054e0 con 0x7faddc101ca0 2024-02-26T04:55:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.049+0000 7fade35f7640 1 -- 172.21.15.60:0/2707948398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faddc19f1e0 con 0x7faddc101ca0 2024-02-26T04:55:56.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.049+0000 7fadda7fc640 1 -- 172.21.15.60:0/2707948398 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd403e450 con 0x7faddc101ca0 2024-02-26T04:55:56.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.049+0000 7fade35f7640 1 -- 172.21.15.60:0/2707948398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faddc19f740 con 0x7faddc101ca0 2024-02-26T04:55:56.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.050+0000 7fade35f7640 1 -- 172.21.15.60:0/2707948398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faddc0fd100 con 0x7faddc101ca0 2024-02-26T04:55:56.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.051+0000 7fadda7fc640 1 -- 172.21.15.60:0/2707948398 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fadd40035f0 con 0x7faddc101ca0 2024-02-26T04:55:56.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.052+0000 7fadda7fc640 1 --2- 172.21.15.60:0/2707948398 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fadc40a21d0 0x7fadc40a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:56.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.052+0000 7fade0b92640 1 --2- 172.21.15.60:0/2707948398 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fadc40a21d0 0x7fadc40a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:56.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.052+0000 7fadda7fc640 1 -- 172.21.15.60:0/2707948398 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fadd40f04e0 con 0x7faddc101ca0 2024-02-26T04:55:56.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.052+0000 7fade0b92640 1 --2- 172.21.15.60:0/2707948398 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fadc40a21d0 0x7fadc40a4690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faddc198650 tx=0x7fadd0005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.056+0000 7fadda7fc640 1 -- 172.21.15.60:0/2707948398 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadd40b9220 con 0x7faddc101ca0 2024-02-26T04:55:56.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:56 smithi060 ceph-mon[32382]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 1 op/s 2024-02-26T04:55:56.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:56 smithi060 ceph-mon[32382]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:56.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:56 smithi060 ceph-mon[32382]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:56.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:56 smithi060 ceph-mon[32382]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:56.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:56 smithi060 ceph-mon[32382]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-26T04:55:56.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.279+0000 7fade35f7640 1 -- 172.21.15.60:0/2707948398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faddc19f370 con 0x7faddc101ca0 2024-02-26T04:55:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.280+0000 7fadda7fc640 1 -- 172.21.15.60:0/2707948398 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11347 (secure 0 0 0) 0x7faddc19f370 con 0x7faddc101ca0 2024-02-26T04:55:56.282 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:56.282 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:55:54.140330+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24476,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T04:55:56.283 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-02-26T04:55:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.282+0000 7fade35f7640 1 -- 172.21.15.60:0/2707948398 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fadc40a21d0 msgr2=0x7fadc40a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.282+0000 7fade35f7640 1 --2- 172.21.15.60:0/2707948398 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fadc40a21d0 0x7fadc40a4690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faddc198650 tx=0x7fadd0005d90 comp rx=0 tx=0).stop 2024-02-26T04:55:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.282+0000 7fade35f7640 1 -- 172.21.15.60:0/2707948398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faddc101ca0 msgr2=0x7faddc197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.282+0000 7fade35f7640 1 --2- 172.21.15.60:0/2707948398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faddc101ca0 0x7faddc197140 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fadd4005ab0 tx=0x7fadd40042f0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.282+0000 7fade35f7640 1 -- 172.21.15.60:0/2707948398 shutdown_connections 2024-02-26T04:55:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.282+0000 7fade35f7640 1 --2- 172.21.15.60:0/2707948398 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fadc40a21d0 0x7fadc40a4690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.282+0000 7fade35f7640 1 --2- 172.21.15.60:0/2707948398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faddc106f00 0x7faddc19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.282+0000 7fade35f7640 1 --2- 172.21.15.60:0/2707948398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faddc1045d0 0x7faddc197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.282+0000 7fade35f7640 1 --2- 172.21.15.60:0/2707948398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faddc101ca0 0x7faddc197140 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.282+0000 7fade35f7640 1 -- 172.21.15.60:0/2707948398 >> 172.21.15.60:0/2707948398 conn(0x7faddc0fb610 msgr2=0x7faddc1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.282+0000 7fade35f7640 1 -- 172.21.15.60:0/2707948398 shutdown_connections 2024-02-26T04:55:56.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.282+0000 7fade35f7640 1 -- 172.21.15.60:0/2707948398 wait complete. 2024-02-26T04:55:56.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:55:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.396+0000 7fe3d59c8640 1 -- 172.21.15.60:0/2641710539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3d01000c0 msgr2=0x7fe3d01004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.396+0000 7fe3d59c8640 1 --2- 172.21.15.60:0/2641710539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3d01000c0 0x7fe3d01004e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009ca0 tx=0x7fe3c802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.397+0000 7fe3d59c8640 1 -- 172.21.15.60:0/2641710539 shutdown_connections 2024-02-26T04:55:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.397+0000 7fe3d59c8640 1 --2- 172.21.15.60:0/2641710539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d0101c70 0x7fe3d0108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.397+0000 7fe3d59c8640 1 --2- 172.21.15.60:0/2641710539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d01012b0 0x7fe3d0101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.397+0000 7fe3d59c8640 1 --2- 172.21.15.60:0/2641710539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3d01000c0 0x7fe3d01004e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.397+0000 7fe3d59c8640 1 -- 172.21.15.60:0/2641710539 >> 172.21.15.60:0/2641710539 conn(0x7fe3d00fb630 msgr2=0x7fe3d00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.397+0000 7fe3d59c8640 1 -- 172.21.15.60:0/2641710539 shutdown_connections 2024-02-26T04:55:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.397+0000 7fe3d59c8640 1 -- 172.21.15.60:0/2641710539 wait complete. 2024-02-26T04:55:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.397+0000 7fe3d59c8640 1 Processor -- start 2024-02-26T04:55:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.397+0000 7fe3d59c8640 1 -- start start 2024-02-26T04:55:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.397+0000 7fe3d59c8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d01000c0 0x7fe3d006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.397+0000 7fe3d59c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3d01012b0 0x7fe3d019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.397+0000 7fe3d59c8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d0101c70 0x7fe3d019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.397+0000 7fe3d59c8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d019ede0 con 0x7fe3d01012b0 2024-02-26T04:55:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3d59c8640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d019ef50 con 0x7fe3d0101c70 2024-02-26T04:55:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3d59c8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d019f110 con 0x7fe3d01000c0 2024-02-26T04:55:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3ce7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3d01012b0 0x7fe3d019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3ce7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3d01012b0 0x7fe3d019bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51116/0 (socket says 172.21.15.60:51116) 2024-02-26T04:55:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3ce7fc640 1 -- 172.21.15.60:0/1407571056 learned_addr learned my addr 172.21.15.60:0/1407571056 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3cf7fe640 1 --2- 172.21.15.60:0/1407571056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d0101c70 0x7fe3d019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3ceffd640 1 --2- 172.21.15.60:0/1407571056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d01000c0 0x7fe3d006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3ce7fc640 1 -- 172.21.15.60:0/1407571056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d01000c0 msgr2=0x7fe3d006a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3ce7fc640 1 --2- 172.21.15.60:0/1407571056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d01000c0 0x7fe3d006a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3ce7fc640 1 -- 172.21.15.60:0/1407571056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d0101c70 msgr2=0x7fe3d019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3ce7fc640 1 --2- 172.21.15.60:0/1407571056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d0101c70 0x7fe3d019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3ce7fc640 1 -- 172.21.15.60:0/1407571056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c8009900 con 0x7fe3d01012b0 2024-02-26T04:55:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3ce7fc640 1 --2- 172.21.15.60:0/1407571056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3d01012b0 0x7fe3d019bfe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800b4a0 tx=0x7fe3b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3d49c6640 1 -- 172.21.15.60:0/1407571056 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8010610 con 0x7fe3d01012b0 2024-02-26T04:55:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3d49c6640 1 -- 172.21.15.60:0/1407571056 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3b8010c50 con 0x7fe3d01012b0 2024-02-26T04:55:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3d59c8640 1 -- 172.21.15.60:0/1407571056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d019f390 con 0x7fe3d01012b0 2024-02-26T04:55:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3d49c6640 1 -- 172.21.15.60:0/1407571056 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8017e20 con 0x7fe3d01012b0 2024-02-26T04:55:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.398+0000 7fe3d59c8640 1 -- 172.21.15.60:0/1407571056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d019f850 con 0x7fe3d01012b0 2024-02-26T04:55:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.399+0000 7fe3d59c8640 1 -- 172.21.15.60:0/1407571056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d01073e0 con 0x7fe3d01012b0 2024-02-26T04:55:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.400+0000 7fe3d49c6640 1 -- 172.21.15.60:0/1407571056 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe3b8017480 con 0x7fe3d01012b0 2024-02-26T04:55:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.401+0000 7fe3d49c6640 1 --2- 172.21.15.60:0/1407571056 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe3bc0a2150 0x7fe3bc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.401+0000 7fe3ceffd640 1 --2- 172.21.15.60:0/1407571056 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe3bc0a2150 0x7fe3bc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.401+0000 7fe3d49c6640 1 -- 172.21.15.60:0/1407571056 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe3b80cd650 con 0x7fe3d01012b0 2024-02-26T04:55:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.401+0000 7fe3ceffd640 1 --2- 172.21.15.60:0/1407571056 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe3bc0a2150 0x7fe3bc0a4610 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe3c802c700 tx=0x7fe3c80093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.405+0000 7fe3d49c6640 1 -- 172.21.15.60:0/1407571056 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe3b8096330 con 0x7fe3d01012b0 2024-02-26T04:55:56.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:56 smithi156 ceph-mon[35805]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 1 op/s 2024-02-26T04:55:56.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:56 smithi156 ceph-mon[35805]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:56.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:56 smithi156 ceph-mon[35805]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:56.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:56 smithi156 ceph-mon[35805]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:56.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:56 smithi156 ceph-mon[35805]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-26T04:55:56.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:56 smithi191 ceph-mon[36352]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 1 op/s 2024-02-26T04:55:56.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:56 smithi191 ceph-mon[36352]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:56.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:56 smithi191 ceph-mon[36352]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:56.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:56 smithi191 ceph-mon[36352]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:56.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:56 smithi191 ceph-mon[36352]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-26T04:55:56.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.588+0000 7fe3d59c8640 1 -- 172.21.15.60:0/1407571056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3d010bcc0 con 0x7fe3d01012b0 2024-02-26T04:55:56.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.589+0000 7fe3d49c6640 1 -- 172.21.15.60:0/1407571056 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24126 (secure 0 0 0) 0x7fe3b80999f0 con 0x7fe3d01012b0 2024-02-26T04:55:56.593 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:56.593 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:55:32.581481+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:55:56.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.591+0000 7fe3d59c8640 1 -- 172.21.15.60:0/1407571056 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe3bc0a2150 msgr2=0x7fe3bc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.591+0000 7fe3d59c8640 1 --2- 172.21.15.60:0/1407571056 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe3bc0a2150 0x7fe3bc0a4610 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe3c802c700 tx=0x7fe3c80093f0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.591+0000 7fe3d59c8640 1 -- 172.21.15.60:0/1407571056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3d01012b0 msgr2=0x7fe3d019bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.591+0000 7fe3d59c8640 1 --2- 172.21.15.60:0/1407571056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3d01012b0 0x7fe3d019bfe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800b4a0 tx=0x7fe3b800b960 comp rx=0 tx=0).stop 2024-02-26T04:55:56.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.591+0000 7fe3d59c8640 1 -- 172.21.15.60:0/1407571056 shutdown_connections 2024-02-26T04:55:56.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.591+0000 7fe3d59c8640 1 --2- 172.21.15.60:0/1407571056 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe3bc0a2150 0x7fe3bc0a4610 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.591+0000 7fe3d59c8640 1 --2- 172.21.15.60:0/1407571056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d0101c70 0x7fe3d019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.591+0000 7fe3d59c8640 1 --2- 172.21.15.60:0/1407571056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3d01012b0 0x7fe3d019bfe0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.591+0000 7fe3d59c8640 1 --2- 172.21.15.60:0/1407571056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d01000c0 0x7fe3d006a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.591+0000 7fe3d59c8640 1 -- 172.21.15.60:0/1407571056 >> 172.21.15.60:0/1407571056 conn(0x7fe3d00fb630 msgr2=0x7fe3d01045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:56.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.591+0000 7fe3d59c8640 1 -- 172.21.15.60:0/1407571056 shutdown_connections 2024-02-26T04:55:56.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.591+0000 7fe3d59c8640 1 -- 172.21.15.60:0/1407571056 wait complete. 2024-02-26T04:55:56.605 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-26T04:55:56.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T04:55:56.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.706+0000 7f2a044d5640 1 -- 172.21.15.60:0/1239732445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29fc101c80 msgr2=0x7f29fc104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.706+0000 7f2a044d5640 1 --2- 172.21.15.60:0/1239732445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29fc101c80 0x7f29fc104070 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f29f8009900 tx=0x7f29f802bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:56.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.706+0000 7f2a044d5640 1 -- 172.21.15.60:0/1239732445 shutdown_connections 2024-02-26T04:55:56.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.706+0000 7f2a044d5640 1 --2- 172.21.15.60:0/1239732445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29fc106ee0 0x7f29fc1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.706+0000 7f2a044d5640 1 --2- 172.21.15.60:0/1239732445 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29fc1045b0 0x7f29fc1069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.706+0000 7f2a044d5640 1 --2- 172.21.15.60:0/1239732445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29fc101c80 0x7f29fc104070 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.706+0000 7f2a044d5640 1 -- 172.21.15.60:0/1239732445 >> 172.21.15.60:0/1239732445 conn(0x7f29fc0fb630 msgr2=0x7f29fc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:56.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.707+0000 7f2a044d5640 1 -- 172.21.15.60:0/1239732445 shutdown_connections 2024-02-26T04:55:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.707+0000 7f2a044d5640 1 -- 172.21.15.60:0/1239732445 wait complete. 2024-02-26T04:55:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.707+0000 7f2a044d5640 1 Processor -- start 2024-02-26T04:55:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.707+0000 7f2a044d5640 1 -- start start 2024-02-26T04:55:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.707+0000 7f2a044d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29fc101c80 0x7f29fc1015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.707+0000 7f2a044d5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29fc1045b0 0x7f29fc0ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.707+0000 7f2a044d5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29fc106ee0 0x7f29fc100170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.707+0000 7f2a044d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc1008d0 con 0x7f29fc101c80 2024-02-26T04:55:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.707+0000 7f2a044d5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc100a40 con 0x7f29fc1045b0 2024-02-26T04:55:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.707+0000 7f2a044d5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc1a1260 con 0x7f29fc106ee0 2024-02-26T04:55:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.707+0000 7f2a02271640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29fc101c80 0x7f29fc1015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.707+0000 7f2a02271640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29fc101c80 0x7f29fc1015f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51134/0 (socket says 172.21.15.60:51134) 2024-02-26T04:55:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.707+0000 7f2a02271640 1 -- 172.21.15.60:0/61526844 learned_addr learned my addr 172.21.15.60:0/61526844 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f2a01a70640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29fc1045b0 0x7f29fc0ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f2a02271640 1 -- 172.21.15.60:0/61526844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29fc106ee0 msgr2=0x7f29fc100170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f2a02a72640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29fc106ee0 0x7f29fc100170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:56.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f2a02271640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29fc106ee0 0x7f29fc100170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f2a02271640 1 -- 172.21.15.60:0/61526844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29fc1045b0 msgr2=0x7f29fc0ffc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f2a02271640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29fc1045b0 0x7f29fc0ffc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f2a02271640 1 -- 172.21.15.60:0/61526844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f8009560 con 0x7f29fc101c80 2024-02-26T04:55:56.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f2a01a70640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29fc1045b0 0x7f29fc0ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:56.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f2a02a72640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29fc106ee0 0x7f29fc100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:55:56.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f2a02271640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29fc101c80 0x7f29fc1015f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f29f8036040 tx=0x7f29f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:56.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f29f37fe640 1 -- 172.21.15.60:0/61526844 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f80044e0 con 0x7f29fc101c80 2024-02-26T04:55:56.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f29f37fe640 1 -- 172.21.15.60:0/61526844 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29f80054e0 con 0x7f29fc101c80 2024-02-26T04:55:56.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f2a044d5640 1 -- 172.21.15.60:0/61526844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29fc1a14b0 con 0x7f29fc101c80 2024-02-26T04:55:56.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f29f37fe640 1 -- 172.21.15.60:0/61526844 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8003ec0 con 0x7f29fc101c80 2024-02-26T04:55:56.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.708+0000 7f2a044d5640 1 -- 172.21.15.60:0/61526844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29fc1a19c0 con 0x7f29fc101c80 2024-02-26T04:55:56.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.709+0000 7f2a044d5640 1 -- 172.21.15.60:0/61526844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29fc0fd120 con 0x7f29fc101c80 2024-02-26T04:55:56.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.710+0000 7f29f37fe640 1 -- 172.21.15.60:0/61526844 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f29f80035f0 con 0x7f29fc101c80 2024-02-26T04:55:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.710+0000 7f29f37fe640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f29e80a2150 0x7f29e80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.711+0000 7f2a01a70640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f29e80a2150 0x7f29e80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.711+0000 7f29f37fe640 1 -- 172.21.15.60:0/61526844 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f29f80f0420 con 0x7f29fc101c80 2024-02-26T04:55:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.711+0000 7f2a01a70640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f29e80a2150 0x7f29e80a4610 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f29ec005d00 tx=0x7f29ec005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:56.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.715+0000 7f29f37fe640 1 -- 172.21.15.60:0/61526844 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f29f80b9100 con 0x7f29fc101c80 2024-02-26T04:55:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.892+0000 7f2a044d5640 1 -- 172.21.15.60:0/61526844 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29fc02ceb0 con 0x7f29e80a2150 2024-02-26T04:55:56.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.905+0000 7f29f37fe640 1 -- 172.21.15.60:0/61526844 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423487 (secure 0 0 0) 0x7f29fc02ceb0 con 0x7f29e80a2150 2024-02-26T04:55:56.915 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:56.918 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-02-26T04:55:56.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.909+0000 7f2a044d5640 1 -- 172.21.15.60:0/61526844 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f29e80a2150 msgr2=0x7f29e80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.909+0000 7f2a044d5640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f29e80a2150 0x7f29e80a4610 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f29ec005d00 tx=0x7f29ec005ed0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.909+0000 7f2a044d5640 1 -- 172.21.15.60:0/61526844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29fc101c80 msgr2=0x7f29fc1015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:56.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.909+0000 7f2a044d5640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29fc101c80 0x7f29fc1015f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f29f8036040 tx=0x7f29f80042f0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.909+0000 7f2a044d5640 1 -- 172.21.15.60:0/61526844 shutdown_connections 2024-02-26T04:55:56.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.909+0000 7f2a044d5640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f29e80a2150 0x7f29e80a4610 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.909+0000 7f2a044d5640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29fc106ee0 0x7f29fc100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.909+0000 7f2a044d5640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29fc1045b0 0x7f29fc0ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.909+0000 7f2a044d5640 1 --2- 172.21.15.60:0/61526844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29fc101c80 0x7f29fc1015f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.909+0000 7f2a044d5640 1 -- 172.21.15.60:0/61526844 >> 172.21.15.60:0/61526844 conn(0x7f29fc0fb630 msgr2=0x7f29fc1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.909+0000 7f2a044d5640 1 -- 172.21.15.60:0/61526844 shutdown_connections 2024-02-26T04:55:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:56.909+0000 7f2a044d5640 1 -- 172.21.15.60:0/61526844 wait complete. 2024-02-26T04:55:56.922 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-02-26T04:55:56.312173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604240,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":148,"num_read_kb":129,"num_write":297,"num_write_kb":2988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381612,"kb_used_data":11624,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560212,"statfs":{"total":1151940427776,"available":1151549657088,"internally_reserved":0,"allocated":11902976,"data_stored":7535777,"data_compressed":282837,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11013,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":22,"num_read_kb":20,"num_write":70,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003285"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770611+0000","last_change":"2024-02-26T04:55:28.208832+0000","last_active":"2024-02-26T04:55:33.770611+0000","last_peered":"2024-02-26T04:55:33.770611+0000","last_clean":"2024-02-26T04:55:33.770611+0000","last_became_active":"2024-02-26T04:55:28.208454+0000","last_became_peered":"2024-02-26T04:55:28.208454+0000","last_unstale":"2024-02-26T04:55:33.770611+0000","last_undegraded":"2024-02-26T04:55:33.770611+0000","last_fullsized":"2024-02-26T04:55:33.770611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:55:54.132384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.214555+0000","last_change":"2024-02-26T04:55:28.203876+0000","last_active":"2024-02-26T04:55:33.214555+0000","last_peered":"2024-02-26T04:55:33.214555+0000","last_clean":"2024-02-26T04:55:33.214555+0000","last_became_active":"2024-02-26T04:55:28.203399+0000","last_became_peered":"2024-02-26T04:55:28.203399+0000","last_unstale":"2024-02-26T04:55:33.214555+0000","last_undegraded":"2024-02-26T04:55:33.214555+0000","last_fullsized":"2024-02-26T04:55:33.214555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:31:40.927929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.214285+0000","last_change":"2024-02-26T04:55:28.211388+0000","last_active":"2024-02-26T04:55:33.214285+0000","last_peered":"2024-02-26T04:55:33.214285+0000","last_clean":"2024-02-26T04:55:33.214285+0000","last_became_active":"2024-02-26T04:55:28.209007+0000","last_became_peered":"2024-02-26T04:55:28.209007+0000","last_unstale":"2024-02-26T04:55:33.214285+0000","last_undegraded":"2024-02-26T04:55:33.214285+0000","last_fullsized":"2024-02-26T04:55:33.214285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:37:01.197589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931553+0000","last_change":"2024-02-26T04:55:28.216378+0000","last_active":"2024-02-26T04:55:33.931553+0000","last_peered":"2024-02-26T04:55:33.931553+0000","last_clean":"2024-02-26T04:55:33.931553+0000","last_became_active":"2024-02-26T04:55:28.216074+0000","last_became_peered":"2024-02-26T04:55:28.216074+0000","last_unstale":"2024-02-26T04:55:33.931553+0000","last_undegraded":"2024-02-26T04:55:33.931553+0000","last_fullsized":"2024-02-26T04:55:33.931553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:15:20.460699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134370+0000","last_change":"2024-02-26T04:55:28.213026+0000","last_active":"2024-02-26T04:55:34.134370+0000","last_peered":"2024-02-26T04:55:34.134370+0000","last_clean":"2024-02-26T04:55:34.134370+0000","last_became_active":"2024-02-26T04:55:28.212706+0000","last_became_peered":"2024-02-26T04:55:28.212706+0000","last_unstale":"2024-02-26T04:55:34.134370+0000","last_undegraded":"2024-02-26T04:55:34.134370+0000","last_fullsized":"2024-02-26T04:55:34.134370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:57:35.961956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.117804+0000","last_change":"2024-02-26T04:55:28.202123+0000","last_active":"2024-02-26T04:55:34.117804+0000","last_peered":"2024-02-26T04:55:34.117804+0000","last_clean":"2024-02-26T04:55:34.117804+0000","last_became_active":"2024-02-26T04:55:28.201733+0000","last_became_peered":"2024-02-26T04:55:28.201733+0000","last_unstale":"2024-02-26T04:55:34.117804+0000","last_undegraded":"2024-02-26T04:55:34.117804+0000","last_fullsized":"2024-02-26T04:55:34.117804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:43:11.300006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213167+0000","last_change":"2024-02-26T04:55:28.199591+0000","last_active":"2024-02-26T04:55:33.213167+0000","last_peered":"2024-02-26T04:55:33.213167+0000","last_clean":"2024-02-26T04:55:33.213167+0000","last_became_active":"2024-02-26T04:55:28.199139+0000","last_became_peered":"2024-02-26T04:55:28.199139+0000","last_unstale":"2024-02-26T04:55:33.213167+0000","last_undegraded":"2024-02-26T04:55:33.213167+0000","last_fullsized":"2024-02-26T04:55:33.213167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:43:22.177447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.133805+0000","last_change":"2024-02-26T04:55:28.214631+0000","last_active":"2024-02-26T04:55:34.133805+0000","last_peered":"2024-02-26T04:55:34.133805+0000","last_clean":"2024-02-26T04:55:34.133805+0000","last_became_active":"2024-02-26T04:55:28.214290+0000","last_became_peered":"2024-02-26T04:55:28.214290+0000","last_unstale":"2024-02-26T04:55:34.133805+0000","last_undegraded":"2024-02-26T04:55:34.133805+0000","last_fullsized":"2024-02-26T04:55:34.133805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:30:11.494833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134548+0000","last_change":"2024-02-26T04:55:28.212808+0000","last_active":"2024-02-26T04:55:34.134548+0000","last_peered":"2024-02-26T04:55:34.134548+0000","last_clean":"2024-02-26T04:55:34.134548+0000","last_became_active":"2024-02-26T04:55:28.212331+0000","last_became_peered":"2024-02-26T04:55:28.212331+0000","last_unstale":"2024-02-26T04:55:34.134548+0000","last_undegraded":"2024-02-26T04:55:34.134548+0000","last_fullsized":"2024-02-26T04:55:34.134548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:27:02.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.118306+0000","last_change":"2024-02-26T04:55:28.204176+0000","last_active":"2024-02-26T04:55:34.118306+0000","last_peered":"2024-02-26T04:55:34.118306+0000","last_clean":"2024-02-26T04:55:34.118306+0000","last_became_active":"2024-02-26T04:55:28.203821+0000","last_became_peered":"2024-02-26T04:55:28.203821+0000","last_unstale":"2024-02-26T04:55:34.118306+0000","last_undegraded":"2024-02-26T04:55:34.118306+0000","last_fullsized":"2024-02-26T04:55:34.118306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:24:03.132650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213828+0000","last_change":"2024-02-26T04:55:28.215958+0000","last_active":"2024-02-26T04:55:33.213828+0000","last_peered":"2024-02-26T04:55:33.213828+0000","last_clean":"2024-02-26T04:55:33.213828+0000","last_became_active":"2024-02-26T04:55:28.215594+0000","last_became_peered":"2024-02-26T04:55:28.215594+0000","last_unstale":"2024-02-26T04:55:33.213828+0000","last_undegraded":"2024-02-26T04:55:33.213828+0000","last_fullsized":"2024-02-26T04:55:33.213828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:45:47.915379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931492+0000","last_change":"2024-02-26T04:55:28.218392+0000","last_active":"2024-02-26T04:55:33.931492+0000","last_peered":"2024-02-26T04:55:33.931492+0000","last_clean":"2024-02-26T04:55:33.931492+0000","last_became_active":"2024-02-26T04:55:28.217933+0000","last_became_peered":"2024-02-26T04:55:28.217933+0000","last_unstale":"2024-02-26T04:55:33.931492+0000","last_undegraded":"2024-02-26T04:55:33.931492+0000","last_fullsized":"2024-02-26T04:55:33.931492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:30:38.880649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213476+0000","last_change":"2024-02-26T04:55:28.197099+0000","last_active":"2024-02-26T04:55:33.213476+0000","last_peered":"2024-02-26T04:55:33.213476+0000","last_clean":"2024-02-26T04:55:33.213476+0000","last_became_active":"2024-02-26T04:55:28.196755+0000","last_became_peered":"2024-02-26T04:55:28.196755+0000","last_unstale":"2024-02-26T04:55:33.213476+0000","last_undegraded":"2024-02-26T04:55:33.213476+0000","last_fullsized":"2024-02-26T04:55:33.213476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:37:31.685845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213634+0000","last_change":"2024-02-26T04:55:28.212064+0000","last_active":"2024-02-26T04:55:33.213634+0000","last_peered":"2024-02-26T04:55:33.213634+0000","last_clean":"2024-02-26T04:55:33.213634+0000","last_became_active":"2024-02-26T04:55:28.208742+0000","last_became_peered":"2024-02-26T04:55:28.208742+0000","last_unstale":"2024-02-26T04:55:33.213634+0000","last_undegraded":"2024-02-26T04:55:33.213634+0000","last_fullsized":"2024-02-26T04:55:33.213634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:38:19.742468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213988+0000","last_change":"2024-02-26T04:55:28.212408+0000","last_active":"2024-02-26T04:55:33.213988+0000","last_peered":"2024-02-26T04:55:33.213988+0000","last_clean":"2024-02-26T04:55:33.213988+0000","last_became_active":"2024-02-26T04:55:28.211391+0000","last_became_peered":"2024-02-26T04:55:28.211391+0000","last_unstale":"2024-02-26T04:55:33.213988+0000","last_undegraded":"2024-02-26T04:55:33.213988+0000","last_fullsized":"2024-02-26T04:55:33.213988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:27:37.957529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.133908+0000","last_change":"2024-02-26T04:55:28.222810+0000","last_active":"2024-02-26T04:55:34.133908+0000","last_peered":"2024-02-26T04:55:34.133908+0000","last_clean":"2024-02-26T04:55:34.133908+0000","last_became_active":"2024-02-26T04:55:28.222341+0000","last_became_peered":"2024-02-26T04:55:28.222341+0000","last_unstale":"2024-02-26T04:55:34.133908+0000","last_undegraded":"2024-02-26T04:55:34.133908+0000","last_fullsized":"2024-02-26T04:55:34.133908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:59:21.627517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770476+0000","last_change":"2024-02-26T04:55:28.208377+0000","last_active":"2024-02-26T04:55:33.770476+0000","last_peered":"2024-02-26T04:55:33.770476+0000","last_clean":"2024-02-26T04:55:33.770476+0000","last_became_active":"2024-02-26T04:55:28.207932+0000","last_became_peered":"2024-02-26T04:55:28.207932+0000","last_unstale":"2024-02-26T04:55:33.770476+0000","last_undegraded":"2024-02-26T04:55:33.770476+0000","last_fullsized":"2024-02-26T04:55:33.770476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:15:39.586191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134464+0000","last_change":"2024-02-26T04:55:28.201105+0000","last_active":"2024-02-26T04:55:34.134464+0000","last_peered":"2024-02-26T04:55:34.134464+0000","last_clean":"2024-02-26T04:55:34.134464+0000","last_became_active":"2024-02-26T04:55:28.200796+0000","last_became_peered":"2024-02-26T04:55:28.200796+0000","last_unstale":"2024-02-26T04:55:34.134464+0000","last_undegraded":"2024-02-26T04:55:34.134464+0000","last_fullsized":"2024-02-26T04:55:34.134464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:53:43.713055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.105300+0000","last_change":"2024-02-26T04:55:28.215725+0000","last_active":"2024-02-26T04:55:34.105300+0000","last_peered":"2024-02-26T04:55:34.105300+0000","last_clean":"2024-02-26T04:55:34.105300+0000","last_became_active":"2024-02-26T04:55:28.215341+0000","last_became_peered":"2024-02-26T04:55:28.215341+0000","last_unstale":"2024-02-26T04:55:34.105300+0000","last_undegraded":"2024-02-26T04:55:34.105300+0000","last_fullsized":"2024-02-26T04:55:34.105300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:26:26.960474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.104999+0000","last_change":"2024-02-26T04:55:32.233303+0000","last_active":"2024-02-26T04:55:34.104999+0000","last_peered":"2024-02-26T04:55:34.104999+0000","last_clean":"2024-02-26T04:55:34.104999+0000","last_became_active":"2024-02-26T04:55:32.232901+0000","last_became_peered":"2024-02-26T04:55:32.232901+0000","last_unstale":"2024-02-26T04:55:34.104999+0000","last_undegraded":"2024-02-26T04:55:34.104999+0000","last_fullsized":"2024-02-26T04:55:34.104999+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:37:49.659668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:53.550443+0000","last_change":"2024-02-26T04:55:27.199682+0000","last_active":"2024-02-26T04:55:53.550443+0000","last_peered":"2024-02-26T04:55:53.550443+0000","last_clean":"2024-02-26T04:55:53.550443+0000","last_became_active":"2024-02-26T04:55:27.199293+0000","last_became_peered":"2024-02-26T04:55:27.199293+0000","last_unstale":"2024-02-26T04:55:53.550443+0000","last_undegraded":"2024-02-26T04:55:53.550443+0000","last_fullsized":"2024-02-26T04:55:53.550443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:23:21.685615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134361+0000","last_change":"2024-02-26T04:55:28.205365+0000","last_active":"2024-02-26T04:55:34.134361+0000","last_peered":"2024-02-26T04:55:34.134361+0000","last_clean":"2024-02-26T04:55:34.134361+0000","last_became_active":"2024-02-26T04:55:28.204980+0000","last_became_peered":"2024-02-26T04:55:28.204980+0000","last_unstale":"2024-02-26T04:55:34.134361+0000","last_undegraded":"2024-02-26T04:55:34.134361+0000","last_fullsized":"2024-02-26T04:55:34.134361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:57:24.148691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931178+0000","last_change":"2024-02-26T04:55:32.233142+0000","last_active":"2024-02-26T04:55:33.931178+0000","last_peered":"2024-02-26T04:55:33.931178+0000","last_clean":"2024-02-26T04:55:33.931178+0000","last_became_active":"2024-02-26T04:55:32.232722+0000","last_became_peered":"2024-02-26T04:55:32.232722+0000","last_unstale":"2024-02-26T04:55:33.931178+0000","last_undegraded":"2024-02-26T04:55:33.931178+0000","last_fullsized":"2024-02-26T04:55:33.931178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:30:20.906104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134682+0000","last_change":"2024-02-26T04:55:27.203420+0000","last_active":"2024-02-26T04:55:34.134682+0000","last_peered":"2024-02-26T04:55:34.134682+0000","last_clean":"2024-02-26T04:55:34.134682+0000","last_became_active":"2024-02-26T04:55:27.202968+0000","last_became_peered":"2024-02-26T04:55:27.202968+0000","last_unstale":"2024-02-26T04:55:34.134682+0000","last_undegraded":"2024-02-26T04:55:34.134682+0000","last_fullsized":"2024-02-26T04:55:34.134682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:44.496528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.214483+0000","last_change":"2024-02-26T04:55:28.217422+0000","last_active":"2024-02-26T04:55:33.214483+0000","last_peered":"2024-02-26T04:55:33.214483+0000","last_clean":"2024-02-26T04:55:33.214483+0000","last_became_active":"2024-02-26T04:55:28.217103+0000","last_became_peered":"2024-02-26T04:55:28.217103+0000","last_unstale":"2024-02-26T04:55:33.214483+0000","last_undegraded":"2024-02-26T04:55:33.214483+0000","last_fullsized":"2024-02-26T04:55:33.214483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:08:37.075467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931314+0000","last_change":"2024-02-26T04:55:32.233322+0000","last_active":"2024-02-26T04:55:33.931314+0000","last_peered":"2024-02-26T04:55:33.931314+0000","last_clean":"2024-02-26T04:55:33.931314+0000","last_became_active":"2024-02-26T04:55:32.232679+0000","last_became_peered":"2024-02-26T04:55:32.232679+0000","last_unstale":"2024-02-26T04:55:33.931314+0000","last_undegraded":"2024-02-26T04:55:33.931314+0000","last_fullsized":"2024-02-26T04:55:33.931314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:10.775924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:53.547570+0000","last_change":"2024-02-26T04:55:27.205960+0000","last_active":"2024-02-26T04:55:53.547570+0000","last_peered":"2024-02-26T04:55:53.547570+0000","last_clean":"2024-02-26T04:55:53.547570+0000","last_became_active":"2024-02-26T04:55:27.205604+0000","last_became_peered":"2024-02-26T04:55:27.205604+0000","last_unstale":"2024-02-26T04:55:53.547570+0000","last_undegraded":"2024-02-26T04:55:53.547570+0000","last_fullsized":"2024-02-26T04:55:53.547570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:45:25.002672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134236+0000","last_change":"2024-02-26T04:55:28.210669+0000","last_active":"2024-02-26T04:55:34.134236+0000","last_peered":"2024-02-26T04:55:34.134236+0000","last_clean":"2024-02-26T04:55:34.134236+0000","last_became_active":"2024-02-26T04:55:28.210322+0000","last_became_peered":"2024-02-26T04:55:28.210322+0000","last_unstale":"2024-02-26T04:55:34.134236+0000","last_undegraded":"2024-02-26T04:55:34.134236+0000","last_fullsized":"2024-02-26T04:55:34.134236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:57:29.121762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931225+0000","last_change":"2024-02-26T04:55:32.229077+0000","last_active":"2024-02-26T04:55:33.931225+0000","last_peered":"2024-02-26T04:55:33.931225+0000","last_clean":"2024-02-26T04:55:33.931225+0000","last_became_active":"2024-02-26T04:55:32.228670+0000","last_became_peered":"2024-02-26T04:55:32.228670+0000","last_unstale":"2024-02-26T04:55:33.931225+0000","last_undegraded":"2024-02-26T04:55:33.931225+0000","last_fullsized":"2024-02-26T04:55:33.931225+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:35:20.843351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134508+0000","last_change":"2024-02-26T04:55:27.206088+0000","last_active":"2024-02-26T04:55:34.134508+0000","last_peered":"2024-02-26T04:55:34.134508+0000","last_clean":"2024-02-26T04:55:34.134508+0000","last_became_active":"2024-02-26T04:55:27.205696+0000","last_became_peered":"2024-02-26T04:55:27.205696+0000","last_unstale":"2024-02-26T04:55:34.134508+0000","last_undegraded":"2024-02-26T04:55:34.134508+0000","last_fullsized":"2024-02-26T04:55:34.134508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:14:03.552251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134860+0000","last_change":"2024-02-26T04:55:28.215124+0000","last_active":"2024-02-26T04:55:34.134860+0000","last_peered":"2024-02-26T04:55:34.134860+0000","last_clean":"2024-02-26T04:55:34.134860+0000","last_became_active":"2024-02-26T04:55:28.214618+0000","last_became_peered":"2024-02-26T04:55:28.214618+0000","last_unstale":"2024-02-26T04:55:34.134860+0000","last_undegraded":"2024-02-26T04:55:34.134860+0000","last_fullsized":"2024-02-26T04:55:34.134860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:52:25.581632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.105233+0000","last_change":"2024-02-26T04:55:32.228629+0000","last_active":"2024-02-26T04:55:34.105233+0000","last_peered":"2024-02-26T04:55:34.105233+0000","last_clean":"2024-02-26T04:55:34.105233+0000","last_became_active":"2024-02-26T04:55:32.227608+0000","last_became_peered":"2024-02-26T04:55:32.227608+0000","last_unstale":"2024-02-26T04:55:34.105233+0000","last_undegraded":"2024-02-26T04:55:34.105233+0000","last_fullsized":"2024-02-26T04:55:34.105233+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:10:42.429352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:54.151407+0000","last_change":"2024-02-26T04:55:27.201261+0000","last_active":"2024-02-26T04:55:54.151407+0000","last_peered":"2024-02-26T04:55:54.151407+0000","last_clean":"2024-02-26T04:55:54.151407+0000","last_became_active":"2024-02-26T04:55:27.200892+0000","last_became_peered":"2024-02-26T04:55:27.200892+0000","last_unstale":"2024-02-26T04:55:54.151407+0000","last_undegraded":"2024-02-26T04:55:54.151407+0000","last_fullsized":"2024-02-26T04:55:54.151407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:34:32.774143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134974+0000","last_change":"2024-02-26T04:55:28.200445+0000","last_active":"2024-02-26T04:55:34.134974+0000","last_peered":"2024-02-26T04:55:34.134974+0000","last_clean":"2024-02-26T04:55:34.134974+0000","last_became_active":"2024-02-26T04:55:28.199959+0000","last_became_peered":"2024-02-26T04:55:28.199959+0000","last_unstale":"2024-02-26T04:55:34.134974+0000","last_undegraded":"2024-02-26T04:55:34.134974+0000","last_fullsized":"2024-02-26T04:55:34.134974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:36:27.397644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.105171+0000","last_change":"2024-02-26T04:55:32.232619+0000","last_active":"2024-02-26T04:55:34.105171+0000","last_peered":"2024-02-26T04:55:34.105171+0000","last_clean":"2024-02-26T04:55:34.105171+0000","last_became_active":"2024-02-26T04:55:32.232112+0000","last_became_peered":"2024-02-26T04:55:32.232112+0000","last_unstale":"2024-02-26T04:55:34.105171+0000","last_undegraded":"2024-02-26T04:55:34.105171+0000","last_fullsized":"2024-02-26T04:55:34.105171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:19.741393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:53.548966+0000","last_change":"2024-02-26T04:55:27.197376+0000","last_active":"2024-02-26T04:55:53.548966+0000","last_peered":"2024-02-26T04:55:53.548966+0000","last_clean":"2024-02-26T04:55:53.548966+0000","last_became_active":"2024-02-26T04:55:27.195258+0000","last_became_peered":"2024-02-26T04:55:27.195258+0000","last_unstale":"2024-02-26T04:55:53.548966+0000","last_undegraded":"2024-02-26T04:55:53.548966+0000","last_fullsized":"2024-02-26T04:55:53.548966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:41:51.284525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134542+0000","last_change":"2024-02-26T04:55:28.214337+0000","last_active":"2024-02-26T04:55:34.134542+0000","last_peered":"2024-02-26T04:55:34.134542+0000","last_clean":"2024-02-26T04:55:34.134542+0000","last_became_active":"2024-02-26T04:55:28.213966+0000","last_became_peered":"2024-02-26T04:55:28.213966+0000","last_unstale":"2024-02-26T04:55:34.134542+0000","last_undegraded":"2024-02-26T04:55:34.134542+0000","last_fullsized":"2024-02-26T04:55:34.134542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:42.974084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770538+0000","last_change":"2024-02-26T04:55:32.235579+0000","last_active":"2024-02-26T04:55:33.770538+0000","last_peered":"2024-02-26T04:55:33.770538+0000","last_clean":"2024-02-26T04:55:33.770538+0000","last_became_active":"2024-02-26T04:55:32.235241+0000","last_became_peered":"2024-02-26T04:55:32.235241+0000","last_unstale":"2024-02-26T04:55:33.770538+0000","last_undegraded":"2024-02-26T04:55:33.770538+0000","last_fullsized":"2024-02-26T04:55:33.770538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:06.319874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.212853+0000","last_change":"2024-02-26T04:55:27.197689+0000","last_active":"2024-02-26T04:55:33.212853+0000","last_peered":"2024-02-26T04:55:33.212853+0000","last_clean":"2024-02-26T04:55:33.212853+0000","last_became_active":"2024-02-26T04:55:27.197285+0000","last_became_peered":"2024-02-26T04:55:27.197285+0000","last_unstale":"2024-02-26T04:55:33.212853+0000","last_undegraded":"2024-02-26T04:55:33.212853+0000","last_fullsized":"2024-02-26T04:55:33.212853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:45:36.688902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.133692+0000","last_change":"2024-02-26T04:55:28.201902+0000","last_active":"2024-02-26T04:55:34.133692+0000","last_peered":"2024-02-26T04:55:34.133692+0000","last_clean":"2024-02-26T04:55:34.133692+0000","last_became_active":"2024-02-26T04:55:28.201604+0000","last_became_peered":"2024-02-26T04:55:28.201604+0000","last_unstale":"2024-02-26T04:55:34.133692+0000","last_undegraded":"2024-02-26T04:55:34.133692+0000","last_fullsized":"2024-02-26T04:55:34.133692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:48:57.711555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.214449+0000","last_change":"2024-02-26T04:55:32.230153+0000","last_active":"2024-02-26T04:55:33.214449+0000","last_peered":"2024-02-26T04:55:33.214449+0000","last_clean":"2024-02-26T04:55:33.214449+0000","last_became_active":"2024-02-26T04:55:32.229695+0000","last_became_peered":"2024-02-26T04:55:32.229695+0000","last_unstale":"2024-02-26T04:55:33.214449+0000","last_undegraded":"2024-02-26T04:55:33.214449+0000","last_fullsized":"2024-02-26T04:55:33.214449+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:15:39.575727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134355+0000","last_change":"2024-02-26T04:55:27.200943+0000","last_active":"2024-02-26T04:55:34.134355+0000","last_peered":"2024-02-26T04:55:34.134355+0000","last_clean":"2024-02-26T04:55:34.134355+0000","last_became_active":"2024-02-26T04:55:27.200565+0000","last_became_peered":"2024-02-26T04:55:27.200565+0000","last_unstale":"2024-02-26T04:55:34.134355+0000","last_undegraded":"2024-02-26T04:55:34.134355+0000","last_fullsized":"2024-02-26T04:55:34.134355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:54:19.670631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134026+0000","last_change":"2024-02-26T04:55:28.212232+0000","last_active":"2024-02-26T04:55:34.134026+0000","last_peered":"2024-02-26T04:55:34.134026+0000","last_clean":"2024-02-26T04:55:34.134026+0000","last_became_active":"2024-02-26T04:55:28.211875+0000","last_became_peered":"2024-02-26T04:55:28.211875+0000","last_unstale":"2024-02-26T04:55:34.134026+0000","last_undegraded":"2024-02-26T04:55:34.134026+0000","last_fullsized":"2024-02-26T04:55:34.134026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:55.379271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.133802+0000","last_change":"2024-02-26T04:55:32.234961+0000","last_active":"2024-02-26T04:55:34.133802+0000","last_peered":"2024-02-26T04:55:34.133802+0000","last_clean":"2024-02-26T04:55:34.133802+0000","last_became_active":"2024-02-26T04:55:32.234455+0000","last_became_peered":"2024-02-26T04:55:32.234455+0000","last_unstale":"2024-02-26T04:55:34.133802+0000","last_undegraded":"2024-02-26T04:55:34.133802+0000","last_fullsized":"2024-02-26T04:55:34.133802+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:00:44.322341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:52.544122+0000","last_change":"2024-02-26T04:55:27.199036+0000","last_active":"2024-02-26T04:55:52.544122+0000","last_peered":"2024-02-26T04:55:52.544122+0000","last_clean":"2024-02-26T04:55:52.544122+0000","last_became_active":"2024-02-26T04:55:27.198694+0000","last_became_peered":"2024-02-26T04:55:27.198694+0000","last_unstale":"2024-02-26T04:55:52.544122+0000","last_undegraded":"2024-02-26T04:55:52.544122+0000","last_fullsized":"2024-02-26T04:55:52.544122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:31:41.935123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134487+0000","last_change":"2024-02-26T04:55:28.209330+0000","last_active":"2024-02-26T04:55:34.134487+0000","last_peered":"2024-02-26T04:55:34.134487+0000","last_clean":"2024-02-26T04:55:34.134487+0000","last_became_active":"2024-02-26T04:55:28.208974+0000","last_became_peered":"2024-02-26T04:55:28.208974+0000","last_unstale":"2024-02-26T04:55:34.134487+0000","last_undegraded":"2024-02-26T04:55:34.134487+0000","last_fullsized":"2024-02-26T04:55:34.134487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:21:54.924526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.133990+0000","last_change":"2024-02-26T04:55:32.228033+0000","last_active":"2024-02-26T04:55:34.133990+0000","last_peered":"2024-02-26T04:55:34.133990+0000","last_clean":"2024-02-26T04:55:34.133990+0000","last_became_active":"2024-02-26T04:55:32.224273+0000","last_became_peered":"2024-02-26T04:55:32.224273+0000","last_unstale":"2024-02-26T04:55:34.133990+0000","last_undegraded":"2024-02-26T04:55:34.133990+0000","last_fullsized":"2024-02-26T04:55:34.133990+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:25:13.618421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213651+0000","last_change":"2024-02-26T04:55:27.202833+0000","last_active":"2024-02-26T04:55:33.213651+0000","last_peered":"2024-02-26T04:55:33.213651+0000","last_clean":"2024-02-26T04:55:33.213651+0000","last_became_active":"2024-02-26T04:55:27.202167+0000","last_became_peered":"2024-02-26T04:55:27.202167+0000","last_unstale":"2024-02-26T04:55:33.213651+0000","last_undegraded":"2024-02-26T04:55:33.213651+0000","last_fullsized":"2024-02-26T04:55:33.213651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:55:46.605763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.214410+0000","last_change":"2024-02-26T04:55:28.212055+0000","last_active":"2024-02-26T04:55:33.214410+0000","last_peered":"2024-02-26T04:55:33.214410+0000","last_clean":"2024-02-26T04:55:33.214410+0000","last_became_active":"2024-02-26T04:55:28.211233+0000","last_became_peered":"2024-02-26T04:55:28.211233+0000","last_unstale":"2024-02-26T04:55:33.214410+0000","last_undegraded":"2024-02-26T04:55:33.214410+0000","last_fullsized":"2024-02-26T04:55:33.214410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:31:35.300693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931616+0000","last_change":"2024-02-26T04:55:32.225736+0000","last_active":"2024-02-26T04:55:33.931616+0000","last_peered":"2024-02-26T04:55:33.931616+0000","last_clean":"2024-02-26T04:55:33.931616+0000","last_became_active":"2024-02-26T04:55:32.225186+0000","last_became_peered":"2024-02-26T04:55:32.225186+0000","last_unstale":"2024-02-26T04:55:33.931616+0000","last_undegraded":"2024-02-26T04:55:33.931616+0000","last_fullsized":"2024-02-26T04:55:33.931616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:13:48.999538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:52.542965+0000","last_change":"2024-02-26T04:55:27.200389+0000","last_active":"2024-02-26T04:55:52.542965+0000","last_peered":"2024-02-26T04:55:52.542965+0000","last_clean":"2024-02-26T04:55:52.542965+0000","last_became_active":"2024-02-26T04:55:27.200019+0000","last_became_peered":"2024-02-26T04:55:27.200019+0000","last_unstale":"2024-02-26T04:55:52.542965+0000","last_undegraded":"2024-02-26T04:55:52.542965+0000","last_fullsized":"2024-02-26T04:55:52.542965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:59:11.820988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134429+0000","last_change":"2024-02-26T04:55:28.220309+0000","last_active":"2024-02-26T04:55:34.134429+0000","last_peered":"2024-02-26T04:55:34.134429+0000","last_clean":"2024-02-26T04:55:34.134429+0000","last_became_active":"2024-02-26T04:55:28.219987+0000","last_became_peered":"2024-02-26T04:55:28.219987+0000","last_unstale":"2024-02-26T04:55:34.134429+0000","last_undegraded":"2024-02-26T04:55:34.134429+0000","last_fullsized":"2024-02-26T04:55:34.134429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:39:34.713802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134159+0000","last_change":"2024-02-26T04:55:32.236817+0000","last_active":"2024-02-26T04:55:34.134159+0000","last_peered":"2024-02-26T04:55:34.134159+0000","last_clean":"2024-02-26T04:55:34.134159+0000","last_became_active":"2024-02-26T04:55:32.235818+0000","last_became_peered":"2024-02-26T04:55:32.235818+0000","last_unstale":"2024-02-26T04:55:34.134159+0000","last_undegraded":"2024-02-26T04:55:34.134159+0000","last_fullsized":"2024-02-26T04:55:34.134159+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:56:54.121684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'15","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:54.493399+0000","last_change":"2024-02-26T04:55:27.201582+0000","last_active":"2024-02-26T04:55:54.493399+0000","last_peered":"2024-02-26T04:55:54.493399+0000","last_clean":"2024-02-26T04:55:54.493399+0000","last_became_active":"2024-02-26T04:55:27.201011+0000","last_became_peered":"2024-02-26T04:55:27.201011+0000","last_unstale":"2024-02-26T04:55:54.493399+0000","last_undegraded":"2024-02-26T04:55:54.493399+0000","last_fullsized":"2024-02-26T04:55:54.493399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:07:01.888924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10442,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134795+0000","last_change":"2024-02-26T04:55:28.199313+0000","last_active":"2024-02-26T04:55:34.134795+0000","last_peered":"2024-02-26T04:55:34.134795+0000","last_clean":"2024-02-26T04:55:34.134795+0000","last_became_active":"2024-02-26T04:55:28.198973+0000","last_became_peered":"2024-02-26T04:55:28.198973+0000","last_unstale":"2024-02-26T04:55:34.134795+0000","last_undegraded":"2024-02-26T04:55:34.134795+0000","last_fullsized":"2024-02-26T04:55:34.134795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:33:49.005899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.135331+0000","last_change":"2024-02-26T04:55:32.234529+0000","last_active":"2024-02-26T04:55:34.135331+0000","last_peered":"2024-02-26T04:55:34.135331+0000","last_clean":"2024-02-26T04:55:34.135331+0000","last_became_active":"2024-02-26T04:55:32.234105+0000","last_became_peered":"2024-02-26T04:55:32.234105+0000","last_unstale":"2024-02-26T04:55:34.135331+0000","last_undegraded":"2024-02-26T04:55:34.135331+0000","last_fullsized":"2024-02-26T04:55:34.135331+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:17:40.909267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:53.549314+0000","last_change":"2024-02-26T04:55:27.209178+0000","last_active":"2024-02-26T04:55:53.549314+0000","last_peered":"2024-02-26T04:55:53.549314+0000","last_clean":"2024-02-26T04:55:53.549314+0000","last_became_active":"2024-02-26T04:55:27.208834+0000","last_became_peered":"2024-02-26T04:55:27.208834+0000","last_unstale":"2024-02-26T04:55:53.549314+0000","last_undegraded":"2024-02-26T04:55:53.549314+0000","last_fullsized":"2024-02-26T04:55:53.549314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:28.187333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213044+0000","last_change":"2024-02-26T04:55:28.215208+0000","last_active":"2024-02-26T04:55:33.213044+0000","last_peered":"2024-02-26T04:55:33.213044+0000","last_clean":"2024-02-26T04:55:33.213044+0000","last_became_active":"2024-02-26T04:55:28.212135+0000","last_became_peered":"2024-02-26T04:55:28.212135+0000","last_unstale":"2024-02-26T04:55:33.213044+0000","last_undegraded":"2024-02-26T04:55:33.213044+0000","last_fullsized":"2024-02-26T04:55:33.213044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:51.269557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770214+0000","last_change":"2024-02-26T04:55:32.235439+0000","last_active":"2024-02-26T04:55:33.770214+0000","last_peered":"2024-02-26T04:55:33.770214+0000","last_clean":"2024-02-26T04:55:33.770214+0000","last_became_active":"2024-02-26T04:55:32.234987+0000","last_became_peered":"2024-02-26T04:55:32.234987+0000","last_unstale":"2024-02-26T04:55:33.770214+0000","last_undegraded":"2024-02-26T04:55:33.770214+0000","last_fullsized":"2024-02-26T04:55:33.770214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:31:00.637846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:52.543737+0000","last_change":"2024-02-26T04:55:27.205498+0000","last_active":"2024-02-26T04:55:52.543737+0000","last_peered":"2024-02-26T04:55:52.543737+0000","last_clean":"2024-02-26T04:55:52.543737+0000","last_became_active":"2024-02-26T04:55:27.205179+0000","last_became_peered":"2024-02-26T04:55:27.205179+0000","last_unstale":"2024-02-26T04:55:52.543737+0000","last_undegraded":"2024-02-26T04:55:52.543737+0000","last_fullsized":"2024-02-26T04:55:52.543737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:02:55.491896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213479+0000","last_change":"2024-02-26T04:55:28.201110+0000","last_active":"2024-02-26T04:55:33.213479+0000","last_peered":"2024-02-26T04:55:33.213479+0000","last_clean":"2024-02-26T04:55:33.213479+0000","last_became_active":"2024-02-26T04:55:28.200725+0000","last_became_peered":"2024-02-26T04:55:28.200725+0000","last_unstale":"2024-02-26T04:55:33.213479+0000","last_undegraded":"2024-02-26T04:55:33.213479+0000","last_fullsized":"2024-02-26T04:55:33.213479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:50.377925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134816+0000","last_change":"2024-02-26T04:55:32.240328+0000","last_active":"2024-02-26T04:55:34.134816+0000","last_peered":"2024-02-26T04:55:34.134816+0000","last_clean":"2024-02-26T04:55:34.134816+0000","last_became_active":"2024-02-26T04:55:32.238662+0000","last_became_peered":"2024-02-26T04:55:32.238662+0000","last_unstale":"2024-02-26T04:55:34.134816+0000","last_undegraded":"2024-02-26T04:55:34.134816+0000","last_fullsized":"2024-02-26T04:55:34.134816+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:20:22.938290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:54.151933+0000","last_change":"2024-02-26T04:55:27.197338+0000","last_active":"2024-02-26T04:55:54.151933+0000","last_peered":"2024-02-26T04:55:54.151933+0000","last_clean":"2024-02-26T04:55:54.151933+0000","last_became_active":"2024-02-26T04:55:27.196898+0000","last_became_peered":"2024-02-26T04:55:27.196898+0000","last_unstale":"2024-02-26T04:55:54.151933+0000","last_undegraded":"2024-02-26T04:55:54.151933+0000","last_fullsized":"2024-02-26T04:55:54.151933+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:27:09.635379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213239+0000","last_change":"2024-02-26T04:55:28.200665+0000","last_active":"2024-02-26T04:55:33.213239+0000","last_peered":"2024-02-26T04:55:33.213239+0000","last_clean":"2024-02-26T04:55:33.213239+0000","last_became_active":"2024-02-26T04:55:28.200329+0000","last_became_peered":"2024-02-26T04:55:28.200329+0000","last_unstale":"2024-02-26T04:55:33.213239+0000","last_undegraded":"2024-02-26T04:55:33.213239+0000","last_fullsized":"2024-02-26T04:55:33.213239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:27.090259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213899+0000","last_change":"2024-02-26T04:55:32.225861+0000","last_active":"2024-02-26T04:55:33.213899+0000","last_peered":"2024-02-26T04:55:33.213899+0000","last_clean":"2024-02-26T04:55:33.213899+0000","last_became_active":"2024-02-26T04:55:32.225187+0000","last_became_peered":"2024-02-26T04:55:32.225187+0000","last_unstale":"2024-02-26T04:55:33.213899+0000","last_undegraded":"2024-02-26T04:55:33.213899+0000","last_fullsized":"2024-02-26T04:55:33.213899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:14:09.077383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:53.550158+0000","last_change":"2024-02-26T04:55:27.200321+0000","last_active":"2024-02-26T04:55:53.550158+0000","last_peered":"2024-02-26T04:55:53.550158+0000","last_clean":"2024-02-26T04:55:53.550158+0000","last_became_active":"2024-02-26T04:55:27.199978+0000","last_became_peered":"2024-02-26T04:55:27.199978+0000","last_unstale":"2024-02-26T04:55:53.550158+0000","last_undegraded":"2024-02-26T04:55:53.550158+0000","last_fullsized":"2024-02-26T04:55:53.550158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:08:20.381010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134602+0000","last_change":"2024-02-26T04:55:28.213940+0000","last_active":"2024-02-26T04:55:34.134602+0000","last_peered":"2024-02-26T04:55:34.134602+0000","last_clean":"2024-02-26T04:55:34.134602+0000","last_became_active":"2024-02-26T04:55:28.213590+0000","last_became_peered":"2024-02-26T04:55:28.213590+0000","last_unstale":"2024-02-26T04:55:34.134602+0000","last_undegraded":"2024-02-26T04:55:34.134602+0000","last_fullsized":"2024-02-26T04:55:34.134602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:29:29.151391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134626+0000","last_change":"2024-02-26T04:55:32.232669+0000","last_active":"2024-02-26T04:55:34.134626+0000","last_peered":"2024-02-26T04:55:34.134626+0000","last_clean":"2024-02-26T04:55:34.134626+0000","last_became_active":"2024-02-26T04:55:32.232288+0000","last_became_peered":"2024-02-26T04:55:32.232288+0000","last_unstale":"2024-02-26T04:55:34.134626+0000","last_undegraded":"2024-02-26T04:55:34.134626+0000","last_fullsized":"2024-02-26T04:55:34.134626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:31:42.436516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134627+0000","last_change":"2024-02-26T04:55:27.205993+0000","last_active":"2024-02-26T04:55:34.134627+0000","last_peered":"2024-02-26T04:55:34.134627+0000","last_clean":"2024-02-26T04:55:34.134627+0000","last_became_active":"2024-02-26T04:55:27.205594+0000","last_became_peered":"2024-02-26T04:55:27.205594+0000","last_unstale":"2024-02-26T04:55:34.134627+0000","last_undegraded":"2024-02-26T04:55:34.134627+0000","last_fullsized":"2024-02-26T04:55:34.134627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:44:19.993925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770427+0000","last_change":"2024-02-26T04:55:28.203429+0000","last_active":"2024-02-26T04:55:33.770427+0000","last_peered":"2024-02-26T04:55:33.770427+0000","last_clean":"2024-02-26T04:55:33.770427+0000","last_became_active":"2024-02-26T04:55:28.203002+0000","last_became_peered":"2024-02-26T04:55:28.203002+0000","last_unstale":"2024-02-26T04:55:33.770427+0000","last_undegraded":"2024-02-26T04:55:33.770427+0000","last_fullsized":"2024-02-26T04:55:33.770427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:08:05.872241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.135146+0000","last_change":"2024-02-26T04:55:32.239323+0000","last_active":"2024-02-26T04:55:34.135146+0000","last_peered":"2024-02-26T04:55:34.135146+0000","last_clean":"2024-02-26T04:55:34.135146+0000","last_became_active":"2024-02-26T04:55:32.238843+0000","last_became_peered":"2024-02-26T04:55:32.238843+0000","last_unstale":"2024-02-26T04:55:34.135146+0000","last_undegraded":"2024-02-26T04:55:34.135146+0000","last_fullsized":"2024-02-26T04:55:34.135146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:39:56.645429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134409+0000","last_change":"2024-02-26T04:55:27.203222+0000","last_active":"2024-02-26T04:55:34.134409+0000","last_peered":"2024-02-26T04:55:34.134409+0000","last_clean":"2024-02-26T04:55:34.134409+0000","last_became_active":"2024-02-26T04:55:27.202845+0000","last_became_peered":"2024-02-26T04:55:27.202845+0000","last_unstale":"2024-02-26T04:55:34.134409+0000","last_undegraded":"2024-02-26T04:55:34.134409+0000","last_fullsized":"2024-02-26T04:55:34.134409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:16:07.524260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.118135+0000","last_change":"2024-02-26T04:55:28.209122+0000","last_active":"2024-02-26T04:55:34.118135+0000","last_peered":"2024-02-26T04:55:34.118135+0000","last_clean":"2024-02-26T04:55:34.118135+0000","last_became_active":"2024-02-26T04:55:28.207598+0000","last_became_peered":"2024-02-26T04:55:28.207598+0000","last_unstale":"2024-02-26T04:55:34.118135+0000","last_undegraded":"2024-02-26T04:55:34.118135+0000","last_fullsized":"2024-02-26T04:55:34.118135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:02:10.763515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213215+0000","last_change":"2024-02-26T04:55:32.227333+0000","last_active":"2024-02-26T04:55:33.213215+0000","last_peered":"2024-02-26T04:55:33.213215+0000","last_clean":"2024-02-26T04:55:33.213215+0000","last_became_active":"2024-02-26T04:55:32.226956+0000","last_became_peered":"2024-02-26T04:55:32.226956+0000","last_unstale":"2024-02-26T04:55:33.213215+0000","last_undegraded":"2024-02-26T04:55:33.213215+0000","last_fullsized":"2024-02-26T04:55:33.213215+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:51:40.710242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213253+0000","last_change":"2024-02-26T04:55:27.198781+0000","last_active":"2024-02-26T04:55:33.213253+0000","last_peered":"2024-02-26T04:55:33.213253+0000","last_clean":"2024-02-26T04:55:33.213253+0000","last_became_active":"2024-02-26T04:55:27.198288+0000","last_became_peered":"2024-02-26T04:55:27.198288+0000","last_unstale":"2024-02-26T04:55:33.213253+0000","last_undegraded":"2024-02-26T04:55:33.213253+0000","last_fullsized":"2024-02-26T04:55:33.213253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:52:18.665816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.117975+0000","last_change":"2024-02-26T04:55:28.213044+0000","last_active":"2024-02-26T04:55:34.117975+0000","last_peered":"2024-02-26T04:55:34.117975+0000","last_clean":"2024-02-26T04:55:34.117975+0000","last_became_active":"2024-02-26T04:55:28.212712+0000","last_became_peered":"2024-02-26T04:55:28.212712+0000","last_unstale":"2024-02-26T04:55:34.117975+0000","last_undegraded":"2024-02-26T04:55:34.117975+0000","last_fullsized":"2024-02-26T04:55:34.117975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:02:09.155551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213698+0000","last_change":"2024-02-26T04:55:32.231867+0000","last_active":"2024-02-26T04:55:33.213698+0000","last_peered":"2024-02-26T04:55:33.213698+0000","last_clean":"2024-02-26T04:55:33.213698+0000","last_became_active":"2024-02-26T04:55:32.231450+0000","last_became_peered":"2024-02-26T04:55:32.231450+0000","last_unstale":"2024-02-26T04:55:33.213698+0000","last_undegraded":"2024-02-26T04:55:33.213698+0000","last_fullsized":"2024-02-26T04:55:33.213698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:12:14.367427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134231+0000","last_change":"2024-02-26T04:55:28.207746+0000","last_active":"2024-02-26T04:55:34.134231+0000","last_peered":"2024-02-26T04:55:34.134231+0000","last_clean":"2024-02-26T04:55:34.134231+0000","last_became_active":"2024-02-26T04:55:28.207438+0000","last_became_peered":"2024-02-26T04:55:28.207438+0000","last_unstale":"2024-02-26T04:55:34.134231+0000","last_undegraded":"2024-02-26T04:55:34.134231+0000","last_fullsized":"2024-02-26T04:55:34.134231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:09.409635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.212835+0000","last_change":"2024-02-26T04:55:27.192233+0000","last_active":"2024-02-26T04:55:33.212835+0000","last_peered":"2024-02-26T04:55:33.212835+0000","last_clean":"2024-02-26T04:55:33.212835+0000","last_became_active":"2024-02-26T04:55:27.191868+0000","last_became_peered":"2024-02-26T04:55:27.191868+0000","last_unstale":"2024-02-26T04:55:33.212835+0000","last_undegraded":"2024-02-26T04:55:33.212835+0000","last_fullsized":"2024-02-26T04:55:33.212835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:37:58.430504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213634+0000","last_change":"2024-02-26T04:55:32.226577+0000","last_active":"2024-02-26T04:55:33.213634+0000","last_peered":"2024-02-26T04:55:33.213634+0000","last_clean":"2024-02-26T04:55:33.213634+0000","last_became_active":"2024-02-26T04:55:32.226148+0000","last_became_peered":"2024-02-26T04:55:32.226148+0000","last_unstale":"2024-02-26T04:55:33.213634+0000","last_undegraded":"2024-02-26T04:55:33.213634+0000","last_fullsized":"2024-02-26T04:55:33.213634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:34:53.099211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134303+0000","last_change":"2024-02-26T04:55:28.217865+0000","last_active":"2024-02-26T04:55:34.134303+0000","last_peered":"2024-02-26T04:55:34.134303+0000","last_clean":"2024-02-26T04:55:34.134303+0000","last_became_active":"2024-02-26T04:55:28.217075+0000","last_became_peered":"2024-02-26T04:55:28.217075+0000","last_unstale":"2024-02-26T04:55:34.134303+0000","last_undegraded":"2024-02-26T04:55:34.134303+0000","last_fullsized":"2024-02-26T04:55:34.134303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:30:38.720694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:54.147218+0000","last_change":"2024-02-26T04:55:27.204300+0000","last_active":"2024-02-26T04:55:54.147218+0000","last_peered":"2024-02-26T04:55:54.147218+0000","last_clean":"2024-02-26T04:55:54.147218+0000","last_became_active":"2024-02-26T04:55:27.203933+0000","last_became_peered":"2024-02-26T04:55:27.203933+0000","last_unstale":"2024-02-26T04:55:54.147218+0000","last_undegraded":"2024-02-26T04:55:54.147218+0000","last_fullsized":"2024-02-26T04:55:54.147218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:19:36.247209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213080+0000","last_change":"2024-02-26T04:54:13.145088+0000","last_active":"2024-02-26T04:55:33.213080+0000","last_peered":"2024-02-26T04:55:33.213080+0000","last_clean":"2024-02-26T04:55:33.213080+0000","last_became_active":"2024-02-26T04:54:13.144657+0000","last_became_peered":"2024-02-26T04:54:13.144657+0000","last_unstale":"2024-02-26T04:55:33.213080+0000","last_undegraded":"2024-02-26T04:55:33.213080+0000","last_fullsized":"2024-02-26T04:55:33.213080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:50:45.246330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:50:45.246330+0000","last_clean_scrub_stamp":"2024-02-26T04:50:45.246330+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:00:50.842020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134999+0000","last_change":"2024-02-26T04:55:32.240124+0000","last_active":"2024-02-26T04:55:34.134999+0000","last_peered":"2024-02-26T04:55:34.134999+0000","last_clean":"2024-02-26T04:55:34.134999+0000","last_became_active":"2024-02-26T04:55:32.239624+0000","last_became_peered":"2024-02-26T04:55:32.239624+0000","last_unstale":"2024-02-26T04:55:34.134999+0000","last_undegraded":"2024-02-26T04:55:34.134999+0000","last_fullsized":"2024-02-26T04:55:34.134999+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:40:33.770917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.212704+0000","last_change":"2024-02-26T04:55:28.204631+0000","last_active":"2024-02-26T04:55:33.212704+0000","last_peered":"2024-02-26T04:55:33.212704+0000","last_clean":"2024-02-26T04:55:33.212704+0000","last_became_active":"2024-02-26T04:55:28.204274+0000","last_became_peered":"2024-02-26T04:55:28.204274+0000","last_unstale":"2024-02-26T04:55:33.212704+0000","last_undegraded":"2024-02-26T04:55:33.212704+0000","last_fullsized":"2024-02-26T04:55:33.212704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:18:18.336015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213144+0000","last_change":"2024-02-26T04:55:27.201719+0000","last_active":"2024-02-26T04:55:33.213144+0000","last_peered":"2024-02-26T04:55:33.213144+0000","last_clean":"2024-02-26T04:55:33.213144+0000","last_became_active":"2024-02-26T04:55:27.199060+0000","last_became_peered":"2024-02-26T04:55:27.199060+0000","last_unstale":"2024-02-26T04:55:33.213144+0000","last_undegraded":"2024-02-26T04:55:33.213144+0000","last_fullsized":"2024-02-26T04:55:33.213144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:37:37.620970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770175+0000","last_change":"2024-02-26T04:55:32.228900+0000","last_active":"2024-02-26T04:55:33.770175+0000","last_peered":"2024-02-26T04:55:33.770175+0000","last_clean":"2024-02-26T04:55:33.770175+0000","last_became_active":"2024-02-26T04:55:32.228262+0000","last_became_peered":"2024-02-26T04:55:32.228262+0000","last_unstale":"2024-02-26T04:55:33.770175+0000","last_undegraded":"2024-02-26T04:55:33.770175+0000","last_fullsized":"2024-02-26T04:55:33.770175+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:33.069756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134886+0000","last_change":"2024-02-26T04:55:27.205264+0000","last_active":"2024-02-26T04:55:34.134886+0000","last_peered":"2024-02-26T04:55:34.134886+0000","last_clean":"2024-02-26T04:55:34.134886+0000","last_became_active":"2024-02-26T04:55:27.204882+0000","last_became_peered":"2024-02-26T04:55:27.204882+0000","last_unstale":"2024-02-26T04:55:34.134886+0000","last_undegraded":"2024-02-26T04:55:34.134886+0000","last_fullsized":"2024-02-26T04:55:34.134886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:14:19.497725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213379+0000","last_change":"2024-02-26T04:55:28.200264+0000","last_active":"2024-02-26T04:55:33.213379+0000","last_peered":"2024-02-26T04:55:33.213379+0000","last_clean":"2024-02-26T04:55:33.213379+0000","last_became_active":"2024-02-26T04:55:28.199913+0000","last_became_peered":"2024-02-26T04:55:28.199913+0000","last_unstale":"2024-02-26T04:55:33.213379+0000","last_undegraded":"2024-02-26T04:55:33.213379+0000","last_fullsized":"2024-02-26T04:55:33.213379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:54:51.529149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134861+0000","last_change":"2024-02-26T04:55:32.233362+0000","last_active":"2024-02-26T04:55:34.134861+0000","last_peered":"2024-02-26T04:55:34.134861+0000","last_clean":"2024-02-26T04:55:34.134861+0000","last_became_active":"2024-02-26T04:55:32.232907+0000","last_became_peered":"2024-02-26T04:55:32.232907+0000","last_unstale":"2024-02-26T04:55:34.134861+0000","last_undegraded":"2024-02-26T04:55:34.134861+0000","last_fullsized":"2024-02-26T04:55:34.134861+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:49:11.889863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:53.549361+0000","last_change":"2024-02-26T04:55:27.207371+0000","last_active":"2024-02-26T04:55:53.549361+0000","last_peered":"2024-02-26T04:55:53.549361+0000","last_clean":"2024-02-26T04:55:53.549361+0000","last_became_active":"2024-02-26T04:55:27.207047+0000","last_became_peered":"2024-02-26T04:55:27.207047+0000","last_unstale":"2024-02-26T04:55:53.549361+0000","last_undegraded":"2024-02-26T04:55:53.549361+0000","last_fullsized":"2024-02-26T04:55:53.549361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:29.575633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134823+0000","last_change":"2024-02-26T04:55:28.207132+0000","last_active":"2024-02-26T04:55:34.134823+0000","last_peered":"2024-02-26T04:55:34.134823+0000","last_clean":"2024-02-26T04:55:34.134823+0000","last_became_active":"2024-02-26T04:55:28.206376+0000","last_became_peered":"2024-02-26T04:55:28.206376+0000","last_unstale":"2024-02-26T04:55:34.134823+0000","last_undegraded":"2024-02-26T04:55:34.134823+0000","last_fullsized":"2024-02-26T04:55:34.134823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:17:53.802386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.104876+0000","last_change":"2024-02-26T04:55:32.226381+0000","last_active":"2024-02-26T04:55:34.104876+0000","last_peered":"2024-02-26T04:55:34.104876+0000","last_clean":"2024-02-26T04:55:34.104876+0000","last_became_active":"2024-02-26T04:55:32.225940+0000","last_became_peered":"2024-02-26T04:55:32.225940+0000","last_unstale":"2024-02-26T04:55:34.104876+0000","last_undegraded":"2024-02-26T04:55:34.104876+0000","last_fullsized":"2024-02-26T04:55:34.104876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:51:20.190514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:52.541337+0000","last_change":"2024-02-26T04:55:27.200246+0000","last_active":"2024-02-26T04:55:52.541337+0000","last_peered":"2024-02-26T04:55:52.541337+0000","last_clean":"2024-02-26T04:55:52.541337+0000","last_became_active":"2024-02-26T04:55:27.199906+0000","last_became_peered":"2024-02-26T04:55:27.199906+0000","last_unstale":"2024-02-26T04:55:52.541337+0000","last_undegraded":"2024-02-26T04:55:52.541337+0000","last_fullsized":"2024-02-26T04:55:52.541337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:29.037230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.214405+0000","last_change":"2024-02-26T04:55:28.215357+0000","last_active":"2024-02-26T04:55:33.214405+0000","last_peered":"2024-02-26T04:55:33.214405+0000","last_clean":"2024-02-26T04:55:33.214405+0000","last_became_active":"2024-02-26T04:55:28.212379+0000","last_became_peered":"2024-02-26T04:55:28.212379+0000","last_unstale":"2024-02-26T04:55:33.214405+0000","last_undegraded":"2024-02-26T04:55:33.214405+0000","last_fullsized":"2024-02-26T04:55:33.214405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:49.560837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931647+0000","last_change":"2024-02-26T04:55:32.227765+0000","last_active":"2024-02-26T04:55:33.931647+0000","last_peered":"2024-02-26T04:55:33.931647+0000","last_clean":"2024-02-26T04:55:33.931647+0000","last_became_active":"2024-02-26T04:55:32.227346+0000","last_became_peered":"2024-02-26T04:55:32.227346+0000","last_unstale":"2024-02-26T04:55:33.931647+0000","last_undegraded":"2024-02-26T04:55:33.931647+0000","last_fullsized":"2024-02-26T04:55:33.931647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:05:12.634958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:52.542768+0000","last_change":"2024-02-26T04:55:27.199979+0000","last_active":"2024-02-26T04:55:52.542768+0000","last_peered":"2024-02-26T04:55:52.542768+0000","last_clean":"2024-02-26T04:55:52.542768+0000","last_became_active":"2024-02-26T04:55:27.199618+0000","last_became_peered":"2024-02-26T04:55:27.199618+0000","last_unstale":"2024-02-26T04:55:52.542768+0000","last_undegraded":"2024-02-26T04:55:52.542768+0000","last_fullsized":"2024-02-26T04:55:52.542768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:20:59.649473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134000+0000","last_change":"2024-02-26T04:55:28.202253+0000","last_active":"2024-02-26T04:55:34.134000+0000","last_peered":"2024-02-26T04:55:34.134000+0000","last_clean":"2024-02-26T04:55:34.134000+0000","last_became_active":"2024-02-26T04:55:28.199825+0000","last_became_peered":"2024-02-26T04:55:28.199825+0000","last_unstale":"2024-02-26T04:55:34.134000+0000","last_undegraded":"2024-02-26T04:55:34.134000+0000","last_fullsized":"2024-02-26T04:55:34.134000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:21:59.148995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134880+0000","last_change":"2024-02-26T04:55:32.239327+0000","last_active":"2024-02-26T04:55:34.134880+0000","last_peered":"2024-02-26T04:55:34.134880+0000","last_clean":"2024-02-26T04:55:34.134880+0000","last_became_active":"2024-02-26T04:55:32.238742+0000","last_became_peered":"2024-02-26T04:55:32.238742+0000","last_unstale":"2024-02-26T04:55:34.134880+0000","last_undegraded":"2024-02-26T04:55:34.134880+0000","last_fullsized":"2024-02-26T04:55:34.134880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:38:08.821549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.133939+0000","last_change":"2024-02-26T04:55:27.206529+0000","last_active":"2024-02-26T04:55:34.133939+0000","last_peered":"2024-02-26T04:55:34.133939+0000","last_clean":"2024-02-26T04:55:34.133939+0000","last_became_active":"2024-02-26T04:55:27.206203+0000","last_became_peered":"2024-02-26T04:55:27.206203+0000","last_unstale":"2024-02-26T04:55:34.133939+0000","last_undegraded":"2024-02-26T04:55:34.133939+0000","last_fullsized":"2024-02-26T04:55:34.133939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:24.491832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.769955+0000","last_change":"2024-02-26T04:55:28.205915+0000","last_active":"2024-02-26T04:55:33.769955+0000","last_peered":"2024-02-26T04:55:33.769955+0000","last_clean":"2024-02-26T04:55:33.769955+0000","last_became_active":"2024-02-26T04:55:28.205576+0000","last_became_peered":"2024-02-26T04:55:28.205576+0000","last_unstale":"2024-02-26T04:55:33.769955+0000","last_undegraded":"2024-02-26T04:55:33.769955+0000","last_fullsized":"2024-02-26T04:55:33.769955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:18:54.639778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931418+0000","last_change":"2024-02-26T04:55:32.226253+0000","last_active":"2024-02-26T04:55:33.931418+0000","last_peered":"2024-02-26T04:55:33.931418+0000","last_clean":"2024-02-26T04:55:33.931418+0000","last_became_active":"2024-02-26T04:55:32.225815+0000","last_became_peered":"2024-02-26T04:55:32.225815+0000","last_unstale":"2024-02-26T04:55:33.931418+0000","last_undegraded":"2024-02-26T04:55:33.931418+0000","last_fullsized":"2024-02-26T04:55:33.931418+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:22.764756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.104806+0000","last_change":"2024-02-26T04:55:27.209066+0000","last_active":"2024-02-26T04:55:34.104806+0000","last_peered":"2024-02-26T04:55:34.104806+0000","last_clean":"2024-02-26T04:55:34.104806+0000","last_became_active":"2024-02-26T04:55:27.208556+0000","last_became_peered":"2024-02-26T04:55:27.208556+0000","last_unstale":"2024-02-26T04:55:34.104806+0000","last_undegraded":"2024-02-26T04:55:34.104806+0000","last_fullsized":"2024-02-26T04:55:34.104806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:14:34.950262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134229+0000","last_change":"2024-02-26T04:55:28.213065+0000","last_active":"2024-02-26T04:55:34.134229+0000","last_peered":"2024-02-26T04:55:34.134229+0000","last_clean":"2024-02-26T04:55:34.134229+0000","last_became_active":"2024-02-26T04:55:28.211112+0000","last_became_peered":"2024-02-26T04:55:28.211112+0000","last_unstale":"2024-02-26T04:55:34.134229+0000","last_undegraded":"2024-02-26T04:55:34.134229+0000","last_fullsized":"2024-02-26T04:55:34.134229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:41:10.070183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931604+0000","last_change":"2024-02-26T04:55:32.229838+0000","last_active":"2024-02-26T04:55:33.931604+0000","last_peered":"2024-02-26T04:55:33.931604+0000","last_clean":"2024-02-26T04:55:33.931604+0000","last_became_active":"2024-02-26T04:55:32.228696+0000","last_became_peered":"2024-02-26T04:55:32.228696+0000","last_unstale":"2024-02-26T04:55:33.931604+0000","last_undegraded":"2024-02-26T04:55:33.931604+0000","last_fullsized":"2024-02-26T04:55:33.931604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:59:42.359301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:52.543601+0000","last_change":"2024-02-26T04:55:27.205688+0000","last_active":"2024-02-26T04:55:52.543601+0000","last_peered":"2024-02-26T04:55:52.543601+0000","last_clean":"2024-02-26T04:55:52.543601+0000","last_became_active":"2024-02-26T04:55:27.205306+0000","last_became_peered":"2024-02-26T04:55:27.205306+0000","last_unstale":"2024-02-26T04:55:52.543601+0000","last_undegraded":"2024-02-26T04:55:52.543601+0000","last_fullsized":"2024-02-26T04:55:52.543601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:23:50.203565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213413+0000","last_change":"2024-02-26T04:55:28.210073+0000","last_active":"2024-02-26T04:55:33.213413+0000","last_peered":"2024-02-26T04:55:33.213413+0000","last_clean":"2024-02-26T04:55:33.213413+0000","last_became_active":"2024-02-26T04:55:28.208475+0000","last_became_peered":"2024-02-26T04:55:28.208475+0000","last_unstale":"2024-02-26T04:55:33.213413+0000","last_undegraded":"2024-02-26T04:55:33.213413+0000","last_fullsized":"2024-02-26T04:55:33.213413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:06:49.513916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134148+0000","last_change":"2024-02-26T04:55:32.228708+0000","last_active":"2024-02-26T04:55:34.134148+0000","last_peered":"2024-02-26T04:55:34.134148+0000","last_clean":"2024-02-26T04:55:34.134148+0000","last_became_active":"2024-02-26T04:55:32.228304+0000","last_became_peered":"2024-02-26T04:55:32.228304+0000","last_unstale":"2024-02-26T04:55:34.134148+0000","last_undegraded":"2024-02-26T04:55:34.134148+0000","last_fullsized":"2024-02-26T04:55:34.134148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:53:02.390712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:53.551484+0000","last_change":"2024-02-26T04:55:27.199761+0000","last_active":"2024-02-26T04:55:53.551484+0000","last_peered":"2024-02-26T04:55:53.551484+0000","last_clean":"2024-02-26T04:55:53.551484+0000","last_became_active":"2024-02-26T04:55:27.199113+0000","last_became_peered":"2024-02-26T04:55:27.199113+0000","last_unstale":"2024-02-26T04:55:53.551484+0000","last_undegraded":"2024-02-26T04:55:53.551484+0000","last_fullsized":"2024-02-26T04:55:53.551484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:14:10.631886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213832+0000","last_change":"2024-02-26T04:55:28.211484+0000","last_active":"2024-02-26T04:55:33.213832+0000","last_peered":"2024-02-26T04:55:33.213832+0000","last_clean":"2024-02-26T04:55:33.213832+0000","last_became_active":"2024-02-26T04:55:28.211131+0000","last_became_peered":"2024-02-26T04:55:28.211131+0000","last_unstale":"2024-02-26T04:55:33.213832+0000","last_undegraded":"2024-02-26T04:55:33.213832+0000","last_fullsized":"2024-02-26T04:55:33.213832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:54:09.393847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213586+0000","last_change":"2024-02-26T04:55:32.227493+0000","last_active":"2024-02-26T04:55:33.213586+0000","last_peered":"2024-02-26T04:55:33.213586+0000","last_clean":"2024-02-26T04:55:33.213586+0000","last_became_active":"2024-02-26T04:55:32.227106+0000","last_became_peered":"2024-02-26T04:55:32.227106+0000","last_unstale":"2024-02-26T04:55:33.213586+0000","last_undegraded":"2024-02-26T04:55:33.213586+0000","last_fullsized":"2024-02-26T04:55:33.213586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:33.040966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134963+0000","last_change":"2024-02-26T04:55:27.205331+0000","last_active":"2024-02-26T04:55:34.134963+0000","last_peered":"2024-02-26T04:55:34.134963+0000","last_clean":"2024-02-26T04:55:34.134963+0000","last_became_active":"2024-02-26T04:55:27.204998+0000","last_became_peered":"2024-02-26T04:55:27.204998+0000","last_unstale":"2024-02-26T04:55:34.134963+0000","last_undegraded":"2024-02-26T04:55:34.134963+0000","last_fullsized":"2024-02-26T04:55:34.134963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:10:03.671016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213586+0000","last_change":"2024-02-26T04:55:28.222116+0000","last_active":"2024-02-26T04:55:33.213586+0000","last_peered":"2024-02-26T04:55:33.213586+0000","last_clean":"2024-02-26T04:55:33.213586+0000","last_became_active":"2024-02-26T04:55:28.221758+0000","last_became_peered":"2024-02-26T04:55:28.221758+0000","last_unstale":"2024-02-26T04:55:33.213586+0000","last_undegraded":"2024-02-26T04:55:33.213586+0000","last_fullsized":"2024-02-26T04:55:33.213586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:35:57.637410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134081+0000","last_change":"2024-02-26T04:55:32.232222+0000","last_active":"2024-02-26T04:55:34.134081+0000","last_peered":"2024-02-26T04:55:34.134081+0000","last_clean":"2024-02-26T04:55:34.134081+0000","last_became_active":"2024-02-26T04:55:32.231836+0000","last_became_peered":"2024-02-26T04:55:32.231836+0000","last_unstale":"2024-02-26T04:55:34.134081+0000","last_undegraded":"2024-02-26T04:55:34.134081+0000","last_fullsized":"2024-02-26T04:55:34.134081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:18:00.766266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213725+0000","last_change":"2024-02-26T04:55:27.200385+0000","last_active":"2024-02-26T04:55:33.213725+0000","last_peered":"2024-02-26T04:55:33.213725+0000","last_clean":"2024-02-26T04:55:33.213725+0000","last_became_active":"2024-02-26T04:55:27.200046+0000","last_became_peered":"2024-02-26T04:55:27.200046+0000","last_unstale":"2024-02-26T04:55:33.213725+0000","last_undegraded":"2024-02-26T04:55:33.213725+0000","last_fullsized":"2024-02-26T04:55:33.213725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:01:58.112457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213819+0000","last_change":"2024-02-26T04:55:28.220702+0000","last_active":"2024-02-26T04:55:33.213819+0000","last_peered":"2024-02-26T04:55:33.213819+0000","last_clean":"2024-02-26T04:55:33.213819+0000","last_became_active":"2024-02-26T04:55:28.220386+0000","last_became_peered":"2024-02-26T04:55:28.220386+0000","last_unstale":"2024-02-26T04:55:33.213819+0000","last_undegraded":"2024-02-26T04:55:33.213819+0000","last_fullsized":"2024-02-26T04:55:33.213819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:40:37.774831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770054+0000","last_change":"2024-02-26T04:55:28.206409+0000","last_active":"2024-02-26T04:55:33.770054+0000","last_peered":"2024-02-26T04:55:33.770054+0000","last_clean":"2024-02-26T04:55:33.770054+0000","last_became_active":"2024-02-26T04:55:28.206046+0000","last_became_peered":"2024-02-26T04:55:28.206046+0000","last_unstale":"2024-02-26T04:55:33.770054+0000","last_undegraded":"2024-02-26T04:55:33.770054+0000","last_fullsized":"2024-02-26T04:55:33.770054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:35:43.277376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.105094+0000","last_change":"2024-02-26T04:55:28.216146+0000","last_active":"2024-02-26T04:55:34.105094+0000","last_peered":"2024-02-26T04:55:34.105094+0000","last_clean":"2024-02-26T04:55:34.105094+0000","last_became_active":"2024-02-26T04:55:28.215800+0000","last_became_peered":"2024-02-26T04:55:28.215800+0000","last_unstale":"2024-02-26T04:55:34.105094+0000","last_undegraded":"2024-02-26T04:55:34.105094+0000","last_fullsized":"2024-02-26T04:55:34.105094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:18:47.902694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.104662+0000","last_change":"2024-02-26T04:55:28.221587+0000","last_active":"2024-02-26T04:55:34.104662+0000","last_peered":"2024-02-26T04:55:34.104662+0000","last_clean":"2024-02-26T04:55:34.104662+0000","last_became_active":"2024-02-26T04:55:28.220990+0000","last_became_peered":"2024-02-26T04:55:28.220990+0000","last_unstale":"2024-02-26T04:55:34.104662+0000","last_undegraded":"2024-02-26T04:55:34.104662+0000","last_fullsized":"2024-02-26T04:55:34.104662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:24:13.302081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.214169+0000","last_change":"2024-02-26T04:55:28.220586+0000","last_active":"2024-02-26T04:55:33.214169+0000","last_peered":"2024-02-26T04:55:33.214169+0000","last_clean":"2024-02-26T04:55:33.214169+0000","last_became_active":"2024-02-26T04:55:28.219585+0000","last_became_peered":"2024-02-26T04:55:28.219585+0000","last_unstale":"2024-02-26T04:55:33.214169+0000","last_undegraded":"2024-02-26T04:55:33.214169+0000","last_fullsized":"2024-02-26T04:55:33.214169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:41.786142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213364+0000","last_change":"2024-02-26T04:55:28.204599+0000","last_active":"2024-02-26T04:55:33.213364+0000","last_peered":"2024-02-26T04:55:33.213364+0000","last_clean":"2024-02-26T04:55:33.213364+0000","last_became_active":"2024-02-26T04:55:28.204252+0000","last_became_peered":"2024-02-26T04:55:28.204252+0000","last_unstale":"2024-02-26T04:55:33.213364+0000","last_undegraded":"2024-02-26T04:55:33.213364+0000","last_fullsized":"2024-02-26T04:55:33.213364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:30:19.807897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770536+0000","last_change":"2024-02-26T04:55:28.214233+0000","last_active":"2024-02-26T04:55:33.770536+0000","last_peered":"2024-02-26T04:55:33.770536+0000","last_clean":"2024-02-26T04:55:33.770536+0000","last_became_active":"2024-02-26T04:55:28.213927+0000","last_became_peered":"2024-02-26T04:55:28.213927+0000","last_unstale":"2024-02-26T04:55:33.770536+0000","last_undegraded":"2024-02-26T04:55:33.770536+0000","last_fullsized":"2024-02-26T04:55:33.770536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:31:07.967134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.105257+0000","last_change":"2024-02-26T04:55:28.217610+0000","last_active":"2024-02-26T04:55:34.105257+0000","last_peered":"2024-02-26T04:55:34.105257+0000","last_clean":"2024-02-26T04:55:34.105257+0000","last_became_active":"2024-02-26T04:55:28.217263+0000","last_became_peered":"2024-02-26T04:55:28.217263+0000","last_unstale":"2024-02-26T04:55:34.105257+0000","last_undegraded":"2024-02-26T04:55:34.105257+0000","last_fullsized":"2024-02-26T04:55:34.105257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:28:18.831887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134962+0000","last_change":"2024-02-26T04:55:28.207297+0000","last_active":"2024-02-26T04:55:34.134962+0000","last_peered":"2024-02-26T04:55:34.134962+0000","last_clean":"2024-02-26T04:55:34.134962+0000","last_became_active":"2024-02-26T04:55:28.206811+0000","last_became_peered":"2024-02-26T04:55:28.206811+0000","last_unstale":"2024-02-26T04:55:34.134962+0000","last_undegraded":"2024-02-26T04:55:34.134962+0000","last_fullsized":"2024-02-26T04:55:34.134962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:12:01.254968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134098+0000","last_change":"2024-02-26T04:55:28.217247+0000","last_active":"2024-02-26T04:55:34.134098+0000","last_peered":"2024-02-26T04:55:34.134098+0000","last_clean":"2024-02-26T04:55:34.134098+0000","last_became_active":"2024-02-26T04:55:28.214691+0000","last_became_peered":"2024-02-26T04:55:28.214691+0000","last_unstale":"2024-02-26T04:55:34.134098+0000","last_undegraded":"2024-02-26T04:55:34.134098+0000","last_fullsized":"2024-02-26T04:55:34.134098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:27:45.564520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134033+0000","last_change":"2024-02-26T04:55:28.210491+0000","last_active":"2024-02-26T04:55:34.134033+0000","last_peered":"2024-02-26T04:55:34.134033+0000","last_clean":"2024-02-26T04:55:34.134033+0000","last_became_active":"2024-02-26T04:55:28.210144+0000","last_became_peered":"2024-02-26T04:55:28.210144+0000","last_unstale":"2024-02-26T04:55:34.134033+0000","last_undegraded":"2024-02-26T04:55:34.134033+0000","last_fullsized":"2024-02-26T04:55:34.134033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:11.214716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134235+0000","last_change":"2024-02-26T04:55:28.216037+0000","last_active":"2024-02-26T04:55:34.134235+0000","last_peered":"2024-02-26T04:55:34.134235+0000","last_clean":"2024-02-26T04:55:34.134235+0000","last_became_active":"2024-02-26T04:55:28.215712+0000","last_became_peered":"2024-02-26T04:55:28.215712+0000","last_unstale":"2024-02-26T04:55:34.134235+0000","last_undegraded":"2024-02-26T04:55:34.134235+0000","last_fullsized":"2024-02-26T04:55:34.134235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:38:55.437642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213346+0000","last_change":"2024-02-26T04:55:28.205469+0000","last_active":"2024-02-26T04:55:33.213346+0000","last_peered":"2024-02-26T04:55:33.213346+0000","last_clean":"2024-02-26T04:55:33.213346+0000","last_became_active":"2024-02-26T04:55:28.205137+0000","last_became_peered":"2024-02-26T04:55:28.205137+0000","last_unstale":"2024-02-26T04:55:33.213346+0000","last_undegraded":"2024-02-26T04:55:33.213346+0000","last_fullsized":"2024-02-26T04:55:33.213346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:50.425010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770144+0000","last_change":"2024-02-26T04:55:28.213066+0000","last_active":"2024-02-26T04:55:33.770144+0000","last_peered":"2024-02-26T04:55:33.770144+0000","last_clean":"2024-02-26T04:55:33.770144+0000","last_became_active":"2024-02-26T04:55:28.212706+0000","last_became_peered":"2024-02-26T04:55:28.212706+0000","last_unstale":"2024-02-26T04:55:33.770144+0000","last_undegraded":"2024-02-26T04:55:33.770144+0000","last_fullsized":"2024-02-26T04:55:33.770144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:21:24.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281347,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":13872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":22,"num_read_kb":20,"num_write":116,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":68735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":477215,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57599999999999996}]},{"osd":1,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.623},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.72699999999999998}]},{"osd":2,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":4,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59799999999999998}]},{"osd":5,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.502},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.45600000000000002}]},{"osd":6,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.60899999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.63400000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":9,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.71299999999999997}]},{"osd":10,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.78800000000000003}]}]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1073003,"data_compressed":50407,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34499999999999997}]},{"osd":2,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.623}]},{"osd":3,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.47699999999999998}]},{"osd":4,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.439},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.495}]},{"osd":5,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65000000000000002}]},{"osd":6,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.53200000000000003}]},{"osd":7,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.72699999999999998}]},{"osd":8,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.69199999999999995}]},{"osd":9,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.66000000000000003}]},{"osd":11,"last update":"Mon Feb 26 04:55:37 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.67100000000000004}]}]},{"osd":9,"up_from":67,"seq":287762808861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":478033,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.46700000000000003,"15min":0.48199999999999998},"min":{"1min":0.29899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59199999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.48799999999999999,"15min":0.502},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.28100000000000003}]},{"osd":1,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.63400000000000001,"15min":0.65000000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57699999999999996}]},{"osd":2,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67100000000000004,"15min":0.67700000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.77700000000000002,"15min":0.80900000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99399999999999999,"5min":3.173,"15min":3.173},"last":0.40899999999999997}]},{"osd":3,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.51400000000000001,"15min":0.52800000000000002},"min":{"1min":0.28399999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.65100000000000002,"5min":0.753,"15min":0.753},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.52400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.30599999999999999}]},{"osd":4,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.66800000000000004,"15min":0.69199999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82199999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.496},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.65400000000000003,"15min":0.67000000000000004},"min":{"1min":0.36499999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.51700000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.69799999999999995,"15min":0.70999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.377}]},{"osd":6,"last update":"Mon Feb 26 04:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.73799999999999999,"15min":0.76300000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.63},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76400000000000001,"15min":0.78200000000000003},"min":{"1min":0.443,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46100000000000002}]},{"osd":8,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.90400000000000003,"15min":0.93799999999999994},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.278,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.434},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.79200000000000004,"15min":0.81100000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.56200000000000006}]},{"osd":10,"last update":"Mon Feb 26 04:55:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.61199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":8,"up_from":59,"seq":253403070498,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":477841,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.36199999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.626},{"interface":"front","average":{"1min":0.502,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.745,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60699999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.79100000000000004}]},{"osd":2,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42699999999999999,"15min":0.42599999999999999},"min":{"1min":0.32600000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.433,"15min":0.42999999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.51700000000000002}]},{"osd":3,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.65700000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55900000000000005,"15min":0.56399999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71899999999999997,"5min":0.755,"15min":0.755},"last":0.82499999999999996}]},{"osd":4,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.42599999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78600000000000003,"5min":0.871,"15min":0.871},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75900000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.754}]},{"osd":5,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76700000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67100000000000004,"15min":0.67400000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.93899999999999995}]},{"osd":7,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67000000000000004,"15min":0.66400000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.98499999999999999}]},{"osd":9,"last update":"Mon Feb 26 04:55:46 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.68799999999999994,"15min":0.67400000000000004},"min":{"1min":0.49199999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.874}]},{"osd":10,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.026}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]}]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1015808,"data_stored":1068165,"data_compressed":50407,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67000000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.40799999999999997}]},{"osd":1,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.41699999999999998,"15min":0.41599999999999998},"min":{"1min":0.28499999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.439,"15min":0.437},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.71999999999999997}]},{"osd":2,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.41199999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72399999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.75900000000000001}]},{"osd":3,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70899999999999996,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.73399999999999999,"5min":0.877,"15min":0.877},"last":0.77200000000000002}]},{"osd":4,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53100000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80800000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69899999999999995}]},{"osd":5,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.624},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.45200000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83099999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.84699999999999998}]},{"osd":6,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68799999999999994,"15min":0.68700000000000006},"min":{"1min":0.57099999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67600000000000005,"15min":0.67700000000000005},"min":{"1min":0.47299999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94099999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.875}]},{"osd":8,"last update":"Mon Feb 26 04:55:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72699999999999998,"15min":0.73299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.86399999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.89000000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.92800000000000005}]},{"osd":9,"last update":"Mon Feb 26 04:55:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.54100000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.89700000000000002}]},{"osd":10,"last update":"Mon Feb 26 04:55:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.98199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":962560,"data_stored":477215,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40200000000000002,"15min":0.38300000000000001},"min":{"1min":0.32200000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.442,"15min":0.42399999999999999},"min":{"1min":0.34000000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.72199999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45300000000000001}]},{"osd":1,"last update":"Mon Feb 26 04:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59199999999999997,"15min":0.56999999999999995},"min":{"1min":0.48299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.51200000000000001}]},{"osd":2,"last update":"Mon Feb 26 04:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.79000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.94799999999999995,"15min":1.0860000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.72899999999999998,"5min":14.884,"15min":14.884},"last":0.496}]},{"osd":3,"last update":"Mon Feb 26 04:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.45400000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82699999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.55200000000000005}]},{"osd":4,"last update":"Mon Feb 26 04:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65400000000000003,"15min":0.63400000000000001},"min":{"1min":0.499,"5min":0.38,"15min":0.38},"max":{"1min":0.84199999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":1.0049999999999999,"15min":1.143},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96899999999999997,"5min":14.898,"15min":14.898},"last":0.626}]},{"osd":5,"last update":"Mon Feb 26 04:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65300000000000002,"15min":0.63700000000000001},"min":{"1min":0.45500000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89600000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":1.008,"15min":1.008},"last":0.70799999999999996}]},{"osd":7,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.749,"15min":0.73699999999999999},"min":{"1min":0.60999999999999999,"5min":0.498,"15min":0.498},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.70899999999999996,"15min":0.68999999999999995},"min":{"1min":0.57299999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.59099999999999997}]},{"osd":8,"last update":"Mon Feb 26 04:55:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76500000000000001,"15min":0.77000000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69799999999999995}]},{"osd":9,"last update":"Mon Feb 26 04:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.78700000000000003,"15min":0.80500000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.78700000000000003,"15min":0.80300000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.66500000000000004}]},{"osd":10,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.72299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":500356,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51500000000000001,"15min":0.48899999999999999},"min":{"1min":0.36799999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.49399999999999999},"min":{"1min":0.40200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77300000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46800000000000003}]},{"osd":2,"last update":"Mon Feb 26 04:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.53500000000000003},"min":{"1min":0.436,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.56899999999999995,"15min":0.53300000000000003},"min":{"1min":0.45600000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85399999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66400000000000003}]},{"osd":3,"last update":"Mon Feb 26 04:55:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.58399999999999996,"15min":0.56100000000000005},"min":{"1min":0.435,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.439}]},{"osd":4,"last update":"Mon Feb 26 04:55:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.435,"15min":0.41499999999999998},"min":{"1min":0.28399999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.443,"15min":0.41999999999999998},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70799999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.53700000000000003}]},{"osd":5,"last update":"Mon Feb 26 04:55:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56599999999999995,"15min":0.54000000000000004},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.76500000000000001}]},{"osd":6,"last update":"Mon Feb 26 04:55:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.40000000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.5},{"interface":"front","average":{"1min":0.752,"5min":0.67900000000000005,"15min":0.65300000000000002},"min":{"1min":0.47899999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.64200000000000002}]},{"osd":7,"last update":"Mon Feb 26 04:54:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.71599999999999997}]},{"osd":8,"last update":"Mon Feb 26 04:55:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.71799999999999997,"15min":0.70899999999999996},"min":{"1min":0.57999999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.748},{"interface":"front","average":{"1min":0.751,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.55300000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.84199999999999997}]},{"osd":9,"last update":"Mon Feb 26 04:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1080000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69699999999999995}]},{"osd":10,"last update":"Mon Feb 26 04:55:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.80100000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":966656,"data_stored":477305,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 04:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52100000000000002,"15min":0.503},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68500000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81799999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53500000000000003}]},{"osd":2,"last update":"Mon Feb 26 04:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55400000000000005,"15min":0.52600000000000002},"min":{"1min":0.371,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58299999999999996,"15min":0.54800000000000004},"min":{"1min":0.41899999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.70899999999999996}]},{"osd":3,"last update":"Mon Feb 26 04:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.42399999999999999,"15min":0.40699999999999997},"min":{"1min":0.32700000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.42699999999999999,"15min":0.40300000000000002},"min":{"1min":0.29999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.32400000000000001}]},{"osd":4,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.45200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.82499999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73199999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54500000000000004}]},{"osd":5,"last update":"Mon Feb 26 04:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57699999999999996,"15min":0.55100000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84999999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61199999999999999,"15min":0.58199999999999996},"min":{"1min":0.41999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0269999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Mon Feb 26 04:55:37 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52500000000000002,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56000000000000005,"15min":0.54000000000000004},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.35399999999999998}]},{"osd":7,"last update":"Mon Feb 26 04:54:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.91200000000000003,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Feb 26 04:55:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76700000000000002,"15min":0.77300000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77700000000000002}]},{"osd":9,"last update":"Mon Feb 26 04:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.71799999999999997,"15min":0.73199999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79300000000000004,"15min":0.80500000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.5}]},{"osd":10,"last update":"Mon Feb 26 04:55:12 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.64500000000000002}]},{"osd":11,"last update":"Mon Feb 26 04:55:37 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.73899999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":482407,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.497,"15min":0.45900000000000002},"min":{"1min":0.33600000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.504,"15min":0.46600000000000003},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.70099999999999996}]},{"osd":1,"last update":"Mon Feb 26 04:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52800000000000002,"15min":0.495},"min":{"1min":0.39100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.54300000000000004,"15min":0.51700000000000002},"min":{"1min":0.37,"5min":0.318,"15min":0.318},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.60499999999999998}]},{"osd":3,"last update":"Mon Feb 26 04:55:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.443,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76200000000000001,"5min":0.87,"15min":0.87},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57999999999999996,"15min":0.56399999999999995},"min":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67000000000000004}]},{"osd":4,"last update":"Mon Feb 26 04:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.48599999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.57399999999999995,"15min":0.56000000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:55:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.46899999999999997,"15min":0.47699999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.82999999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.29299999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.70299999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42299999999999999}]},{"osd":6,"last update":"Mon Feb 26 04:55:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79100000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82199999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.748}]},{"osd":7,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.45200000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88400000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.65500000000000003,"15min":0.65400000000000003},"min":{"1min":0.51400000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.80900000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52400000000000002}]},{"osd":8,"last update":"Mon Feb 26 04:55:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.35199999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93100000000000005,"5min":1.115,"15min":1.115},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.374,"5min":0.314,"15min":0.314},"max":{"1min":0.81499999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56399999999999995}]},{"osd":9,"last update":"Mon Feb 26 04:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67000000000000004,"15min":0.66400000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.624},{"interface":"front","average":{"1min":0.75,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.64200000000000002}]},{"osd":10,"last update":"Mon Feb 26 04:55:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.76100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":477319,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:20 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.27700000000000002,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.33800000000000002,"15min":0.32900000000000001},"min":{"1min":0.25700000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.45800000000000002,"5min":0.502,"15min":0.502},"last":0.28199999999999997}]},{"osd":1,"last update":"Mon Feb 26 04:55:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.51500000000000001},"min":{"1min":0.42199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78200000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53700000000000003,"15min":0.51900000000000002},"min":{"1min":0.34499999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45500000000000002}]},{"osd":2,"last update":"Mon Feb 26 04:55:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.433,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.57999999999999996,"15min":0.55800000000000005},"min":{"1min":0.38,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80900000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.53900000000000003}]},{"osd":4,"last update":"Mon Feb 26 04:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.46500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999}]},{"osd":5,"last update":"Mon Feb 26 04:55:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.36299999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.497},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.47999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.436}]},{"osd":6,"last update":"Mon Feb 26 04:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.47499999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.74399999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.34100000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41799999999999998}]},{"osd":7,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Feb 26 04:55:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.874,"5min":0.995,"15min":0.995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.878,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63100000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:54:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.51800000000000002}]},{"osd":10,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.68899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":477859,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:54:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.51100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79500000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42899999999999999}]},{"osd":1,"last update":"Mon Feb 26 04:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.40500000000000003,"15min":0.36799999999999999},"min":{"1min":0.29499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.40400000000000003,"15min":0.38},"min":{"1min":0.32500000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.435}]},{"osd":2,"last update":"Mon Feb 26 04:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54100000000000004,"15min":0.51900000000000002},"min":{"1min":0.38900000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55500000000000005,"15min":0.53500000000000003},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":0.74299999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.71299999999999997}]},{"osd":3,"last update":"Mon Feb 26 04:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.59599999999999997,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.4619999999999997,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.57699999999999996,"15min":0.53700000000000003},"min":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":0.41099999999999998}]},{"osd":5,"last update":"Mon Feb 26 04:55:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80000000000000004,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.5},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.501,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81399999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.745}]},{"osd":6,"last update":"Mon Feb 26 04:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.61099999999999999,"15min":0.57899999999999996},"min":{"1min":0.46300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":4.4130000000000003,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.68400000000000005,"15min":0.65100000000000002},"min":{"1min":0.48999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.1639999999999997,"5min":4.1639999999999997,"15min":4.1639999999999997},"last":0.60199999999999998}]},{"osd":7,"last update":"Mon Feb 26 04:55:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79800000000000004,"5min":0.87,"15min":0.87},"last":0.621},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.58599999999999997,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.2300000000000004,"5min":4.2300000000000004,"15min":4.2300000000000004},"last":0.57399999999999995}]},{"osd":8,"last update":"Mon Feb 26 04:55:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68300000000000005,"15min":0.67600000000000005},"min":{"1min":0.54600000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.87,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.51400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83999999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.85699999999999998}]},{"osd":9,"last update":"Mon Feb 26 04:55:46 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":0.82399999999999995,"15min":0.747},"min":{"1min":0.55600000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":10.051,"5min":10.051,"15min":10.051},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1220000000000001,"5min":0.81000000000000005,"15min":0.75800000000000001},"min":{"1min":0.59999999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":6.3360000000000003,"5min":6.3360000000000003,"15min":6.3360000000000003},"last":0.83499999999999996}]},{"osd":10,"last update":"Mon Feb 26 04:55:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]}]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1069059,"data_compressed":50407,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48699999999999999,"15min":0.48199999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.35099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69399999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.44500000000000001}]},{"osd":1,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54300000000000004,"15min":0.52500000000000002},"min":{"1min":0.38800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84599999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59699999999999998}]},{"osd":2,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.35399999999999998},"min":{"1min":0.28299999999999997,"5min":0.22,"15min":0.22},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.375,"5min":0.38400000000000001,"15min":0.375},"min":{"1min":0.26100000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57599999999999996,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.33800000000000002}]},{"osd":3,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.33700000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50700000000000001,"15min":0.49199999999999999},"min":{"1min":0.36599999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46500000000000002}]},{"osd":4,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58899999999999997,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.376,"5min":0.372,"15min":0.372},"max":{"1min":0.752,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.56699999999999995}]},{"osd":6,"last update":"Mon Feb 26 04:55:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59399999999999997,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79500000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.61399999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:54:55 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81699999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.41299999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70799999999999996}]},{"osd":8,"last update":"Mon Feb 26 04:55:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.57299999999999995,"15min":0.58299999999999996},"min":{"1min":0.32800000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.56799999999999995,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51400000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68200000000000005,"15min":0.67700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.73099999999999998}]},{"osd":10,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.628},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":40471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":11792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":12020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T04:55:56.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T04:55:57.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.063+0000 7f49c97f7640 1 -- 172.21.15.60:0/582935712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49c41011a0 msgr2=0x7f49c41015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.063+0000 7f49c97f7640 1 --2- 172.21.15.60:0/582935712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49c41011a0 0x7f49c41015c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009900 tx=0x7f49bc02bc30 comp rx=0 tx=0).stop 2024-02-26T04:55:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.063+0000 7f49c97f7640 1 -- 172.21.15.60:0/582935712 shutdown_connections 2024-02-26T04:55:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.063+0000 7f49c97f7640 1 --2- 172.21.15.60:0/582935712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c4102d50 0x7f49c41095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.063+0000 7f49c97f7640 1 --2- 172.21.15.60:0/582935712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49c4102390 0x7f49c4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.063+0000 7f49c97f7640 1 --2- 172.21.15.60:0/582935712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49c41011a0 0x7f49c41015c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.063+0000 7f49c97f7640 1 -- 172.21.15.60:0/582935712 >> 172.21.15.60:0/582935712 conn(0x7f49c40fc730 msgr2=0x7f49c40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.063+0000 7f49c97f7640 1 -- 172.21.15.60:0/582935712 shutdown_connections 2024-02-26T04:55:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.063+0000 7f49c97f7640 1 -- 172.21.15.60:0/582935712 wait complete. 2024-02-26T04:55:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.063+0000 7f49c97f7640 1 Processor -- start 2024-02-26T04:55:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c97f7640 1 -- start start 2024-02-26T04:55:57.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c97f7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c41011a0 0x7f49c41972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c97f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49c4102390 0x7f49c41977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c97f7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49c4102d50 0x7f49c419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c97f7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c419ed90 con 0x7f49c4102390 2024-02-26T04:55:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c97f7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c419ef00 con 0x7f49c41011a0 2024-02-26T04:55:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c97f7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c419f0c0 con 0x7f49c4102d50 2024-02-26T04:55:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49c4102390 0x7f49c41977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49c4102390 0x7f49c41977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51154/0 (socket says 172.21.15.60:51154) 2024-02-26T04:55:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c27fc640 1 -- 172.21.15.60:0/2042512861 learned_addr learned my addr 172.21.15.60:0/2042512861 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c2ffd640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c41011a0 0x7f49c41972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c37fe640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49c4102d50 0x7f49c419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c27fc640 1 -- 172.21.15.60:0/2042512861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49c4102d50 msgr2=0x7f49c419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c27fc640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49c4102d50 0x7f49c419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c27fc640 1 -- 172.21.15.60:0/2042512861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c41011a0 msgr2=0x7f49c41972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:57.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c27fc640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c41011a0 0x7f49c41972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:57.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c27fc640 1 -- 172.21.15.60:0/2042512861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc009560 con 0x7f49c4102390 2024-02-26T04:55:57.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c37fe640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49c4102d50 0x7f49c419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:55:57.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.064+0000 7f49c27fc640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49c4102390 0x7f49c41977e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00d770 tx=0x7f49ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:57.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.065+0000 7f49bbfff640 1 -- 172.21.15.60:0/2042512861 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac009c30 con 0x7f49c4102390 2024-02-26T04:55:57.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.065+0000 7f49c97f7640 1 -- 172.21.15.60:0/2042512861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c419f3a0 con 0x7f49c4102390 2024-02-26T04:55:57.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.065+0000 7f49bbfff640 1 -- 172.21.15.60:0/2042512861 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49ac00de80 con 0x7f49c4102390 2024-02-26T04:55:57.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.065+0000 7f49bbfff640 1 -- 172.21.15.60:0/2042512861 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac004e80 con 0x7f49c4102390 2024-02-26T04:55:57.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.065+0000 7f49c97f7640 1 -- 172.21.15.60:0/2042512861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c419f860 con 0x7f49c4102390 2024-02-26T04:55:57.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.066+0000 7f49c97f7640 1 -- 172.21.15.60:0/2042512861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c4108510 con 0x7f49c4102390 2024-02-26T04:55:57.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.067+0000 7f49bbfff640 1 -- 172.21.15.60:0/2042512861 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f49ac005020 con 0x7f49c4102390 2024-02-26T04:55:57.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.067+0000 7f49bbfff640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49b00a2150 0x7f49b00a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:57.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.067+0000 7f49c2ffd640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49b00a2150 0x7f49b00a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:57.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.067+0000 7f49bbfff640 1 -- 172.21.15.60:0/2042512861 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f49ac0cdac0 con 0x7f49c4102390 2024-02-26T04:55:57.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.068+0000 7f49c2ffd640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49b00a2150 0x7f49b00a4610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f49bc005ab0 tx=0x7f49bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:57.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.072+0000 7f49bbfff640 1 -- 172.21.15.60:0/2042512861 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f49ac0966f0 con 0x7f49c4102390 2024-02-26T04:55:57.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:57 smithi060 ceph-mon[32382]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:57 smithi060 ceph-mon[32382]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:57 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2707948398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:57.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:57 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1407571056' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:57.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.253+0000 7f49c97f7640 1 -- 172.21.15.60:0/2042512861 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f49c402ce40 con 0x7f49b00a2150 2024-02-26T04:55:57.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.265+0000 7f49bbfff640 1 -- 172.21.15.60:0/2042512861 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423487 (secure 0 0 0) 0x7f49c402ce40 con 0x7f49b00a2150 2024-02-26T04:55:57.272 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:55:57.275 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-02-26T04:55:57.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.269+0000 7f49c97f7640 1 -- 172.21.15.60:0/2042512861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49b00a2150 msgr2=0x7f49b00a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:57.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.269+0000 7f49c97f7640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49b00a2150 0x7f49b00a4610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f49bc005ab0 tx=0x7f49bc005fd0 comp rx=0 tx=0).stop 2024-02-26T04:55:57.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.269+0000 7f49c97f7640 1 -- 172.21.15.60:0/2042512861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49c4102390 msgr2=0x7f49c41977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:57.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.269+0000 7f49c97f7640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49c4102390 0x7f49c41977e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00d770 tx=0x7f49ac00dc30 comp rx=0 tx=0).stop 2024-02-26T04:55:57.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.269+0000 7f49c97f7640 1 -- 172.21.15.60:0/2042512861 shutdown_connections 2024-02-26T04:55:57.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.269+0000 7f49c97f7640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49b00a2150 0x7f49b00a4610 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:57.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.269+0000 7f49c97f7640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49c4102d50 0x7f49c419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:57.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.269+0000 7f49c97f7640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49c4102390 0x7f49c41977e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:57.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.269+0000 7f49c97f7640 1 --2- 172.21.15.60:0/2042512861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c41011a0 0x7f49c41972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:57.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.269+0000 7f49c97f7640 1 -- 172.21.15.60:0/2042512861 >> 172.21.15.60:0/2042512861 conn(0x7f49c40fc730 msgr2=0x7f49c41056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:57.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.269+0000 7f49c97f7640 1 -- 172.21.15.60:0/2042512861 shutdown_connections 2024-02-26T04:55:57.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:57.269+0000 7f49c97f7640 1 -- 172.21.15.60:0/2042512861 wait complete. 2024-02-26T04:55:57.283 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-02-26T04:55:56.312173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604240,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":148,"num_read_kb":129,"num_write":297,"num_write_kb":2988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381612,"kb_used_data":11624,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560212,"statfs":{"total":1151940427776,"available":1151549657088,"internally_reserved":0,"allocated":11902976,"data_stored":7535777,"data_compressed":282837,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11013,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":22,"num_read_kb":20,"num_write":70,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003285"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770611+0000","last_change":"2024-02-26T04:55:28.208832+0000","last_active":"2024-02-26T04:55:33.770611+0000","last_peered":"2024-02-26T04:55:33.770611+0000","last_clean":"2024-02-26T04:55:33.770611+0000","last_became_active":"2024-02-26T04:55:28.208454+0000","last_became_peered":"2024-02-26T04:55:28.208454+0000","last_unstale":"2024-02-26T04:55:33.770611+0000","last_undegraded":"2024-02-26T04:55:33.770611+0000","last_fullsized":"2024-02-26T04:55:33.770611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:55:54.132384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.214555+0000","last_change":"2024-02-26T04:55:28.203876+0000","last_active":"2024-02-26T04:55:33.214555+0000","last_peered":"2024-02-26T04:55:33.214555+0000","last_clean":"2024-02-26T04:55:33.214555+0000","last_became_active":"2024-02-26T04:55:28.203399+0000","last_became_peered":"2024-02-26T04:55:28.203399+0000","last_unstale":"2024-02-26T04:55:33.214555+0000","last_undegraded":"2024-02-26T04:55:33.214555+0000","last_fullsized":"2024-02-26T04:55:33.214555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:31:40.927929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.214285+0000","last_change":"2024-02-26T04:55:28.211388+0000","last_active":"2024-02-26T04:55:33.214285+0000","last_peered":"2024-02-26T04:55:33.214285+0000","last_clean":"2024-02-26T04:55:33.214285+0000","last_became_active":"2024-02-26T04:55:28.209007+0000","last_became_peered":"2024-02-26T04:55:28.209007+0000","last_unstale":"2024-02-26T04:55:33.214285+0000","last_undegraded":"2024-02-26T04:55:33.214285+0000","last_fullsized":"2024-02-26T04:55:33.214285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:37:01.197589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931553+0000","last_change":"2024-02-26T04:55:28.216378+0000","last_active":"2024-02-26T04:55:33.931553+0000","last_peered":"2024-02-26T04:55:33.931553+0000","last_clean":"2024-02-26T04:55:33.931553+0000","last_became_active":"2024-02-26T04:55:28.216074+0000","last_became_peered":"2024-02-26T04:55:28.216074+0000","last_unstale":"2024-02-26T04:55:33.931553+0000","last_undegraded":"2024-02-26T04:55:33.931553+0000","last_fullsized":"2024-02-26T04:55:33.931553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:15:20.460699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134370+0000","last_change":"2024-02-26T04:55:28.213026+0000","last_active":"2024-02-26T04:55:34.134370+0000","last_peered":"2024-02-26T04:55:34.134370+0000","last_clean":"2024-02-26T04:55:34.134370+0000","last_became_active":"2024-02-26T04:55:28.212706+0000","last_became_peered":"2024-02-26T04:55:28.212706+0000","last_unstale":"2024-02-26T04:55:34.134370+0000","last_undegraded":"2024-02-26T04:55:34.134370+0000","last_fullsized":"2024-02-26T04:55:34.134370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:57:35.961956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.117804+0000","last_change":"2024-02-26T04:55:28.202123+0000","last_active":"2024-02-26T04:55:34.117804+0000","last_peered":"2024-02-26T04:55:34.117804+0000","last_clean":"2024-02-26T04:55:34.117804+0000","last_became_active":"2024-02-26T04:55:28.201733+0000","last_became_peered":"2024-02-26T04:55:28.201733+0000","last_unstale":"2024-02-26T04:55:34.117804+0000","last_undegraded":"2024-02-26T04:55:34.117804+0000","last_fullsized":"2024-02-26T04:55:34.117804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:43:11.300006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213167+0000","last_change":"2024-02-26T04:55:28.199591+0000","last_active":"2024-02-26T04:55:33.213167+0000","last_peered":"2024-02-26T04:55:33.213167+0000","last_clean":"2024-02-26T04:55:33.213167+0000","last_became_active":"2024-02-26T04:55:28.199139+0000","last_became_peered":"2024-02-26T04:55:28.199139+0000","last_unstale":"2024-02-26T04:55:33.213167+0000","last_undegraded":"2024-02-26T04:55:33.213167+0000","last_fullsized":"2024-02-26T04:55:33.213167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:43:22.177447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.133805+0000","last_change":"2024-02-26T04:55:28.214631+0000","last_active":"2024-02-26T04:55:34.133805+0000","last_peered":"2024-02-26T04:55:34.133805+0000","last_clean":"2024-02-26T04:55:34.133805+0000","last_became_active":"2024-02-26T04:55:28.214290+0000","last_became_peered":"2024-02-26T04:55:28.214290+0000","last_unstale":"2024-02-26T04:55:34.133805+0000","last_undegraded":"2024-02-26T04:55:34.133805+0000","last_fullsized":"2024-02-26T04:55:34.133805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:30:11.494833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134548+0000","last_change":"2024-02-26T04:55:28.212808+0000","last_active":"2024-02-26T04:55:34.134548+0000","last_peered":"2024-02-26T04:55:34.134548+0000","last_clean":"2024-02-26T04:55:34.134548+0000","last_became_active":"2024-02-26T04:55:28.212331+0000","last_became_peered":"2024-02-26T04:55:28.212331+0000","last_unstale":"2024-02-26T04:55:34.134548+0000","last_undegraded":"2024-02-26T04:55:34.134548+0000","last_fullsized":"2024-02-26T04:55:34.134548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:27:02.014173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.118306+0000","last_change":"2024-02-26T04:55:28.204176+0000","last_active":"2024-02-26T04:55:34.118306+0000","last_peered":"2024-02-26T04:55:34.118306+0000","last_clean":"2024-02-26T04:55:34.118306+0000","last_became_active":"2024-02-26T04:55:28.203821+0000","last_became_peered":"2024-02-26T04:55:28.203821+0000","last_unstale":"2024-02-26T04:55:34.118306+0000","last_undegraded":"2024-02-26T04:55:34.118306+0000","last_fullsized":"2024-02-26T04:55:34.118306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:24:03.132650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213828+0000","last_change":"2024-02-26T04:55:28.215958+0000","last_active":"2024-02-26T04:55:33.213828+0000","last_peered":"2024-02-26T04:55:33.213828+0000","last_clean":"2024-02-26T04:55:33.213828+0000","last_became_active":"2024-02-26T04:55:28.215594+0000","last_became_peered":"2024-02-26T04:55:28.215594+0000","last_unstale":"2024-02-26T04:55:33.213828+0000","last_undegraded":"2024-02-26T04:55:33.213828+0000","last_fullsized":"2024-02-26T04:55:33.213828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:45:47.915379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931492+0000","last_change":"2024-02-26T04:55:28.218392+0000","last_active":"2024-02-26T04:55:33.931492+0000","last_peered":"2024-02-26T04:55:33.931492+0000","last_clean":"2024-02-26T04:55:33.931492+0000","last_became_active":"2024-02-26T04:55:28.217933+0000","last_became_peered":"2024-02-26T04:55:28.217933+0000","last_unstale":"2024-02-26T04:55:33.931492+0000","last_undegraded":"2024-02-26T04:55:33.931492+0000","last_fullsized":"2024-02-26T04:55:33.931492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:30:38.880649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213476+0000","last_change":"2024-02-26T04:55:28.197099+0000","last_active":"2024-02-26T04:55:33.213476+0000","last_peered":"2024-02-26T04:55:33.213476+0000","last_clean":"2024-02-26T04:55:33.213476+0000","last_became_active":"2024-02-26T04:55:28.196755+0000","last_became_peered":"2024-02-26T04:55:28.196755+0000","last_unstale":"2024-02-26T04:55:33.213476+0000","last_undegraded":"2024-02-26T04:55:33.213476+0000","last_fullsized":"2024-02-26T04:55:33.213476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:37:31.685845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213634+0000","last_change":"2024-02-26T04:55:28.212064+0000","last_active":"2024-02-26T04:55:33.213634+0000","last_peered":"2024-02-26T04:55:33.213634+0000","last_clean":"2024-02-26T04:55:33.213634+0000","last_became_active":"2024-02-26T04:55:28.208742+0000","last_became_peered":"2024-02-26T04:55:28.208742+0000","last_unstale":"2024-02-26T04:55:33.213634+0000","last_undegraded":"2024-02-26T04:55:33.213634+0000","last_fullsized":"2024-02-26T04:55:33.213634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:38:19.742468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213988+0000","last_change":"2024-02-26T04:55:28.212408+0000","last_active":"2024-02-26T04:55:33.213988+0000","last_peered":"2024-02-26T04:55:33.213988+0000","last_clean":"2024-02-26T04:55:33.213988+0000","last_became_active":"2024-02-26T04:55:28.211391+0000","last_became_peered":"2024-02-26T04:55:28.211391+0000","last_unstale":"2024-02-26T04:55:33.213988+0000","last_undegraded":"2024-02-26T04:55:33.213988+0000","last_fullsized":"2024-02-26T04:55:33.213988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:27:37.957529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.133908+0000","last_change":"2024-02-26T04:55:28.222810+0000","last_active":"2024-02-26T04:55:34.133908+0000","last_peered":"2024-02-26T04:55:34.133908+0000","last_clean":"2024-02-26T04:55:34.133908+0000","last_became_active":"2024-02-26T04:55:28.222341+0000","last_became_peered":"2024-02-26T04:55:28.222341+0000","last_unstale":"2024-02-26T04:55:34.133908+0000","last_undegraded":"2024-02-26T04:55:34.133908+0000","last_fullsized":"2024-02-26T04:55:34.133908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:59:21.627517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770476+0000","last_change":"2024-02-26T04:55:28.208377+0000","last_active":"2024-02-26T04:55:33.770476+0000","last_peered":"2024-02-26T04:55:33.770476+0000","last_clean":"2024-02-26T04:55:33.770476+0000","last_became_active":"2024-02-26T04:55:28.207932+0000","last_became_peered":"2024-02-26T04:55:28.207932+0000","last_unstale":"2024-02-26T04:55:33.770476+0000","last_undegraded":"2024-02-26T04:55:33.770476+0000","last_fullsized":"2024-02-26T04:55:33.770476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:15:39.586191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134464+0000","last_change":"2024-02-26T04:55:28.201105+0000","last_active":"2024-02-26T04:55:34.134464+0000","last_peered":"2024-02-26T04:55:34.134464+0000","last_clean":"2024-02-26T04:55:34.134464+0000","last_became_active":"2024-02-26T04:55:28.200796+0000","last_became_peered":"2024-02-26T04:55:28.200796+0000","last_unstale":"2024-02-26T04:55:34.134464+0000","last_undegraded":"2024-02-26T04:55:34.134464+0000","last_fullsized":"2024-02-26T04:55:34.134464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:53:43.713055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.105300+0000","last_change":"2024-02-26T04:55:28.215725+0000","last_active":"2024-02-26T04:55:34.105300+0000","last_peered":"2024-02-26T04:55:34.105300+0000","last_clean":"2024-02-26T04:55:34.105300+0000","last_became_active":"2024-02-26T04:55:28.215341+0000","last_became_peered":"2024-02-26T04:55:28.215341+0000","last_unstale":"2024-02-26T04:55:34.105300+0000","last_undegraded":"2024-02-26T04:55:34.105300+0000","last_fullsized":"2024-02-26T04:55:34.105300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:26:26.960474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.104999+0000","last_change":"2024-02-26T04:55:32.233303+0000","last_active":"2024-02-26T04:55:34.104999+0000","last_peered":"2024-02-26T04:55:34.104999+0000","last_clean":"2024-02-26T04:55:34.104999+0000","last_became_active":"2024-02-26T04:55:32.232901+0000","last_became_peered":"2024-02-26T04:55:32.232901+0000","last_unstale":"2024-02-26T04:55:34.104999+0000","last_undegraded":"2024-02-26T04:55:34.104999+0000","last_fullsized":"2024-02-26T04:55:34.104999+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:37:49.659668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:53.550443+0000","last_change":"2024-02-26T04:55:27.199682+0000","last_active":"2024-02-26T04:55:53.550443+0000","last_peered":"2024-02-26T04:55:53.550443+0000","last_clean":"2024-02-26T04:55:53.550443+0000","last_became_active":"2024-02-26T04:55:27.199293+0000","last_became_peered":"2024-02-26T04:55:27.199293+0000","last_unstale":"2024-02-26T04:55:53.550443+0000","last_undegraded":"2024-02-26T04:55:53.550443+0000","last_fullsized":"2024-02-26T04:55:53.550443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:23:21.685615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134361+0000","last_change":"2024-02-26T04:55:28.205365+0000","last_active":"2024-02-26T04:55:34.134361+0000","last_peered":"2024-02-26T04:55:34.134361+0000","last_clean":"2024-02-26T04:55:34.134361+0000","last_became_active":"2024-02-26T04:55:28.204980+0000","last_became_peered":"2024-02-26T04:55:28.204980+0000","last_unstale":"2024-02-26T04:55:34.134361+0000","last_undegraded":"2024-02-26T04:55:34.134361+0000","last_fullsized":"2024-02-26T04:55:34.134361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:57:24.148691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931178+0000","last_change":"2024-02-26T04:55:32.233142+0000","last_active":"2024-02-26T04:55:33.931178+0000","last_peered":"2024-02-26T04:55:33.931178+0000","last_clean":"2024-02-26T04:55:33.931178+0000","last_became_active":"2024-02-26T04:55:32.232722+0000","last_became_peered":"2024-02-26T04:55:32.232722+0000","last_unstale":"2024-02-26T04:55:33.931178+0000","last_undegraded":"2024-02-26T04:55:33.931178+0000","last_fullsized":"2024-02-26T04:55:33.931178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:30:20.906104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134682+0000","last_change":"2024-02-26T04:55:27.203420+0000","last_active":"2024-02-26T04:55:34.134682+0000","last_peered":"2024-02-26T04:55:34.134682+0000","last_clean":"2024-02-26T04:55:34.134682+0000","last_became_active":"2024-02-26T04:55:27.202968+0000","last_became_peered":"2024-02-26T04:55:27.202968+0000","last_unstale":"2024-02-26T04:55:34.134682+0000","last_undegraded":"2024-02-26T04:55:34.134682+0000","last_fullsized":"2024-02-26T04:55:34.134682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:44.496528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.214483+0000","last_change":"2024-02-26T04:55:28.217422+0000","last_active":"2024-02-26T04:55:33.214483+0000","last_peered":"2024-02-26T04:55:33.214483+0000","last_clean":"2024-02-26T04:55:33.214483+0000","last_became_active":"2024-02-26T04:55:28.217103+0000","last_became_peered":"2024-02-26T04:55:28.217103+0000","last_unstale":"2024-02-26T04:55:33.214483+0000","last_undegraded":"2024-02-26T04:55:33.214483+0000","last_fullsized":"2024-02-26T04:55:33.214483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:08:37.075467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931314+0000","last_change":"2024-02-26T04:55:32.233322+0000","last_active":"2024-02-26T04:55:33.931314+0000","last_peered":"2024-02-26T04:55:33.931314+0000","last_clean":"2024-02-26T04:55:33.931314+0000","last_became_active":"2024-02-26T04:55:32.232679+0000","last_became_peered":"2024-02-26T04:55:32.232679+0000","last_unstale":"2024-02-26T04:55:33.931314+0000","last_undegraded":"2024-02-26T04:55:33.931314+0000","last_fullsized":"2024-02-26T04:55:33.931314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:10.775924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:53.547570+0000","last_change":"2024-02-26T04:55:27.205960+0000","last_active":"2024-02-26T04:55:53.547570+0000","last_peered":"2024-02-26T04:55:53.547570+0000","last_clean":"2024-02-26T04:55:53.547570+0000","last_became_active":"2024-02-26T04:55:27.205604+0000","last_became_peered":"2024-02-26T04:55:27.205604+0000","last_unstale":"2024-02-26T04:55:53.547570+0000","last_undegraded":"2024-02-26T04:55:53.547570+0000","last_fullsized":"2024-02-26T04:55:53.547570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:45:25.002672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134236+0000","last_change":"2024-02-26T04:55:28.210669+0000","last_active":"2024-02-26T04:55:34.134236+0000","last_peered":"2024-02-26T04:55:34.134236+0000","last_clean":"2024-02-26T04:55:34.134236+0000","last_became_active":"2024-02-26T04:55:28.210322+0000","last_became_peered":"2024-02-26T04:55:28.210322+0000","last_unstale":"2024-02-26T04:55:34.134236+0000","last_undegraded":"2024-02-26T04:55:34.134236+0000","last_fullsized":"2024-02-26T04:55:34.134236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:57:29.121762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931225+0000","last_change":"2024-02-26T04:55:32.229077+0000","last_active":"2024-02-26T04:55:33.931225+0000","last_peered":"2024-02-26T04:55:33.931225+0000","last_clean":"2024-02-26T04:55:33.931225+0000","last_became_active":"2024-02-26T04:55:32.228670+0000","last_became_peered":"2024-02-26T04:55:32.228670+0000","last_unstale":"2024-02-26T04:55:33.931225+0000","last_undegraded":"2024-02-26T04:55:33.931225+0000","last_fullsized":"2024-02-26T04:55:33.931225+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:35:20.843351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134508+0000","last_change":"2024-02-26T04:55:27.206088+0000","last_active":"2024-02-26T04:55:34.134508+0000","last_peered":"2024-02-26T04:55:34.134508+0000","last_clean":"2024-02-26T04:55:34.134508+0000","last_became_active":"2024-02-26T04:55:27.205696+0000","last_became_peered":"2024-02-26T04:55:27.205696+0000","last_unstale":"2024-02-26T04:55:34.134508+0000","last_undegraded":"2024-02-26T04:55:34.134508+0000","last_fullsized":"2024-02-26T04:55:34.134508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:14:03.552251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134860+0000","last_change":"2024-02-26T04:55:28.215124+0000","last_active":"2024-02-26T04:55:34.134860+0000","last_peered":"2024-02-26T04:55:34.134860+0000","last_clean":"2024-02-26T04:55:34.134860+0000","last_became_active":"2024-02-26T04:55:28.214618+0000","last_became_peered":"2024-02-26T04:55:28.214618+0000","last_unstale":"2024-02-26T04:55:34.134860+0000","last_undegraded":"2024-02-26T04:55:34.134860+0000","last_fullsized":"2024-02-26T04:55:34.134860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:52:25.581632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.105233+0000","last_change":"2024-02-26T04:55:32.228629+0000","last_active":"2024-02-26T04:55:34.105233+0000","last_peered":"2024-02-26T04:55:34.105233+0000","last_clean":"2024-02-26T04:55:34.105233+0000","last_became_active":"2024-02-26T04:55:32.227608+0000","last_became_peered":"2024-02-26T04:55:32.227608+0000","last_unstale":"2024-02-26T04:55:34.105233+0000","last_undegraded":"2024-02-26T04:55:34.105233+0000","last_fullsized":"2024-02-26T04:55:34.105233+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:10:42.429352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:54.151407+0000","last_change":"2024-02-26T04:55:27.201261+0000","last_active":"2024-02-26T04:55:54.151407+0000","last_peered":"2024-02-26T04:55:54.151407+0000","last_clean":"2024-02-26T04:55:54.151407+0000","last_became_active":"2024-02-26T04:55:27.200892+0000","last_became_peered":"2024-02-26T04:55:27.200892+0000","last_unstale":"2024-02-26T04:55:54.151407+0000","last_undegraded":"2024-02-26T04:55:54.151407+0000","last_fullsized":"2024-02-26T04:55:54.151407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:34:32.774143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134974+0000","last_change":"2024-02-26T04:55:28.200445+0000","last_active":"2024-02-26T04:55:34.134974+0000","last_peered":"2024-02-26T04:55:34.134974+0000","last_clean":"2024-02-26T04:55:34.134974+0000","last_became_active":"2024-02-26T04:55:28.199959+0000","last_became_peered":"2024-02-26T04:55:28.199959+0000","last_unstale":"2024-02-26T04:55:34.134974+0000","last_undegraded":"2024-02-26T04:55:34.134974+0000","last_fullsized":"2024-02-26T04:55:34.134974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:36:27.397644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.105171+0000","last_change":"2024-02-26T04:55:32.232619+0000","last_active":"2024-02-26T04:55:34.105171+0000","last_peered":"2024-02-26T04:55:34.105171+0000","last_clean":"2024-02-26T04:55:34.105171+0000","last_became_active":"2024-02-26T04:55:32.232112+0000","last_became_peered":"2024-02-26T04:55:32.232112+0000","last_unstale":"2024-02-26T04:55:34.105171+0000","last_undegraded":"2024-02-26T04:55:34.105171+0000","last_fullsized":"2024-02-26T04:55:34.105171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:19.741393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:53.548966+0000","last_change":"2024-02-26T04:55:27.197376+0000","last_active":"2024-02-26T04:55:53.548966+0000","last_peered":"2024-02-26T04:55:53.548966+0000","last_clean":"2024-02-26T04:55:53.548966+0000","last_became_active":"2024-02-26T04:55:27.195258+0000","last_became_peered":"2024-02-26T04:55:27.195258+0000","last_unstale":"2024-02-26T04:55:53.548966+0000","last_undegraded":"2024-02-26T04:55:53.548966+0000","last_fullsized":"2024-02-26T04:55:53.548966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:41:51.284525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134542+0000","last_change":"2024-02-26T04:55:28.214337+0000","last_active":"2024-02-26T04:55:34.134542+0000","last_peered":"2024-02-26T04:55:34.134542+0000","last_clean":"2024-02-26T04:55:34.134542+0000","last_became_active":"2024-02-26T04:55:28.213966+0000","last_became_peered":"2024-02-26T04:55:28.213966+0000","last_unstale":"2024-02-26T04:55:34.134542+0000","last_undegraded":"2024-02-26T04:55:34.134542+0000","last_fullsized":"2024-02-26T04:55:34.134542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:42.974084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770538+0000","last_change":"2024-02-26T04:55:32.235579+0000","last_active":"2024-02-26T04:55:33.770538+0000","last_peered":"2024-02-26T04:55:33.770538+0000","last_clean":"2024-02-26T04:55:33.770538+0000","last_became_active":"2024-02-26T04:55:32.235241+0000","last_became_peered":"2024-02-26T04:55:32.235241+0000","last_unstale":"2024-02-26T04:55:33.770538+0000","last_undegraded":"2024-02-26T04:55:33.770538+0000","last_fullsized":"2024-02-26T04:55:33.770538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:06.319874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.212853+0000","last_change":"2024-02-26T04:55:27.197689+0000","last_active":"2024-02-26T04:55:33.212853+0000","last_peered":"2024-02-26T04:55:33.212853+0000","last_clean":"2024-02-26T04:55:33.212853+0000","last_became_active":"2024-02-26T04:55:27.197285+0000","last_became_peered":"2024-02-26T04:55:27.197285+0000","last_unstale":"2024-02-26T04:55:33.212853+0000","last_undegraded":"2024-02-26T04:55:33.212853+0000","last_fullsized":"2024-02-26T04:55:33.212853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:45:36.688902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.133692+0000","last_change":"2024-02-26T04:55:28.201902+0000","last_active":"2024-02-26T04:55:34.133692+0000","last_peered":"2024-02-26T04:55:34.133692+0000","last_clean":"2024-02-26T04:55:34.133692+0000","last_became_active":"2024-02-26T04:55:28.201604+0000","last_became_peered":"2024-02-26T04:55:28.201604+0000","last_unstale":"2024-02-26T04:55:34.133692+0000","last_undegraded":"2024-02-26T04:55:34.133692+0000","last_fullsized":"2024-02-26T04:55:34.133692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:48:57.711555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.214449+0000","last_change":"2024-02-26T04:55:32.230153+0000","last_active":"2024-02-26T04:55:33.214449+0000","last_peered":"2024-02-26T04:55:33.214449+0000","last_clean":"2024-02-26T04:55:33.214449+0000","last_became_active":"2024-02-26T04:55:32.229695+0000","last_became_peered":"2024-02-26T04:55:32.229695+0000","last_unstale":"2024-02-26T04:55:33.214449+0000","last_undegraded":"2024-02-26T04:55:33.214449+0000","last_fullsized":"2024-02-26T04:55:33.214449+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:15:39.575727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134355+0000","last_change":"2024-02-26T04:55:27.200943+0000","last_active":"2024-02-26T04:55:34.134355+0000","last_peered":"2024-02-26T04:55:34.134355+0000","last_clean":"2024-02-26T04:55:34.134355+0000","last_became_active":"2024-02-26T04:55:27.200565+0000","last_became_peered":"2024-02-26T04:55:27.200565+0000","last_unstale":"2024-02-26T04:55:34.134355+0000","last_undegraded":"2024-02-26T04:55:34.134355+0000","last_fullsized":"2024-02-26T04:55:34.134355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:54:19.670631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134026+0000","last_change":"2024-02-26T04:55:28.212232+0000","last_active":"2024-02-26T04:55:34.134026+0000","last_peered":"2024-02-26T04:55:34.134026+0000","last_clean":"2024-02-26T04:55:34.134026+0000","last_became_active":"2024-02-26T04:55:28.211875+0000","last_became_peered":"2024-02-26T04:55:28.211875+0000","last_unstale":"2024-02-26T04:55:34.134026+0000","last_undegraded":"2024-02-26T04:55:34.134026+0000","last_fullsized":"2024-02-26T04:55:34.134026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:55.379271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.133802+0000","last_change":"2024-02-26T04:55:32.234961+0000","last_active":"2024-02-26T04:55:34.133802+0000","last_peered":"2024-02-26T04:55:34.133802+0000","last_clean":"2024-02-26T04:55:34.133802+0000","last_became_active":"2024-02-26T04:55:32.234455+0000","last_became_peered":"2024-02-26T04:55:32.234455+0000","last_unstale":"2024-02-26T04:55:34.133802+0000","last_undegraded":"2024-02-26T04:55:34.133802+0000","last_fullsized":"2024-02-26T04:55:34.133802+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:00:44.322341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:52.544122+0000","last_change":"2024-02-26T04:55:27.199036+0000","last_active":"2024-02-26T04:55:52.544122+0000","last_peered":"2024-02-26T04:55:52.544122+0000","last_clean":"2024-02-26T04:55:52.544122+0000","last_became_active":"2024-02-26T04:55:27.198694+0000","last_became_peered":"2024-02-26T04:55:27.198694+0000","last_unstale":"2024-02-26T04:55:52.544122+0000","last_undegraded":"2024-02-26T04:55:52.544122+0000","last_fullsized":"2024-02-26T04:55:52.544122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:31:41.935123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134487+0000","last_change":"2024-02-26T04:55:28.209330+0000","last_active":"2024-02-26T04:55:34.134487+0000","last_peered":"2024-02-26T04:55:34.134487+0000","last_clean":"2024-02-26T04:55:34.134487+0000","last_became_active":"2024-02-26T04:55:28.208974+0000","last_became_peered":"2024-02-26T04:55:28.208974+0000","last_unstale":"2024-02-26T04:55:34.134487+0000","last_undegraded":"2024-02-26T04:55:34.134487+0000","last_fullsized":"2024-02-26T04:55:34.134487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:21:54.924526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.133990+0000","last_change":"2024-02-26T04:55:32.228033+0000","last_active":"2024-02-26T04:55:34.133990+0000","last_peered":"2024-02-26T04:55:34.133990+0000","last_clean":"2024-02-26T04:55:34.133990+0000","last_became_active":"2024-02-26T04:55:32.224273+0000","last_became_peered":"2024-02-26T04:55:32.224273+0000","last_unstale":"2024-02-26T04:55:34.133990+0000","last_undegraded":"2024-02-26T04:55:34.133990+0000","last_fullsized":"2024-02-26T04:55:34.133990+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:25:13.618421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213651+0000","last_change":"2024-02-26T04:55:27.202833+0000","last_active":"2024-02-26T04:55:33.213651+0000","last_peered":"2024-02-26T04:55:33.213651+0000","last_clean":"2024-02-26T04:55:33.213651+0000","last_became_active":"2024-02-26T04:55:27.202167+0000","last_became_peered":"2024-02-26T04:55:27.202167+0000","last_unstale":"2024-02-26T04:55:33.213651+0000","last_undegraded":"2024-02-26T04:55:33.213651+0000","last_fullsized":"2024-02-26T04:55:33.213651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:55:46.605763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.214410+0000","last_change":"2024-02-26T04:55:28.212055+0000","last_active":"2024-02-26T04:55:33.214410+0000","last_peered":"2024-02-26T04:55:33.214410+0000","last_clean":"2024-02-26T04:55:33.214410+0000","last_became_active":"2024-02-26T04:55:28.211233+0000","last_became_peered":"2024-02-26T04:55:28.211233+0000","last_unstale":"2024-02-26T04:55:33.214410+0000","last_undegraded":"2024-02-26T04:55:33.214410+0000","last_fullsized":"2024-02-26T04:55:33.214410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:31:35.300693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931616+0000","last_change":"2024-02-26T04:55:32.225736+0000","last_active":"2024-02-26T04:55:33.931616+0000","last_peered":"2024-02-26T04:55:33.931616+0000","last_clean":"2024-02-26T04:55:33.931616+0000","last_became_active":"2024-02-26T04:55:32.225186+0000","last_became_peered":"2024-02-26T04:55:32.225186+0000","last_unstale":"2024-02-26T04:55:33.931616+0000","last_undegraded":"2024-02-26T04:55:33.931616+0000","last_fullsized":"2024-02-26T04:55:33.931616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:13:48.999538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:52.542965+0000","last_change":"2024-02-26T04:55:27.200389+0000","last_active":"2024-02-26T04:55:52.542965+0000","last_peered":"2024-02-26T04:55:52.542965+0000","last_clean":"2024-02-26T04:55:52.542965+0000","last_became_active":"2024-02-26T04:55:27.200019+0000","last_became_peered":"2024-02-26T04:55:27.200019+0000","last_unstale":"2024-02-26T04:55:52.542965+0000","last_undegraded":"2024-02-26T04:55:52.542965+0000","last_fullsized":"2024-02-26T04:55:52.542965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:59:11.820988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134429+0000","last_change":"2024-02-26T04:55:28.220309+0000","last_active":"2024-02-26T04:55:34.134429+0000","last_peered":"2024-02-26T04:55:34.134429+0000","last_clean":"2024-02-26T04:55:34.134429+0000","last_became_active":"2024-02-26T04:55:28.219987+0000","last_became_peered":"2024-02-26T04:55:28.219987+0000","last_unstale":"2024-02-26T04:55:34.134429+0000","last_undegraded":"2024-02-26T04:55:34.134429+0000","last_fullsized":"2024-02-26T04:55:34.134429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:39:34.713802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134159+0000","last_change":"2024-02-26T04:55:32.236817+0000","last_active":"2024-02-26T04:55:34.134159+0000","last_peered":"2024-02-26T04:55:34.134159+0000","last_clean":"2024-02-26T04:55:34.134159+0000","last_became_active":"2024-02-26T04:55:32.235818+0000","last_became_peered":"2024-02-26T04:55:32.235818+0000","last_unstale":"2024-02-26T04:55:34.134159+0000","last_undegraded":"2024-02-26T04:55:34.134159+0000","last_fullsized":"2024-02-26T04:55:34.134159+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:56:54.121684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'15","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:54.493399+0000","last_change":"2024-02-26T04:55:27.201582+0000","last_active":"2024-02-26T04:55:54.493399+0000","last_peered":"2024-02-26T04:55:54.493399+0000","last_clean":"2024-02-26T04:55:54.493399+0000","last_became_active":"2024-02-26T04:55:27.201011+0000","last_became_peered":"2024-02-26T04:55:27.201011+0000","last_unstale":"2024-02-26T04:55:54.493399+0000","last_undegraded":"2024-02-26T04:55:54.493399+0000","last_fullsized":"2024-02-26T04:55:54.493399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:07:01.888924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10442,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134795+0000","last_change":"2024-02-26T04:55:28.199313+0000","last_active":"2024-02-26T04:55:34.134795+0000","last_peered":"2024-02-26T04:55:34.134795+0000","last_clean":"2024-02-26T04:55:34.134795+0000","last_became_active":"2024-02-26T04:55:28.198973+0000","last_became_peered":"2024-02-26T04:55:28.198973+0000","last_unstale":"2024-02-26T04:55:34.134795+0000","last_undegraded":"2024-02-26T04:55:34.134795+0000","last_fullsized":"2024-02-26T04:55:34.134795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:33:49.005899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.135331+0000","last_change":"2024-02-26T04:55:32.234529+0000","last_active":"2024-02-26T04:55:34.135331+0000","last_peered":"2024-02-26T04:55:34.135331+0000","last_clean":"2024-02-26T04:55:34.135331+0000","last_became_active":"2024-02-26T04:55:32.234105+0000","last_became_peered":"2024-02-26T04:55:32.234105+0000","last_unstale":"2024-02-26T04:55:34.135331+0000","last_undegraded":"2024-02-26T04:55:34.135331+0000","last_fullsized":"2024-02-26T04:55:34.135331+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:17:40.909267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:53.549314+0000","last_change":"2024-02-26T04:55:27.209178+0000","last_active":"2024-02-26T04:55:53.549314+0000","last_peered":"2024-02-26T04:55:53.549314+0000","last_clean":"2024-02-26T04:55:53.549314+0000","last_became_active":"2024-02-26T04:55:27.208834+0000","last_became_peered":"2024-02-26T04:55:27.208834+0000","last_unstale":"2024-02-26T04:55:53.549314+0000","last_undegraded":"2024-02-26T04:55:53.549314+0000","last_fullsized":"2024-02-26T04:55:53.549314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:28.187333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213044+0000","last_change":"2024-02-26T04:55:28.215208+0000","last_active":"2024-02-26T04:55:33.213044+0000","last_peered":"2024-02-26T04:55:33.213044+0000","last_clean":"2024-02-26T04:55:33.213044+0000","last_became_active":"2024-02-26T04:55:28.212135+0000","last_became_peered":"2024-02-26T04:55:28.212135+0000","last_unstale":"2024-02-26T04:55:33.213044+0000","last_undegraded":"2024-02-26T04:55:33.213044+0000","last_fullsized":"2024-02-26T04:55:33.213044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:51.269557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770214+0000","last_change":"2024-02-26T04:55:32.235439+0000","last_active":"2024-02-26T04:55:33.770214+0000","last_peered":"2024-02-26T04:55:33.770214+0000","last_clean":"2024-02-26T04:55:33.770214+0000","last_became_active":"2024-02-26T04:55:32.234987+0000","last_became_peered":"2024-02-26T04:55:32.234987+0000","last_unstale":"2024-02-26T04:55:33.770214+0000","last_undegraded":"2024-02-26T04:55:33.770214+0000","last_fullsized":"2024-02-26T04:55:33.770214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:31:00.637846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:52.543737+0000","last_change":"2024-02-26T04:55:27.205498+0000","last_active":"2024-02-26T04:55:52.543737+0000","last_peered":"2024-02-26T04:55:52.543737+0000","last_clean":"2024-02-26T04:55:52.543737+0000","last_became_active":"2024-02-26T04:55:27.205179+0000","last_became_peered":"2024-02-26T04:55:27.205179+0000","last_unstale":"2024-02-26T04:55:52.543737+0000","last_undegraded":"2024-02-26T04:55:52.543737+0000","last_fullsized":"2024-02-26T04:55:52.543737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:02:55.491896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213479+0000","last_change":"2024-02-26T04:55:28.201110+0000","last_active":"2024-02-26T04:55:33.213479+0000","last_peered":"2024-02-26T04:55:33.213479+0000","last_clean":"2024-02-26T04:55:33.213479+0000","last_became_active":"2024-02-26T04:55:28.200725+0000","last_became_peered":"2024-02-26T04:55:28.200725+0000","last_unstale":"2024-02-26T04:55:33.213479+0000","last_undegraded":"2024-02-26T04:55:33.213479+0000","last_fullsized":"2024-02-26T04:55:33.213479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:50.377925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134816+0000","last_change":"2024-02-26T04:55:32.240328+0000","last_active":"2024-02-26T04:55:34.134816+0000","last_peered":"2024-02-26T04:55:34.134816+0000","last_clean":"2024-02-26T04:55:34.134816+0000","last_became_active":"2024-02-26T04:55:32.238662+0000","last_became_peered":"2024-02-26T04:55:32.238662+0000","last_unstale":"2024-02-26T04:55:34.134816+0000","last_undegraded":"2024-02-26T04:55:34.134816+0000","last_fullsized":"2024-02-26T04:55:34.134816+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:20:22.938290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:54.151933+0000","last_change":"2024-02-26T04:55:27.197338+0000","last_active":"2024-02-26T04:55:54.151933+0000","last_peered":"2024-02-26T04:55:54.151933+0000","last_clean":"2024-02-26T04:55:54.151933+0000","last_became_active":"2024-02-26T04:55:27.196898+0000","last_became_peered":"2024-02-26T04:55:27.196898+0000","last_unstale":"2024-02-26T04:55:54.151933+0000","last_undegraded":"2024-02-26T04:55:54.151933+0000","last_fullsized":"2024-02-26T04:55:54.151933+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:27:09.635379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213239+0000","last_change":"2024-02-26T04:55:28.200665+0000","last_active":"2024-02-26T04:55:33.213239+0000","last_peered":"2024-02-26T04:55:33.213239+0000","last_clean":"2024-02-26T04:55:33.213239+0000","last_became_active":"2024-02-26T04:55:28.200329+0000","last_became_peered":"2024-02-26T04:55:28.200329+0000","last_unstale":"2024-02-26T04:55:33.213239+0000","last_undegraded":"2024-02-26T04:55:33.213239+0000","last_fullsized":"2024-02-26T04:55:33.213239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:27.090259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213899+0000","last_change":"2024-02-26T04:55:32.225861+0000","last_active":"2024-02-26T04:55:33.213899+0000","last_peered":"2024-02-26T04:55:33.213899+0000","last_clean":"2024-02-26T04:55:33.213899+0000","last_became_active":"2024-02-26T04:55:32.225187+0000","last_became_peered":"2024-02-26T04:55:32.225187+0000","last_unstale":"2024-02-26T04:55:33.213899+0000","last_undegraded":"2024-02-26T04:55:33.213899+0000","last_fullsized":"2024-02-26T04:55:33.213899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:14:09.077383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:53.550158+0000","last_change":"2024-02-26T04:55:27.200321+0000","last_active":"2024-02-26T04:55:53.550158+0000","last_peered":"2024-02-26T04:55:53.550158+0000","last_clean":"2024-02-26T04:55:53.550158+0000","last_became_active":"2024-02-26T04:55:27.199978+0000","last_became_peered":"2024-02-26T04:55:27.199978+0000","last_unstale":"2024-02-26T04:55:53.550158+0000","last_undegraded":"2024-02-26T04:55:53.550158+0000","last_fullsized":"2024-02-26T04:55:53.550158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:08:20.381010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134602+0000","last_change":"2024-02-26T04:55:28.213940+0000","last_active":"2024-02-26T04:55:34.134602+0000","last_peered":"2024-02-26T04:55:34.134602+0000","last_clean":"2024-02-26T04:55:34.134602+0000","last_became_active":"2024-02-26T04:55:28.213590+0000","last_became_peered":"2024-02-26T04:55:28.213590+0000","last_unstale":"2024-02-26T04:55:34.134602+0000","last_undegraded":"2024-02-26T04:55:34.134602+0000","last_fullsized":"2024-02-26T04:55:34.134602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:29:29.151391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134626+0000","last_change":"2024-02-26T04:55:32.232669+0000","last_active":"2024-02-26T04:55:34.134626+0000","last_peered":"2024-02-26T04:55:34.134626+0000","last_clean":"2024-02-26T04:55:34.134626+0000","last_became_active":"2024-02-26T04:55:32.232288+0000","last_became_peered":"2024-02-26T04:55:32.232288+0000","last_unstale":"2024-02-26T04:55:34.134626+0000","last_undegraded":"2024-02-26T04:55:34.134626+0000","last_fullsized":"2024-02-26T04:55:34.134626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:31:42.436516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134627+0000","last_change":"2024-02-26T04:55:27.205993+0000","last_active":"2024-02-26T04:55:34.134627+0000","last_peered":"2024-02-26T04:55:34.134627+0000","last_clean":"2024-02-26T04:55:34.134627+0000","last_became_active":"2024-02-26T04:55:27.205594+0000","last_became_peered":"2024-02-26T04:55:27.205594+0000","last_unstale":"2024-02-26T04:55:34.134627+0000","last_undegraded":"2024-02-26T04:55:34.134627+0000","last_fullsized":"2024-02-26T04:55:34.134627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:44:19.993925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770427+0000","last_change":"2024-02-26T04:55:28.203429+0000","last_active":"2024-02-26T04:55:33.770427+0000","last_peered":"2024-02-26T04:55:33.770427+0000","last_clean":"2024-02-26T04:55:33.770427+0000","last_became_active":"2024-02-26T04:55:28.203002+0000","last_became_peered":"2024-02-26T04:55:28.203002+0000","last_unstale":"2024-02-26T04:55:33.770427+0000","last_undegraded":"2024-02-26T04:55:33.770427+0000","last_fullsized":"2024-02-26T04:55:33.770427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:08:05.872241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.135146+0000","last_change":"2024-02-26T04:55:32.239323+0000","last_active":"2024-02-26T04:55:34.135146+0000","last_peered":"2024-02-26T04:55:34.135146+0000","last_clean":"2024-02-26T04:55:34.135146+0000","last_became_active":"2024-02-26T04:55:32.238843+0000","last_became_peered":"2024-02-26T04:55:32.238843+0000","last_unstale":"2024-02-26T04:55:34.135146+0000","last_undegraded":"2024-02-26T04:55:34.135146+0000","last_fullsized":"2024-02-26T04:55:34.135146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:39:56.645429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134409+0000","last_change":"2024-02-26T04:55:27.203222+0000","last_active":"2024-02-26T04:55:34.134409+0000","last_peered":"2024-02-26T04:55:34.134409+0000","last_clean":"2024-02-26T04:55:34.134409+0000","last_became_active":"2024-02-26T04:55:27.202845+0000","last_became_peered":"2024-02-26T04:55:27.202845+0000","last_unstale":"2024-02-26T04:55:34.134409+0000","last_undegraded":"2024-02-26T04:55:34.134409+0000","last_fullsized":"2024-02-26T04:55:34.134409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:16:07.524260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.118135+0000","last_change":"2024-02-26T04:55:28.209122+0000","last_active":"2024-02-26T04:55:34.118135+0000","last_peered":"2024-02-26T04:55:34.118135+0000","last_clean":"2024-02-26T04:55:34.118135+0000","last_became_active":"2024-02-26T04:55:28.207598+0000","last_became_peered":"2024-02-26T04:55:28.207598+0000","last_unstale":"2024-02-26T04:55:34.118135+0000","last_undegraded":"2024-02-26T04:55:34.118135+0000","last_fullsized":"2024-02-26T04:55:34.118135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:02:10.763515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213215+0000","last_change":"2024-02-26T04:55:32.227333+0000","last_active":"2024-02-26T04:55:33.213215+0000","last_peered":"2024-02-26T04:55:33.213215+0000","last_clean":"2024-02-26T04:55:33.213215+0000","last_became_active":"2024-02-26T04:55:32.226956+0000","last_became_peered":"2024-02-26T04:55:32.226956+0000","last_unstale":"2024-02-26T04:55:33.213215+0000","last_undegraded":"2024-02-26T04:55:33.213215+0000","last_fullsized":"2024-02-26T04:55:33.213215+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:51:40.710242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213253+0000","last_change":"2024-02-26T04:55:27.198781+0000","last_active":"2024-02-26T04:55:33.213253+0000","last_peered":"2024-02-26T04:55:33.213253+0000","last_clean":"2024-02-26T04:55:33.213253+0000","last_became_active":"2024-02-26T04:55:27.198288+0000","last_became_peered":"2024-02-26T04:55:27.198288+0000","last_unstale":"2024-02-26T04:55:33.213253+0000","last_undegraded":"2024-02-26T04:55:33.213253+0000","last_fullsized":"2024-02-26T04:55:33.213253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:52:18.665816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.117975+0000","last_change":"2024-02-26T04:55:28.213044+0000","last_active":"2024-02-26T04:55:34.117975+0000","last_peered":"2024-02-26T04:55:34.117975+0000","last_clean":"2024-02-26T04:55:34.117975+0000","last_became_active":"2024-02-26T04:55:28.212712+0000","last_became_peered":"2024-02-26T04:55:28.212712+0000","last_unstale":"2024-02-26T04:55:34.117975+0000","last_undegraded":"2024-02-26T04:55:34.117975+0000","last_fullsized":"2024-02-26T04:55:34.117975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:02:09.155551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213698+0000","last_change":"2024-02-26T04:55:32.231867+0000","last_active":"2024-02-26T04:55:33.213698+0000","last_peered":"2024-02-26T04:55:33.213698+0000","last_clean":"2024-02-26T04:55:33.213698+0000","last_became_active":"2024-02-26T04:55:32.231450+0000","last_became_peered":"2024-02-26T04:55:32.231450+0000","last_unstale":"2024-02-26T04:55:33.213698+0000","last_undegraded":"2024-02-26T04:55:33.213698+0000","last_fullsized":"2024-02-26T04:55:33.213698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:12:14.367427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134231+0000","last_change":"2024-02-26T04:55:28.207746+0000","last_active":"2024-02-26T04:55:34.134231+0000","last_peered":"2024-02-26T04:55:34.134231+0000","last_clean":"2024-02-26T04:55:34.134231+0000","last_became_active":"2024-02-26T04:55:28.207438+0000","last_became_peered":"2024-02-26T04:55:28.207438+0000","last_unstale":"2024-02-26T04:55:34.134231+0000","last_undegraded":"2024-02-26T04:55:34.134231+0000","last_fullsized":"2024-02-26T04:55:34.134231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:09.409635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.212835+0000","last_change":"2024-02-26T04:55:27.192233+0000","last_active":"2024-02-26T04:55:33.212835+0000","last_peered":"2024-02-26T04:55:33.212835+0000","last_clean":"2024-02-26T04:55:33.212835+0000","last_became_active":"2024-02-26T04:55:27.191868+0000","last_became_peered":"2024-02-26T04:55:27.191868+0000","last_unstale":"2024-02-26T04:55:33.212835+0000","last_undegraded":"2024-02-26T04:55:33.212835+0000","last_fullsized":"2024-02-26T04:55:33.212835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:37:58.430504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213634+0000","last_change":"2024-02-26T04:55:32.226577+0000","last_active":"2024-02-26T04:55:33.213634+0000","last_peered":"2024-02-26T04:55:33.213634+0000","last_clean":"2024-02-26T04:55:33.213634+0000","last_became_active":"2024-02-26T04:55:32.226148+0000","last_became_peered":"2024-02-26T04:55:32.226148+0000","last_unstale":"2024-02-26T04:55:33.213634+0000","last_undegraded":"2024-02-26T04:55:33.213634+0000","last_fullsized":"2024-02-26T04:55:33.213634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:34:53.099211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134303+0000","last_change":"2024-02-26T04:55:28.217865+0000","last_active":"2024-02-26T04:55:34.134303+0000","last_peered":"2024-02-26T04:55:34.134303+0000","last_clean":"2024-02-26T04:55:34.134303+0000","last_became_active":"2024-02-26T04:55:28.217075+0000","last_became_peered":"2024-02-26T04:55:28.217075+0000","last_unstale":"2024-02-26T04:55:34.134303+0000","last_undegraded":"2024-02-26T04:55:34.134303+0000","last_fullsized":"2024-02-26T04:55:34.134303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:30:38.720694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:54.147218+0000","last_change":"2024-02-26T04:55:27.204300+0000","last_active":"2024-02-26T04:55:54.147218+0000","last_peered":"2024-02-26T04:55:54.147218+0000","last_clean":"2024-02-26T04:55:54.147218+0000","last_became_active":"2024-02-26T04:55:27.203933+0000","last_became_peered":"2024-02-26T04:55:27.203933+0000","last_unstale":"2024-02-26T04:55:54.147218+0000","last_undegraded":"2024-02-26T04:55:54.147218+0000","last_fullsized":"2024-02-26T04:55:54.147218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:19:36.247209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213080+0000","last_change":"2024-02-26T04:54:13.145088+0000","last_active":"2024-02-26T04:55:33.213080+0000","last_peered":"2024-02-26T04:55:33.213080+0000","last_clean":"2024-02-26T04:55:33.213080+0000","last_became_active":"2024-02-26T04:54:13.144657+0000","last_became_peered":"2024-02-26T04:54:13.144657+0000","last_unstale":"2024-02-26T04:55:33.213080+0000","last_undegraded":"2024-02-26T04:55:33.213080+0000","last_fullsized":"2024-02-26T04:55:33.213080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:50:45.246330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:50:45.246330+0000","last_clean_scrub_stamp":"2024-02-26T04:50:45.246330+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:00:50.842020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134999+0000","last_change":"2024-02-26T04:55:32.240124+0000","last_active":"2024-02-26T04:55:34.134999+0000","last_peered":"2024-02-26T04:55:34.134999+0000","last_clean":"2024-02-26T04:55:34.134999+0000","last_became_active":"2024-02-26T04:55:32.239624+0000","last_became_peered":"2024-02-26T04:55:32.239624+0000","last_unstale":"2024-02-26T04:55:34.134999+0000","last_undegraded":"2024-02-26T04:55:34.134999+0000","last_fullsized":"2024-02-26T04:55:34.134999+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:40:33.770917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.212704+0000","last_change":"2024-02-26T04:55:28.204631+0000","last_active":"2024-02-26T04:55:33.212704+0000","last_peered":"2024-02-26T04:55:33.212704+0000","last_clean":"2024-02-26T04:55:33.212704+0000","last_became_active":"2024-02-26T04:55:28.204274+0000","last_became_peered":"2024-02-26T04:55:28.204274+0000","last_unstale":"2024-02-26T04:55:33.212704+0000","last_undegraded":"2024-02-26T04:55:33.212704+0000","last_fullsized":"2024-02-26T04:55:33.212704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:18:18.336015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213144+0000","last_change":"2024-02-26T04:55:27.201719+0000","last_active":"2024-02-26T04:55:33.213144+0000","last_peered":"2024-02-26T04:55:33.213144+0000","last_clean":"2024-02-26T04:55:33.213144+0000","last_became_active":"2024-02-26T04:55:27.199060+0000","last_became_peered":"2024-02-26T04:55:27.199060+0000","last_unstale":"2024-02-26T04:55:33.213144+0000","last_undegraded":"2024-02-26T04:55:33.213144+0000","last_fullsized":"2024-02-26T04:55:33.213144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:37:37.620970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770175+0000","last_change":"2024-02-26T04:55:32.228900+0000","last_active":"2024-02-26T04:55:33.770175+0000","last_peered":"2024-02-26T04:55:33.770175+0000","last_clean":"2024-02-26T04:55:33.770175+0000","last_became_active":"2024-02-26T04:55:32.228262+0000","last_became_peered":"2024-02-26T04:55:32.228262+0000","last_unstale":"2024-02-26T04:55:33.770175+0000","last_undegraded":"2024-02-26T04:55:33.770175+0000","last_fullsized":"2024-02-26T04:55:33.770175+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:33.069756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134886+0000","last_change":"2024-02-26T04:55:27.205264+0000","last_active":"2024-02-26T04:55:34.134886+0000","last_peered":"2024-02-26T04:55:34.134886+0000","last_clean":"2024-02-26T04:55:34.134886+0000","last_became_active":"2024-02-26T04:55:27.204882+0000","last_became_peered":"2024-02-26T04:55:27.204882+0000","last_unstale":"2024-02-26T04:55:34.134886+0000","last_undegraded":"2024-02-26T04:55:34.134886+0000","last_fullsized":"2024-02-26T04:55:34.134886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:14:19.497725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213379+0000","last_change":"2024-02-26T04:55:28.200264+0000","last_active":"2024-02-26T04:55:33.213379+0000","last_peered":"2024-02-26T04:55:33.213379+0000","last_clean":"2024-02-26T04:55:33.213379+0000","last_became_active":"2024-02-26T04:55:28.199913+0000","last_became_peered":"2024-02-26T04:55:28.199913+0000","last_unstale":"2024-02-26T04:55:33.213379+0000","last_undegraded":"2024-02-26T04:55:33.213379+0000","last_fullsized":"2024-02-26T04:55:33.213379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:54:51.529149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134861+0000","last_change":"2024-02-26T04:55:32.233362+0000","last_active":"2024-02-26T04:55:34.134861+0000","last_peered":"2024-02-26T04:55:34.134861+0000","last_clean":"2024-02-26T04:55:34.134861+0000","last_became_active":"2024-02-26T04:55:32.232907+0000","last_became_peered":"2024-02-26T04:55:32.232907+0000","last_unstale":"2024-02-26T04:55:34.134861+0000","last_undegraded":"2024-02-26T04:55:34.134861+0000","last_fullsized":"2024-02-26T04:55:34.134861+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:49:11.889863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:53.549361+0000","last_change":"2024-02-26T04:55:27.207371+0000","last_active":"2024-02-26T04:55:53.549361+0000","last_peered":"2024-02-26T04:55:53.549361+0000","last_clean":"2024-02-26T04:55:53.549361+0000","last_became_active":"2024-02-26T04:55:27.207047+0000","last_became_peered":"2024-02-26T04:55:27.207047+0000","last_unstale":"2024-02-26T04:55:53.549361+0000","last_undegraded":"2024-02-26T04:55:53.549361+0000","last_fullsized":"2024-02-26T04:55:53.549361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:29.575633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134823+0000","last_change":"2024-02-26T04:55:28.207132+0000","last_active":"2024-02-26T04:55:34.134823+0000","last_peered":"2024-02-26T04:55:34.134823+0000","last_clean":"2024-02-26T04:55:34.134823+0000","last_became_active":"2024-02-26T04:55:28.206376+0000","last_became_peered":"2024-02-26T04:55:28.206376+0000","last_unstale":"2024-02-26T04:55:34.134823+0000","last_undegraded":"2024-02-26T04:55:34.134823+0000","last_fullsized":"2024-02-26T04:55:34.134823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:17:53.802386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.104876+0000","last_change":"2024-02-26T04:55:32.226381+0000","last_active":"2024-02-26T04:55:34.104876+0000","last_peered":"2024-02-26T04:55:34.104876+0000","last_clean":"2024-02-26T04:55:34.104876+0000","last_became_active":"2024-02-26T04:55:32.225940+0000","last_became_peered":"2024-02-26T04:55:32.225940+0000","last_unstale":"2024-02-26T04:55:34.104876+0000","last_undegraded":"2024-02-26T04:55:34.104876+0000","last_fullsized":"2024-02-26T04:55:34.104876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:51:20.190514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:52.541337+0000","last_change":"2024-02-26T04:55:27.200246+0000","last_active":"2024-02-26T04:55:52.541337+0000","last_peered":"2024-02-26T04:55:52.541337+0000","last_clean":"2024-02-26T04:55:52.541337+0000","last_became_active":"2024-02-26T04:55:27.199906+0000","last_became_peered":"2024-02-26T04:55:27.199906+0000","last_unstale":"2024-02-26T04:55:52.541337+0000","last_undegraded":"2024-02-26T04:55:52.541337+0000","last_fullsized":"2024-02-26T04:55:52.541337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:29.037230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.214405+0000","last_change":"2024-02-26T04:55:28.215357+0000","last_active":"2024-02-26T04:55:33.214405+0000","last_peered":"2024-02-26T04:55:33.214405+0000","last_clean":"2024-02-26T04:55:33.214405+0000","last_became_active":"2024-02-26T04:55:28.212379+0000","last_became_peered":"2024-02-26T04:55:28.212379+0000","last_unstale":"2024-02-26T04:55:33.214405+0000","last_undegraded":"2024-02-26T04:55:33.214405+0000","last_fullsized":"2024-02-26T04:55:33.214405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:49.560837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931647+0000","last_change":"2024-02-26T04:55:32.227765+0000","last_active":"2024-02-26T04:55:33.931647+0000","last_peered":"2024-02-26T04:55:33.931647+0000","last_clean":"2024-02-26T04:55:33.931647+0000","last_became_active":"2024-02-26T04:55:32.227346+0000","last_became_peered":"2024-02-26T04:55:32.227346+0000","last_unstale":"2024-02-26T04:55:33.931647+0000","last_undegraded":"2024-02-26T04:55:33.931647+0000","last_fullsized":"2024-02-26T04:55:33.931647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:05:12.634958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:52.542768+0000","last_change":"2024-02-26T04:55:27.199979+0000","last_active":"2024-02-26T04:55:52.542768+0000","last_peered":"2024-02-26T04:55:52.542768+0000","last_clean":"2024-02-26T04:55:52.542768+0000","last_became_active":"2024-02-26T04:55:27.199618+0000","last_became_peered":"2024-02-26T04:55:27.199618+0000","last_unstale":"2024-02-26T04:55:52.542768+0000","last_undegraded":"2024-02-26T04:55:52.542768+0000","last_fullsized":"2024-02-26T04:55:52.542768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:20:59.649473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134000+0000","last_change":"2024-02-26T04:55:28.202253+0000","last_active":"2024-02-26T04:55:34.134000+0000","last_peered":"2024-02-26T04:55:34.134000+0000","last_clean":"2024-02-26T04:55:34.134000+0000","last_became_active":"2024-02-26T04:55:28.199825+0000","last_became_peered":"2024-02-26T04:55:28.199825+0000","last_unstale":"2024-02-26T04:55:34.134000+0000","last_undegraded":"2024-02-26T04:55:34.134000+0000","last_fullsized":"2024-02-26T04:55:34.134000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:21:59.148995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134880+0000","last_change":"2024-02-26T04:55:32.239327+0000","last_active":"2024-02-26T04:55:34.134880+0000","last_peered":"2024-02-26T04:55:34.134880+0000","last_clean":"2024-02-26T04:55:34.134880+0000","last_became_active":"2024-02-26T04:55:32.238742+0000","last_became_peered":"2024-02-26T04:55:32.238742+0000","last_unstale":"2024-02-26T04:55:34.134880+0000","last_undegraded":"2024-02-26T04:55:34.134880+0000","last_fullsized":"2024-02-26T04:55:34.134880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:38:08.821549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.133939+0000","last_change":"2024-02-26T04:55:27.206529+0000","last_active":"2024-02-26T04:55:34.133939+0000","last_peered":"2024-02-26T04:55:34.133939+0000","last_clean":"2024-02-26T04:55:34.133939+0000","last_became_active":"2024-02-26T04:55:27.206203+0000","last_became_peered":"2024-02-26T04:55:27.206203+0000","last_unstale":"2024-02-26T04:55:34.133939+0000","last_undegraded":"2024-02-26T04:55:34.133939+0000","last_fullsized":"2024-02-26T04:55:34.133939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:24.491832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.769955+0000","last_change":"2024-02-26T04:55:28.205915+0000","last_active":"2024-02-26T04:55:33.769955+0000","last_peered":"2024-02-26T04:55:33.769955+0000","last_clean":"2024-02-26T04:55:33.769955+0000","last_became_active":"2024-02-26T04:55:28.205576+0000","last_became_peered":"2024-02-26T04:55:28.205576+0000","last_unstale":"2024-02-26T04:55:33.769955+0000","last_undegraded":"2024-02-26T04:55:33.769955+0000","last_fullsized":"2024-02-26T04:55:33.769955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:18:54.639778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931418+0000","last_change":"2024-02-26T04:55:32.226253+0000","last_active":"2024-02-26T04:55:33.931418+0000","last_peered":"2024-02-26T04:55:33.931418+0000","last_clean":"2024-02-26T04:55:33.931418+0000","last_became_active":"2024-02-26T04:55:32.225815+0000","last_became_peered":"2024-02-26T04:55:32.225815+0000","last_unstale":"2024-02-26T04:55:33.931418+0000","last_undegraded":"2024-02-26T04:55:33.931418+0000","last_fullsized":"2024-02-26T04:55:33.931418+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:22.764756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.104806+0000","last_change":"2024-02-26T04:55:27.209066+0000","last_active":"2024-02-26T04:55:34.104806+0000","last_peered":"2024-02-26T04:55:34.104806+0000","last_clean":"2024-02-26T04:55:34.104806+0000","last_became_active":"2024-02-26T04:55:27.208556+0000","last_became_peered":"2024-02-26T04:55:27.208556+0000","last_unstale":"2024-02-26T04:55:34.104806+0000","last_undegraded":"2024-02-26T04:55:34.104806+0000","last_fullsized":"2024-02-26T04:55:34.104806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:14:34.950262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134229+0000","last_change":"2024-02-26T04:55:28.213065+0000","last_active":"2024-02-26T04:55:34.134229+0000","last_peered":"2024-02-26T04:55:34.134229+0000","last_clean":"2024-02-26T04:55:34.134229+0000","last_became_active":"2024-02-26T04:55:28.211112+0000","last_became_peered":"2024-02-26T04:55:28.211112+0000","last_unstale":"2024-02-26T04:55:34.134229+0000","last_undegraded":"2024-02-26T04:55:34.134229+0000","last_fullsized":"2024-02-26T04:55:34.134229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:41:10.070183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.931604+0000","last_change":"2024-02-26T04:55:32.229838+0000","last_active":"2024-02-26T04:55:33.931604+0000","last_peered":"2024-02-26T04:55:33.931604+0000","last_clean":"2024-02-26T04:55:33.931604+0000","last_became_active":"2024-02-26T04:55:32.228696+0000","last_became_peered":"2024-02-26T04:55:32.228696+0000","last_unstale":"2024-02-26T04:55:33.931604+0000","last_undegraded":"2024-02-26T04:55:33.931604+0000","last_fullsized":"2024-02-26T04:55:33.931604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:59:42.359301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:52.543601+0000","last_change":"2024-02-26T04:55:27.205688+0000","last_active":"2024-02-26T04:55:52.543601+0000","last_peered":"2024-02-26T04:55:52.543601+0000","last_clean":"2024-02-26T04:55:52.543601+0000","last_became_active":"2024-02-26T04:55:27.205306+0000","last_became_peered":"2024-02-26T04:55:27.205306+0000","last_unstale":"2024-02-26T04:55:52.543601+0000","last_undegraded":"2024-02-26T04:55:52.543601+0000","last_fullsized":"2024-02-26T04:55:52.543601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:23:50.203565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213413+0000","last_change":"2024-02-26T04:55:28.210073+0000","last_active":"2024-02-26T04:55:33.213413+0000","last_peered":"2024-02-26T04:55:33.213413+0000","last_clean":"2024-02-26T04:55:33.213413+0000","last_became_active":"2024-02-26T04:55:28.208475+0000","last_became_peered":"2024-02-26T04:55:28.208475+0000","last_unstale":"2024-02-26T04:55:33.213413+0000","last_undegraded":"2024-02-26T04:55:33.213413+0000","last_fullsized":"2024-02-26T04:55:33.213413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:06:49.513916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134148+0000","last_change":"2024-02-26T04:55:32.228708+0000","last_active":"2024-02-26T04:55:34.134148+0000","last_peered":"2024-02-26T04:55:34.134148+0000","last_clean":"2024-02-26T04:55:34.134148+0000","last_became_active":"2024-02-26T04:55:32.228304+0000","last_became_peered":"2024-02-26T04:55:32.228304+0000","last_unstale":"2024-02-26T04:55:34.134148+0000","last_undegraded":"2024-02-26T04:55:34.134148+0000","last_fullsized":"2024-02-26T04:55:34.134148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:53:02.390712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:53.551484+0000","last_change":"2024-02-26T04:55:27.199761+0000","last_active":"2024-02-26T04:55:53.551484+0000","last_peered":"2024-02-26T04:55:53.551484+0000","last_clean":"2024-02-26T04:55:53.551484+0000","last_became_active":"2024-02-26T04:55:27.199113+0000","last_became_peered":"2024-02-26T04:55:27.199113+0000","last_unstale":"2024-02-26T04:55:53.551484+0000","last_undegraded":"2024-02-26T04:55:53.551484+0000","last_fullsized":"2024-02-26T04:55:53.551484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:14:10.631886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213832+0000","last_change":"2024-02-26T04:55:28.211484+0000","last_active":"2024-02-26T04:55:33.213832+0000","last_peered":"2024-02-26T04:55:33.213832+0000","last_clean":"2024-02-26T04:55:33.213832+0000","last_became_active":"2024-02-26T04:55:28.211131+0000","last_became_peered":"2024-02-26T04:55:28.211131+0000","last_unstale":"2024-02-26T04:55:33.213832+0000","last_undegraded":"2024-02-26T04:55:33.213832+0000","last_fullsized":"2024-02-26T04:55:33.213832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:54:09.393847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213586+0000","last_change":"2024-02-26T04:55:32.227493+0000","last_active":"2024-02-26T04:55:33.213586+0000","last_peered":"2024-02-26T04:55:33.213586+0000","last_clean":"2024-02-26T04:55:33.213586+0000","last_became_active":"2024-02-26T04:55:32.227106+0000","last_became_peered":"2024-02-26T04:55:32.227106+0000","last_unstale":"2024-02-26T04:55:33.213586+0000","last_undegraded":"2024-02-26T04:55:33.213586+0000","last_fullsized":"2024-02-26T04:55:33.213586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:33.040966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134963+0000","last_change":"2024-02-26T04:55:27.205331+0000","last_active":"2024-02-26T04:55:34.134963+0000","last_peered":"2024-02-26T04:55:34.134963+0000","last_clean":"2024-02-26T04:55:34.134963+0000","last_became_active":"2024-02-26T04:55:27.204998+0000","last_became_peered":"2024-02-26T04:55:27.204998+0000","last_unstale":"2024-02-26T04:55:34.134963+0000","last_undegraded":"2024-02-26T04:55:34.134963+0000","last_fullsized":"2024-02-26T04:55:34.134963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:10:03.671016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213586+0000","last_change":"2024-02-26T04:55:28.222116+0000","last_active":"2024-02-26T04:55:33.213586+0000","last_peered":"2024-02-26T04:55:33.213586+0000","last_clean":"2024-02-26T04:55:33.213586+0000","last_became_active":"2024-02-26T04:55:28.221758+0000","last_became_peered":"2024-02-26T04:55:28.221758+0000","last_unstale":"2024-02-26T04:55:33.213586+0000","last_undegraded":"2024-02-26T04:55:33.213586+0000","last_fullsized":"2024-02-26T04:55:33.213586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:35:57.637410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134081+0000","last_change":"2024-02-26T04:55:32.232222+0000","last_active":"2024-02-26T04:55:34.134081+0000","last_peered":"2024-02-26T04:55:34.134081+0000","last_clean":"2024-02-26T04:55:34.134081+0000","last_became_active":"2024-02-26T04:55:32.231836+0000","last_became_peered":"2024-02-26T04:55:32.231836+0000","last_unstale":"2024-02-26T04:55:34.134081+0000","last_undegraded":"2024-02-26T04:55:34.134081+0000","last_fullsized":"2024-02-26T04:55:34.134081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:18:00.766266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213725+0000","last_change":"2024-02-26T04:55:27.200385+0000","last_active":"2024-02-26T04:55:33.213725+0000","last_peered":"2024-02-26T04:55:33.213725+0000","last_clean":"2024-02-26T04:55:33.213725+0000","last_became_active":"2024-02-26T04:55:27.200046+0000","last_became_peered":"2024-02-26T04:55:27.200046+0000","last_unstale":"2024-02-26T04:55:33.213725+0000","last_undegraded":"2024-02-26T04:55:33.213725+0000","last_fullsized":"2024-02-26T04:55:33.213725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:01:58.112457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213819+0000","last_change":"2024-02-26T04:55:28.220702+0000","last_active":"2024-02-26T04:55:33.213819+0000","last_peered":"2024-02-26T04:55:33.213819+0000","last_clean":"2024-02-26T04:55:33.213819+0000","last_became_active":"2024-02-26T04:55:28.220386+0000","last_became_peered":"2024-02-26T04:55:28.220386+0000","last_unstale":"2024-02-26T04:55:33.213819+0000","last_undegraded":"2024-02-26T04:55:33.213819+0000","last_fullsized":"2024-02-26T04:55:33.213819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:40:37.774831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770054+0000","last_change":"2024-02-26T04:55:28.206409+0000","last_active":"2024-02-26T04:55:33.770054+0000","last_peered":"2024-02-26T04:55:33.770054+0000","last_clean":"2024-02-26T04:55:33.770054+0000","last_became_active":"2024-02-26T04:55:28.206046+0000","last_became_peered":"2024-02-26T04:55:28.206046+0000","last_unstale":"2024-02-26T04:55:33.770054+0000","last_undegraded":"2024-02-26T04:55:33.770054+0000","last_fullsized":"2024-02-26T04:55:33.770054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:35:43.277376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.105094+0000","last_change":"2024-02-26T04:55:28.216146+0000","last_active":"2024-02-26T04:55:34.105094+0000","last_peered":"2024-02-26T04:55:34.105094+0000","last_clean":"2024-02-26T04:55:34.105094+0000","last_became_active":"2024-02-26T04:55:28.215800+0000","last_became_peered":"2024-02-26T04:55:28.215800+0000","last_unstale":"2024-02-26T04:55:34.105094+0000","last_undegraded":"2024-02-26T04:55:34.105094+0000","last_fullsized":"2024-02-26T04:55:34.105094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:18:47.902694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.104662+0000","last_change":"2024-02-26T04:55:28.221587+0000","last_active":"2024-02-26T04:55:34.104662+0000","last_peered":"2024-02-26T04:55:34.104662+0000","last_clean":"2024-02-26T04:55:34.104662+0000","last_became_active":"2024-02-26T04:55:28.220990+0000","last_became_peered":"2024-02-26T04:55:28.220990+0000","last_unstale":"2024-02-26T04:55:34.104662+0000","last_undegraded":"2024-02-26T04:55:34.104662+0000","last_fullsized":"2024-02-26T04:55:34.104662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:24:13.302081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.214169+0000","last_change":"2024-02-26T04:55:28.220586+0000","last_active":"2024-02-26T04:55:33.214169+0000","last_peered":"2024-02-26T04:55:33.214169+0000","last_clean":"2024-02-26T04:55:33.214169+0000","last_became_active":"2024-02-26T04:55:28.219585+0000","last_became_peered":"2024-02-26T04:55:28.219585+0000","last_unstale":"2024-02-26T04:55:33.214169+0000","last_undegraded":"2024-02-26T04:55:33.214169+0000","last_fullsized":"2024-02-26T04:55:33.214169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:41.786142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213364+0000","last_change":"2024-02-26T04:55:28.204599+0000","last_active":"2024-02-26T04:55:33.213364+0000","last_peered":"2024-02-26T04:55:33.213364+0000","last_clean":"2024-02-26T04:55:33.213364+0000","last_became_active":"2024-02-26T04:55:28.204252+0000","last_became_peered":"2024-02-26T04:55:28.204252+0000","last_unstale":"2024-02-26T04:55:33.213364+0000","last_undegraded":"2024-02-26T04:55:33.213364+0000","last_fullsized":"2024-02-26T04:55:33.213364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:30:19.807897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770536+0000","last_change":"2024-02-26T04:55:28.214233+0000","last_active":"2024-02-26T04:55:33.770536+0000","last_peered":"2024-02-26T04:55:33.770536+0000","last_clean":"2024-02-26T04:55:33.770536+0000","last_became_active":"2024-02-26T04:55:28.213927+0000","last_became_peered":"2024-02-26T04:55:28.213927+0000","last_unstale":"2024-02-26T04:55:33.770536+0000","last_undegraded":"2024-02-26T04:55:33.770536+0000","last_fullsized":"2024-02-26T04:55:33.770536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:31:07.967134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.105257+0000","last_change":"2024-02-26T04:55:28.217610+0000","last_active":"2024-02-26T04:55:34.105257+0000","last_peered":"2024-02-26T04:55:34.105257+0000","last_clean":"2024-02-26T04:55:34.105257+0000","last_became_active":"2024-02-26T04:55:28.217263+0000","last_became_peered":"2024-02-26T04:55:28.217263+0000","last_unstale":"2024-02-26T04:55:34.105257+0000","last_undegraded":"2024-02-26T04:55:34.105257+0000","last_fullsized":"2024-02-26T04:55:34.105257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:28:18.831887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134962+0000","last_change":"2024-02-26T04:55:28.207297+0000","last_active":"2024-02-26T04:55:34.134962+0000","last_peered":"2024-02-26T04:55:34.134962+0000","last_clean":"2024-02-26T04:55:34.134962+0000","last_became_active":"2024-02-26T04:55:28.206811+0000","last_became_peered":"2024-02-26T04:55:28.206811+0000","last_unstale":"2024-02-26T04:55:34.134962+0000","last_undegraded":"2024-02-26T04:55:34.134962+0000","last_fullsized":"2024-02-26T04:55:34.134962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:12:01.254968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134098+0000","last_change":"2024-02-26T04:55:28.217247+0000","last_active":"2024-02-26T04:55:34.134098+0000","last_peered":"2024-02-26T04:55:34.134098+0000","last_clean":"2024-02-26T04:55:34.134098+0000","last_became_active":"2024-02-26T04:55:28.214691+0000","last_became_peered":"2024-02-26T04:55:28.214691+0000","last_unstale":"2024-02-26T04:55:34.134098+0000","last_undegraded":"2024-02-26T04:55:34.134098+0000","last_fullsized":"2024-02-26T04:55:34.134098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:27:45.564520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134033+0000","last_change":"2024-02-26T04:55:28.210491+0000","last_active":"2024-02-26T04:55:34.134033+0000","last_peered":"2024-02-26T04:55:34.134033+0000","last_clean":"2024-02-26T04:55:34.134033+0000","last_became_active":"2024-02-26T04:55:28.210144+0000","last_became_peered":"2024-02-26T04:55:28.210144+0000","last_unstale":"2024-02-26T04:55:34.134033+0000","last_undegraded":"2024-02-26T04:55:34.134033+0000","last_fullsized":"2024-02-26T04:55:34.134033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:11.214716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:34.134235+0000","last_change":"2024-02-26T04:55:28.216037+0000","last_active":"2024-02-26T04:55:34.134235+0000","last_peered":"2024-02-26T04:55:34.134235+0000","last_clean":"2024-02-26T04:55:34.134235+0000","last_became_active":"2024-02-26T04:55:28.215712+0000","last_became_peered":"2024-02-26T04:55:28.215712+0000","last_unstale":"2024-02-26T04:55:34.134235+0000","last_undegraded":"2024-02-26T04:55:34.134235+0000","last_fullsized":"2024-02-26T04:55:34.134235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:38:55.437642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.213346+0000","last_change":"2024-02-26T04:55:28.205469+0000","last_active":"2024-02-26T04:55:33.213346+0000","last_peered":"2024-02-26T04:55:33.213346+0000","last_clean":"2024-02-26T04:55:33.213346+0000","last_became_active":"2024-02-26T04:55:28.205137+0000","last_became_peered":"2024-02-26T04:55:28.205137+0000","last_unstale":"2024-02-26T04:55:33.213346+0000","last_undegraded":"2024-02-26T04:55:33.213346+0000","last_fullsized":"2024-02-26T04:55:33.213346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:50.425010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-26T04:55:33.770144+0000","last_change":"2024-02-26T04:55:28.213066+0000","last_active":"2024-02-26T04:55:33.770144+0000","last_peered":"2024-02-26T04:55:33.770144+0000","last_clean":"2024-02-26T04:55:33.770144+0000","last_became_active":"2024-02-26T04:55:28.212706+0000","last_became_peered":"2024-02-26T04:55:28.212706+0000","last_unstale":"2024-02-26T04:55:33.770144+0000","last_undegraded":"2024-02-26T04:55:33.770144+0000","last_fullsized":"2024-02-26T04:55:33.770144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:21:24.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281347,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":13872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":22,"num_read_kb":20,"num_write":116,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":68735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":477215,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57599999999999996}]},{"osd":1,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.623},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.72699999999999998}]},{"osd":2,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":4,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59799999999999998}]},{"osd":5,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.502},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.45600000000000002}]},{"osd":6,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.60899999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.63400000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":9,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.71299999999999997}]},{"osd":10,"last update":"Mon Feb 26 04:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.78800000000000003}]}]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1073003,"data_compressed":50407,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.34499999999999997}]},{"osd":2,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.623}]},{"osd":3,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.47699999999999998}]},{"osd":4,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.439},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.495}]},{"osd":5,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65000000000000002}]},{"osd":6,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.53200000000000003}]},{"osd":7,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.72699999999999998}]},{"osd":8,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.69199999999999995}]},{"osd":9,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.66000000000000003}]},{"osd":11,"last update":"Mon Feb 26 04:55:37 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.67100000000000004}]}]},{"osd":9,"up_from":67,"seq":287762808861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":478033,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.46700000000000003,"15min":0.48199999999999998},"min":{"1min":0.29899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59199999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.48799999999999999,"15min":0.502},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.28100000000000003}]},{"osd":1,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.63400000000000001,"15min":0.65000000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57699999999999996}]},{"osd":2,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67100000000000004,"15min":0.67700000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.77700000000000002,"15min":0.80900000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99399999999999999,"5min":3.173,"15min":3.173},"last":0.40899999999999997}]},{"osd":3,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.51400000000000001,"15min":0.52800000000000002},"min":{"1min":0.28399999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.65100000000000002,"5min":0.753,"15min":0.753},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.52400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.30599999999999999}]},{"osd":4,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.66800000000000004,"15min":0.69199999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82199999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.496},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.65400000000000003,"15min":0.67000000000000004},"min":{"1min":0.36499999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.51700000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.69799999999999995,"15min":0.70999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.377}]},{"osd":6,"last update":"Mon Feb 26 04:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.73799999999999999,"15min":0.76300000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.63},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76400000000000001,"15min":0.78200000000000003},"min":{"1min":0.443,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46100000000000002}]},{"osd":8,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.90400000000000003,"15min":0.93799999999999994},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.278,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.434},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.79200000000000004,"15min":0.81100000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.56200000000000006}]},{"osd":10,"last update":"Mon Feb 26 04:55:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.61199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":8,"up_from":59,"seq":253403070498,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":477841,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.36199999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.626},{"interface":"front","average":{"1min":0.502,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.745,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60699999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.79100000000000004}]},{"osd":2,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42699999999999999,"15min":0.42599999999999999},"min":{"1min":0.32600000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.433,"15min":0.42999999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.51700000000000002}]},{"osd":3,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.65700000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55900000000000005,"15min":0.56399999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71899999999999997,"5min":0.755,"15min":0.755},"last":0.82499999999999996}]},{"osd":4,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.42599999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78600000000000003,"5min":0.871,"15min":0.871},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75900000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.754}]},{"osd":5,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76700000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67100000000000004,"15min":0.67400000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.93899999999999995}]},{"osd":7,"last update":"Mon Feb 26 04:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67000000000000004,"15min":0.66400000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.98499999999999999}]},{"osd":9,"last update":"Mon Feb 26 04:55:46 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.68799999999999994,"15min":0.67400000000000004},"min":{"1min":0.49199999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.874}]},{"osd":10,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.026}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]}]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1015808,"data_stored":1068165,"data_compressed":50407,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67000000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.40799999999999997}]},{"osd":1,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.41699999999999998,"15min":0.41599999999999998},"min":{"1min":0.28499999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.439,"15min":0.437},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.71999999999999997}]},{"osd":2,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.41199999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72399999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.75900000000000001}]},{"osd":3,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70899999999999996,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.73399999999999999,"5min":0.877,"15min":0.877},"last":0.77200000000000002}]},{"osd":4,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53100000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80800000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69899999999999995}]},{"osd":5,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.624},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.45200000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83099999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.84699999999999998}]},{"osd":6,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68799999999999994,"15min":0.68700000000000006},"min":{"1min":0.57099999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92300000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67600000000000005,"15min":0.67700000000000005},"min":{"1min":0.47299999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94099999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.875}]},{"osd":8,"last update":"Mon Feb 26 04:55:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72699999999999998,"15min":0.73299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.86399999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.89000000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.92800000000000005}]},{"osd":9,"last update":"Mon Feb 26 04:55:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.54100000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.89700000000000002}]},{"osd":10,"last update":"Mon Feb 26 04:55:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.98199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":962560,"data_stored":477215,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40200000000000002,"15min":0.38300000000000001},"min":{"1min":0.32200000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.442,"15min":0.42399999999999999},"min":{"1min":0.34000000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.72199999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45300000000000001}]},{"osd":1,"last update":"Mon Feb 26 04:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59199999999999997,"15min":0.56999999999999995},"min":{"1min":0.48299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.51200000000000001}]},{"osd":2,"last update":"Mon Feb 26 04:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.79000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.94799999999999995,"15min":1.0860000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.72899999999999998,"5min":14.884,"15min":14.884},"last":0.496}]},{"osd":3,"last update":"Mon Feb 26 04:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.45400000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82699999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.55200000000000005}]},{"osd":4,"last update":"Mon Feb 26 04:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65400000000000003,"15min":0.63400000000000001},"min":{"1min":0.499,"5min":0.38,"15min":0.38},"max":{"1min":0.84199999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":1.0049999999999999,"15min":1.143},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96899999999999997,"5min":14.898,"15min":14.898},"last":0.626}]},{"osd":5,"last update":"Mon Feb 26 04:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65300000000000002,"15min":0.63700000000000001},"min":{"1min":0.45500000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89600000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":1.008,"15min":1.008},"last":0.70799999999999996}]},{"osd":7,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.749,"15min":0.73699999999999999},"min":{"1min":0.60999999999999999,"5min":0.498,"15min":0.498},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.70899999999999996,"15min":0.68999999999999995},"min":{"1min":0.57299999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.59099999999999997}]},{"osd":8,"last update":"Mon Feb 26 04:55:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76500000000000001,"15min":0.77000000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69799999999999995}]},{"osd":9,"last update":"Mon Feb 26 04:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.78700000000000003,"15min":0.80500000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.78700000000000003,"15min":0.80300000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.66500000000000004}]},{"osd":10,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.72299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":500356,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51500000000000001,"15min":0.48899999999999999},"min":{"1min":0.36799999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.49399999999999999},"min":{"1min":0.40200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77300000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46800000000000003}]},{"osd":2,"last update":"Mon Feb 26 04:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.53500000000000003},"min":{"1min":0.436,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.56899999999999995,"15min":0.53300000000000003},"min":{"1min":0.45600000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85399999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66400000000000003}]},{"osd":3,"last update":"Mon Feb 26 04:55:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.58399999999999996,"15min":0.56100000000000005},"min":{"1min":0.435,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.439}]},{"osd":4,"last update":"Mon Feb 26 04:55:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.435,"15min":0.41499999999999998},"min":{"1min":0.28399999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.443,"15min":0.41999999999999998},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70799999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.53700000000000003}]},{"osd":5,"last update":"Mon Feb 26 04:55:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56599999999999995,"15min":0.54000000000000004},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.76500000000000001}]},{"osd":6,"last update":"Mon Feb 26 04:55:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.40000000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.5},{"interface":"front","average":{"1min":0.752,"5min":0.67900000000000005,"15min":0.65300000000000002},"min":{"1min":0.47899999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.64200000000000002}]},{"osd":7,"last update":"Mon Feb 26 04:54:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.71599999999999997}]},{"osd":8,"last update":"Mon Feb 26 04:55:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.71799999999999997,"15min":0.70899999999999996},"min":{"1min":0.57999999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.748},{"interface":"front","average":{"1min":0.751,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.55300000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.84199999999999997}]},{"osd":9,"last update":"Mon Feb 26 04:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1080000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69699999999999995}]},{"osd":10,"last update":"Mon Feb 26 04:55:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.80100000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":966656,"data_stored":477305,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 04:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52100000000000002,"15min":0.503},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68500000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81799999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53500000000000003}]},{"osd":2,"last update":"Mon Feb 26 04:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55400000000000005,"15min":0.52600000000000002},"min":{"1min":0.371,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58299999999999996,"15min":0.54800000000000004},"min":{"1min":0.41899999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.70899999999999996}]},{"osd":3,"last update":"Mon Feb 26 04:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.42399999999999999,"15min":0.40699999999999997},"min":{"1min":0.32700000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.42699999999999999,"15min":0.40300000000000002},"min":{"1min":0.29999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.32400000000000001}]},{"osd":4,"last update":"Mon Feb 26 04:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.45200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.82499999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73199999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54500000000000004}]},{"osd":5,"last update":"Mon Feb 26 04:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57699999999999996,"15min":0.55100000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84999999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61199999999999999,"15min":0.58199999999999996},"min":{"1min":0.41999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0269999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Mon Feb 26 04:55:37 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52500000000000002,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56000000000000005,"15min":0.54000000000000004},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.35399999999999998}]},{"osd":7,"last update":"Mon Feb 26 04:54:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.91200000000000003,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Feb 26 04:55:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76700000000000002,"15min":0.77300000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77700000000000002}]},{"osd":9,"last update":"Mon Feb 26 04:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.71799999999999997,"15min":0.73199999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79300000000000004,"15min":0.80500000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.5}]},{"osd":10,"last update":"Mon Feb 26 04:55:12 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.64500000000000002}]},{"osd":11,"last update":"Mon Feb 26 04:55:37 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.73899999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":482407,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.497,"15min":0.45900000000000002},"min":{"1min":0.33600000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.504,"15min":0.46600000000000003},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.70099999999999996}]},{"osd":1,"last update":"Mon Feb 26 04:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52800000000000002,"15min":0.495},"min":{"1min":0.39100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.54300000000000004,"15min":0.51700000000000002},"min":{"1min":0.37,"5min":0.318,"15min":0.318},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.60499999999999998}]},{"osd":3,"last update":"Mon Feb 26 04:55:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.443,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76200000000000001,"5min":0.87,"15min":0.87},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57999999999999996,"15min":0.56399999999999995},"min":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67000000000000004}]},{"osd":4,"last update":"Mon Feb 26 04:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.48599999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.57399999999999995,"15min":0.56000000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:55:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.46899999999999997,"15min":0.47699999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.82999999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.29299999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.70299999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42299999999999999}]},{"osd":6,"last update":"Mon Feb 26 04:55:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79100000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82199999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.748}]},{"osd":7,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.45200000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88400000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.65500000000000003,"15min":0.65400000000000003},"min":{"1min":0.51400000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.80900000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52400000000000002}]},{"osd":8,"last update":"Mon Feb 26 04:55:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.35199999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93100000000000005,"5min":1.115,"15min":1.115},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.374,"5min":0.314,"15min":0.314},"max":{"1min":0.81499999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56399999999999995}]},{"osd":9,"last update":"Mon Feb 26 04:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67000000000000004,"15min":0.66400000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.624},{"interface":"front","average":{"1min":0.75,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.64200000000000002}]},{"osd":10,"last update":"Mon Feb 26 04:55:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.76100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":477319,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:20 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.27700000000000002,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.33800000000000002,"15min":0.32900000000000001},"min":{"1min":0.25700000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.45800000000000002,"5min":0.502,"15min":0.502},"last":0.28199999999999997}]},{"osd":1,"last update":"Mon Feb 26 04:55:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.51500000000000001},"min":{"1min":0.42199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78200000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53700000000000003,"15min":0.51900000000000002},"min":{"1min":0.34499999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45500000000000002}]},{"osd":2,"last update":"Mon Feb 26 04:55:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.433,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79700000000000004,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.57999999999999996,"15min":0.55800000000000005},"min":{"1min":0.38,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80900000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.53900000000000003}]},{"osd":4,"last update":"Mon Feb 26 04:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.46500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999}]},{"osd":5,"last update":"Mon Feb 26 04:55:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.36299999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.497},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.47999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.436}]},{"osd":6,"last update":"Mon Feb 26 04:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.47499999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.74399999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.34100000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41799999999999998}]},{"osd":7,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Feb 26 04:55:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.874,"5min":0.995,"15min":0.995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.878,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63100000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:54:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.51800000000000002}]},{"osd":10,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.68899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":477859,"data_compressed":14624,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:54:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.51100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79500000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42899999999999999}]},{"osd":1,"last update":"Mon Feb 26 04:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.40500000000000003,"15min":0.36799999999999999},"min":{"1min":0.29499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.40400000000000003,"15min":0.38},"min":{"1min":0.32500000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.435}]},{"osd":2,"last update":"Mon Feb 26 04:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54100000000000004,"15min":0.51900000000000002},"min":{"1min":0.38900000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55500000000000005,"15min":0.53500000000000003},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":0.74299999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.71299999999999997}]},{"osd":3,"last update":"Mon Feb 26 04:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.59599999999999997,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.4619999999999997,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.57699999999999996,"15min":0.53700000000000003},"min":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":0.41099999999999998}]},{"osd":5,"last update":"Mon Feb 26 04:55:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80000000000000004,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.5},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.501,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81399999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.745}]},{"osd":6,"last update":"Mon Feb 26 04:55:35 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.61099999999999999,"15min":0.57899999999999996},"min":{"1min":0.46300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":4.4130000000000003,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.68400000000000005,"15min":0.65100000000000002},"min":{"1min":0.48999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.1639999999999997,"5min":4.1639999999999997,"15min":4.1639999999999997},"last":0.60199999999999998}]},{"osd":7,"last update":"Mon Feb 26 04:55:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79800000000000004,"5min":0.87,"15min":0.87},"last":0.621},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.58599999999999997,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.2300000000000004,"5min":4.2300000000000004,"15min":4.2300000000000004},"last":0.57399999999999995}]},{"osd":8,"last update":"Mon Feb 26 04:55:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68300000000000005,"15min":0.67600000000000005},"min":{"1min":0.54600000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.87,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.51400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83999999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.85699999999999998}]},{"osd":9,"last update":"Mon Feb 26 04:55:46 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":0.82399999999999995,"15min":0.747},"min":{"1min":0.55600000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":10.051,"5min":10.051,"15min":10.051},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1220000000000001,"5min":0.81000000000000005,"15min":0.75800000000000001},"min":{"1min":0.59999999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":6.3360000000000003,"5min":6.3360000000000003,"15min":6.3360000000000003},"last":0.83499999999999996}]},{"osd":10,"last update":"Mon Feb 26 04:55:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]}]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1069059,"data_compressed":50407,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48699999999999999,"15min":0.48199999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.35099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69399999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.44500000000000001}]},{"osd":1,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54300000000000004,"15min":0.52500000000000002},"min":{"1min":0.38800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84599999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59699999999999998}]},{"osd":2,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.35399999999999998},"min":{"1min":0.28299999999999997,"5min":0.22,"15min":0.22},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.375,"5min":0.38400000000000001,"15min":0.375},"min":{"1min":0.26100000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57599999999999996,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.33800000000000002}]},{"osd":3,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.33700000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50700000000000001,"15min":0.49199999999999999},"min":{"1min":0.36599999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46500000000000002}]},{"osd":4,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58899999999999997,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.376,"5min":0.372,"15min":0.372},"max":{"1min":0.752,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.56699999999999995}]},{"osd":6,"last update":"Mon Feb 26 04:55:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59399999999999997,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79500000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.61399999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:54:55 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81699999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.41299999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70799999999999996}]},{"osd":8,"last update":"Mon Feb 26 04:55:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.57299999999999995,"15min":0.58299999999999996},"min":{"1min":0.32800000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.56799999999999995,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51400000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68200000000000005,"15min":0.67700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.73099999999999998}]},{"osd":10,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.628},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":40471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":11792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":12020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T04:55:57.291 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-26T04:55:57.291 INFO:tasks.cephadm:Setting up client nodes... 2024-02-26T04:55:57.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-26T04:55:57.428 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 04:55:57 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-b[65871]: 2024-02-26T04:55:57.159+0000 7ff16f12d700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-02-26T04:55:57.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:57 smithi156 ceph-mon[35805]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:57.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:57 smithi156 ceph-mon[35805]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:57.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:57 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2707948398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:57.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:57 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1407571056' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:57.511 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:57 smithi191 ceph-mon[36352]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:57.512 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:57 smithi191 ceph-mon[36352]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:55:57.512 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:57 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2707948398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:55:57.512 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:57 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1407571056' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:55:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:55:58.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:58 smithi060 ceph-mon[32382]: pgmap v308: 129 pgs: 129 active+clean; 590 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-02-26T04:55:58.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:58 smithi060 ceph-mon[32382]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:55:58.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:55:58 smithi060 ceph-mon[32382]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:55:58.410 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:58 smithi191 ceph-mon[36352]: pgmap v308: 129 pgs: 129 active+clean; 590 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-02-26T04:55:58.411 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:58 smithi191 ceph-mon[36352]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:55:58.411 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:55:58 smithi191 ceph-mon[36352]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:55:58.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:58 smithi156 ceph-mon[35805]: pgmap v308: 129 pgs: 129 active+clean; 590 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-02-26T04:55:58.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:58 smithi156 ceph-mon[35805]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:55:58.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:55:58 smithi156 ceph-mon[35805]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:55:59.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.411+0000 7f29017dc700 1 -- 172.21.15.60:0/445234205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28fc1027b0 msgr2=0x7f28fc104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:59.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.411+0000 7f29017dc700 1 --2- 172.21.15.60:0/445234205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28fc1027b0 0x7f28fc104ba0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009990 tx=0x7f28ec009ca0 comp rx=0 tx=0).stop 2024-02-26T04:55:59.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.411+0000 7f29017dc700 1 -- 172.21.15.60:0/445234205 shutdown_connections 2024-02-26T04:55:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.411+0000 7f29017dc700 1 --2- 172.21.15.60:0/445234205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28fc107a10 0x7f28fc109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.411+0000 7f29017dc700 1 --2- 172.21.15.60:0/445234205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28fc1027b0 0x7f28fc104ba0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.411+0000 7f29017dc700 1 --2- 172.21.15.60:0/445234205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28fc1050e0 0x7f28fc1074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.411+0000 7f29017dc700 1 -- 172.21.15.60:0/445234205 >> 172.21.15.60:0/445234205 conn(0x7f28fc0fc160 msgr2=0x7f28fc0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.411+0000 7f29017dc700 1 -- 172.21.15.60:0/445234205 shutdown_connections 2024-02-26T04:55:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.411+0000 7f29017dc700 1 -- 172.21.15.60:0/445234205 wait complete. 2024-02-26T04:55:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.412+0000 7f29017dc700 1 Processor -- start 2024-02-26T04:55:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.412+0000 7f29017dc700 1 -- start start 2024-02-26T04:55:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.412+0000 7f29017dc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28fc1027b0 0x7f28fc197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.412+0000 7f29017dc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28fc1050e0 0x7f28fc198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.412+0000 7f29017dc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28fc107a10 0x7f28fc19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:59.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.412+0000 7f29017dc700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc19f670 con 0x7f28fc1027b0 2024-02-26T04:55:59.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.412+0000 7f29017dc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc19f7b0 con 0x7f28fc107a10 2024-02-26T04:55:59.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.412+0000 7f29017dc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc19f940 con 0x7f28fc1050e0 2024-02-26T04:55:59.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.412+0000 7f28faffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28fc1027b0 0x7f28fc197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:59.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.412+0000 7f28faffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28fc1027b0 0x7f28fc197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51174/0 (socket says 172.21.15.60:51174) 2024-02-26T04:55:59.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.412+0000 7f28fa7fc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28fc1050e0 0x7f28fc198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:59.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.412+0000 7f28faffd700 1 -- 172.21.15.60:0/1579296497 learned_addr learned my addr 172.21.15.60:0/1579296497 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:55:59.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.412+0000 7f28fb7fe700 1 --2- 172.21.15.60:0/1579296497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28fc107a10 0x7f28fc19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.413+0000 7f28faffd700 1 -- 172.21.15.60:0/1579296497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28fc1050e0 msgr2=0x7f28fc198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.413+0000 7f28faffd700 1 --2- 172.21.15.60:0/1579296497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28fc1050e0 0x7f28fc198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.413+0000 7f28faffd700 1 -- 172.21.15.60:0/1579296497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28fc107a10 msgr2=0x7f28fc19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.413+0000 7f28faffd700 1 --2- 172.21.15.60:0/1579296497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28fc107a10 0x7f28fc19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.413+0000 7f28faffd700 1 -- 172.21.15.60:0/1579296497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec0095f0 con 0x7f28fc1027b0 2024-02-26T04:55:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.413+0000 7f28fa7fc700 1 --2- 172.21.15.60:0/1579296497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28fc1050e0 0x7f28fc198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:55:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.413+0000 7f28faffd700 1 --2- 172.21.15.60:0/1579296497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28fc1027b0 0x7f28fc197ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f28ec000c00 tx=0x7f28ec00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.413+0000 7f28e3fff700 1 -- 172.21.15.60:0/1579296497 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec01d070 con 0x7f28fc1027b0 2024-02-26T04:55:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.413+0000 7f29017dc700 1 -- 172.21.15.60:0/1579296497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28fc19fbc0 con 0x7f28fc1027b0 2024-02-26T04:55:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.413+0000 7f28e3fff700 1 -- 172.21.15.60:0/1579296497 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28ec022470 con 0x7f28fc1027b0 2024-02-26T04:55:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.413+0000 7f28e3fff700 1 -- 172.21.15.60:0/1579296497 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec017650 con 0x7f28fc1027b0 2024-02-26T04:55:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.413+0000 7f29017dc700 1 -- 172.21.15.60:0/1579296497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28fc1a0030 con 0x7f28fc1027b0 2024-02-26T04:55:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.414+0000 7f29017dc700 1 -- 172.21.15.60:0/1579296497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28fc0fdce0 con 0x7f28fc1027b0 2024-02-26T04:55:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.415+0000 7f28e3fff700 1 -- 172.21.15.60:0/1579296497 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f28ec0225e0 con 0x7f28fc1027b0 2024-02-26T04:55:59.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.416+0000 7f28e3fff700 1 --2- 172.21.15.60:0/1579296497 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f28e40a2310 0x7f28e40a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:55:59.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.416+0000 7f28fa7fc700 1 --2- 172.21.15.60:0/1579296497 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f28e40a2310 0x7f28e40a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:55:59.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.416+0000 7f28e3fff700 1 -- 172.21.15.60:0/1579296497 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f28ec0cf2b0 con 0x7f28fc1027b0 2024-02-26T04:55:59.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.416+0000 7f28fa7fc700 1 --2- 172.21.15.60:0/1579296497 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f28e40a2310 0x7f28e40a47d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f28f000afd0 tx=0x7f28f000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:55:59.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.420+0000 7f28e3fff700 1 -- 172.21.15.60:0/1579296497 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f28ec098030 con 0x7f28fc1027b0 2024-02-26T04:55:59.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.705+0000 7f29017dc700 1 -- 172.21.15.60:0/1579296497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f28fc050180 con 0x7f28fc1027b0 2024-02-26T04:55:59.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.707+0000 7f28e3fff700 1 -- 172.21.15.60:0/1579296497 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f28ec09b6f0 con 0x7f28fc1027b0 2024-02-26T04:55:59.709 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-02-26T04:55:59.709 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQDfGdxl1/8bKhAAnRB1rbAXbVPliz7/xvn7Jw== 2024-02-26T04:55:59.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.709+0000 7f29017dc700 1 -- 172.21.15.60:0/1579296497 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f28e40a2310 msgr2=0x7f28e40a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:59.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.709+0000 7f29017dc700 1 --2- 172.21.15.60:0/1579296497 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f28e40a2310 0x7f28e40a47d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f28f000afd0 tx=0x7f28f000a380 comp rx=0 tx=0).stop 2024-02-26T04:55:59.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.709+0000 7f29017dc700 1 -- 172.21.15.60:0/1579296497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28fc1027b0 msgr2=0x7f28fc197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:55:59.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.709+0000 7f29017dc700 1 --2- 172.21.15.60:0/1579296497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28fc1027b0 0x7f28fc197ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f28ec000c00 tx=0x7f28ec00f690 comp rx=0 tx=0).stop 2024-02-26T04:55:59.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.709+0000 7f29017dc700 1 -- 172.21.15.60:0/1579296497 shutdown_connections 2024-02-26T04:55:59.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.709+0000 7f29017dc700 1 --2- 172.21.15.60:0/1579296497 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f28e40a2310 0x7f28e40a47d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:59.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.709+0000 7f29017dc700 1 --2- 172.21.15.60:0/1579296497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28fc107a10 0x7f28fc19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:59.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.709+0000 7f29017dc700 1 --2- 172.21.15.60:0/1579296497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28fc1027b0 0x7f28fc197ae0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:59.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.709+0000 7f29017dc700 1 --2- 172.21.15.60:0/1579296497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28fc1050e0 0x7f28fc198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:55:59.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.709+0000 7f29017dc700 1 -- 172.21.15.60:0/1579296497 >> 172.21.15.60:0/1579296497 conn(0x7f28fc0fc160 msgr2=0x7f28fc1030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:55:59.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.709+0000 7f29017dc700 1 -- 172.21.15.60:0/1579296497 shutdown_connections 2024-02-26T04:55:59.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:55:59.709+0000 7f29017dc700 1 -- 172.21.15.60:0/1579296497 wait complete. 2024-02-26T04:56:00.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:00 smithi156 ceph-mon[35805]: pgmap v309: 129 pgs: 129 active+clean; 590 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-02-26T04:56:00.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:00 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1579296497' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T04:56:00.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:00 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1579296497' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-26T04:56:00.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:00 smithi060 ceph-mon[32382]: pgmap v309: 129 pgs: 129 active+clean; 590 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-02-26T04:56:00.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:00 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1579296497' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T04:56:00.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:00 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1579296497' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-26T04:56:00.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:00 smithi191 ceph-mon[36352]: pgmap v309: 129 pgs: 129 active+clean; 590 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-02-26T04:56:00.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:00 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1579296497' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-26T04:56:00.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:00 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1579296497' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-26T04:56:00.544 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:56:00.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-26T04:56:00.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-26T04:56:00.591 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-26T04:56:00.591 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-26T04:56:00.592 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph mgr dump --format=json 2024-02-26T04:56:00.873 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:02.358 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:02 smithi060 ceph-mon[32382]: pgmap v310: 129 pgs: 129 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-02-26T04:56:02.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:02 smithi156 ceph-mon[35805]: pgmap v310: 129 pgs: 129 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-02-26T04:56:02.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:02 smithi191 ceph-mon[36352]: pgmap v310: 129 pgs: 129 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-02-26T04:56:02.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.579+0000 7f975228d700 1 -- 172.21.15.60:0/1051731097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974c106b80 msgr2=0x7f974c106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:02.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.579+0000 7f975228d700 1 --2- 172.21.15.60:0/1051731097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974c106b80 0x7f974c106f60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f973c009990 tx=0x7f973c009ca0 comp rx=0 tx=0).stop 2024-02-26T04:56:02.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.579+0000 7f975228d700 1 -- 172.21.15.60:0/1051731097 shutdown_connections 2024-02-26T04:56:02.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.579+0000 7f975228d700 1 --2- 172.21.15.60:0/1051731097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f974c101530 0x7f974c10d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:02.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.579+0000 7f975228d700 1 --2- 172.21.15.60:0/1051731097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974c106b80 0x7f974c106f60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:02.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.579+0000 7f975228d700 1 --2- 172.21.15.60:0/1051731097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974c100b70 0x7f974c100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:02.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.579+0000 7f975228d700 1 -- 172.21.15.60:0/1051731097 >> 172.21.15.60:0/1051731097 conn(0x7f974c077aa0 msgr2=0x7f974c077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:02.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.579+0000 7f975228d700 1 -- 172.21.15.60:0/1051731097 shutdown_connections 2024-02-26T04:56:02.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.579+0000 7f975228d700 1 -- 172.21.15.60:0/1051731097 wait complete. 2024-02-26T04:56:02.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f975228d700 1 Processor -- start 2024-02-26T04:56:02.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f975228d700 1 -- start start 2024-02-26T04:56:02.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f975228d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974c100b70 0x7f974c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:02.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f975228d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974c101530 0x7f974c1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:02.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f975228d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f974c106b80 0x7f974c1a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:02.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f975228d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c199cd0 con 0x7f974c101530 2024-02-26T04:56:02.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f975228d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c1a4450 con 0x7f974c106b80 2024-02-26T04:56:02.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f975228d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c1a45c0 con 0x7f974c100b70 2024-02-26T04:56:02.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f974b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974c101530 0x7f974c1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:02.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f974b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974c101530 0x7f974c1993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38794/0 (socket says 172.21.15.60:38794) 2024-02-26T04:56:02.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f974b7fe700 1 -- 172.21.15.60:0/4080240621 learned_addr learned my addr 172.21.15.60:0/4080240621 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:02.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f974bfff700 1 --2- 172.21.15.60:0/4080240621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974c100b70 0x7f974c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:02.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f975082a700 1 --2- 172.21.15.60:0/4080240621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f974c106b80 0x7f974c1a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:02.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f974b7fe700 1 -- 172.21.15.60:0/4080240621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974c100b70 msgr2=0x7f974c198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:02.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.580+0000 7f974b7fe700 1 --2- 172.21.15.60:0/4080240621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974c100b70 0x7f974c198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:02.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.581+0000 7f974b7fe700 1 -- 172.21.15.60:0/4080240621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f974c106b80 msgr2=0x7f974c1a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:02.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.581+0000 7f974b7fe700 1 --2- 172.21.15.60:0/4080240621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f974c106b80 0x7f974c1a3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:02.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.581+0000 7f974b7fe700 1 -- 172.21.15.60:0/4080240621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c0095f0 con 0x7f974c101530 2024-02-26T04:56:02.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.581+0000 7f974b7fe700 1 --2- 172.21.15.60:0/4080240621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974c101530 0x7f974c1993d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f974000c920 tx=0x7f974000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:02.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.581+0000 7f97497fa700 1 -- 172.21.15.60:0/4080240621 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740007aa0 con 0x7f974c101530 2024-02-26T04:56:02.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.581+0000 7f97497fa700 1 -- 172.21.15.60:0/4080240621 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f974000ce70 con 0x7f974c101530 2024-02-26T04:56:02.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.581+0000 7f975228d700 1 -- 172.21.15.60:0/4080240621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c1a48a0 con 0x7f974c101530 2024-02-26T04:56:02.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.581+0000 7f97497fa700 1 -- 172.21.15.60:0/4080240621 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740005550 con 0x7f974c101530 2024-02-26T04:56:02.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.581+0000 7f975228d700 1 -- 172.21.15.60:0/4080240621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c1a4da0 con 0x7f974c101530 2024-02-26T04:56:02.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.582+0000 7f975228d700 1 -- 172.21.15.60:0/4080240621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c109020 con 0x7f974c101530 2024-02-26T04:56:02.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.583+0000 7f97497fa700 1 -- 172.21.15.60:0/4080240621 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f9740004750 con 0x7f974c101530 2024-02-26T04:56:02.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.584+0000 7f97497fa700 1 --2- 172.21.15.60:0/4080240621 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f97340a21e0 0x7f97340a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:02.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.584+0000 7f974bfff700 1 --2- 172.21.15.60:0/4080240621 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f97340a21e0 0x7f97340a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:02.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.584+0000 7f97497fa700 1 -- 172.21.15.60:0/4080240621 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f97400ce070 con 0x7f974c101530 2024-02-26T04:56:02.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.584+0000 7f974bfff700 1 --2- 172.21.15.60:0/4080240621 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f97340a21e0 0x7f97340a46a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f973c0098e0 tx=0x7f973c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:02.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.588+0000 7f97497fa700 1 -- 172.21.15.60:0/4080240621 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9740096c40 con 0x7f974c101530 2024-02-26T04:56:02.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.847+0000 7f975228d700 1 -- 172.21.15.60:0/4080240621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f974c068930 con 0x7f974c101530 2024-02-26T04:56:02.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.852+0000 7f97497fa700 1 -- 172.21.15.60:0/4080240621 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+261484 (secure 0 0 0) 0x7f974001d020 con 0x7f974c101530 2024-02-26T04:56:02.858 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:56:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.855+0000 7f975228d700 1 -- 172.21.15.60:0/4080240621 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f97340a21e0 msgr2=0x7f97340a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.855+0000 7f975228d700 1 --2- 172.21.15.60:0/4080240621 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f97340a21e0 0x7f97340a46a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f973c0098e0 tx=0x7f973c005960 comp rx=0 tx=0).stop 2024-02-26T04:56:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.855+0000 7f975228d700 1 -- 172.21.15.60:0/4080240621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974c101530 msgr2=0x7f974c1993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.855+0000 7f975228d700 1 --2- 172.21.15.60:0/4080240621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974c101530 0x7f974c1993d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f974000c920 tx=0x7f974000cc30 comp rx=0 tx=0).stop 2024-02-26T04:56:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.855+0000 7f975228d700 1 -- 172.21.15.60:0/4080240621 shutdown_connections 2024-02-26T04:56:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.855+0000 7f975228d700 1 --2- 172.21.15.60:0/4080240621 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f97340a21e0 0x7f97340a46a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.855+0000 7f975228d700 1 --2- 172.21.15.60:0/4080240621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f974c106b80 0x7f974c1a3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.855+0000 7f975228d700 1 --2- 172.21.15.60:0/4080240621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974c100b70 0x7f974c198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.855+0000 7f975228d700 1 --2- 172.21.15.60:0/4080240621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974c101530 0x7f974c1993d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.855+0000 7f975228d700 1 -- 172.21.15.60:0/4080240621 >> 172.21.15.60:0/4080240621 conn(0x7f974c077aa0 msgr2=0x7f974c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.855+0000 7f975228d700 1 -- 172.21.15.60:0/4080240621 shutdown_connections 2024-02-26T04:56:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:02.855+0000 7f975228d700 1 -- 172.21.15.60:0/4080240621 wait complete. 2024-02-26T04:56:03.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:03 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4080240621' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-26T04:56:03.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:03 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4080240621' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-26T04:56:03.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:03 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4080240621' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-26T04:56:03.674 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3458567419},{"type":"v1","addr":"172.21.15.60:6801","nonce":3458567419}]},"active_addr":"172.21.15.60:6801/3458567419","active_change":"2024-02-26T04:48:18.224018+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.60:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":612643565}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":2990090132}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":3636752194}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":4212036338}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":2757574827}]}]} 2024-02-26T04:56:03.678 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-26T04:56:03.678 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-26T04:56:03.679 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-26T04:56:03.926 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:04.330 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:04 smithi060 ceph-mon[32382]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-02-26T04:56:04.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:04 smithi156 ceph-mon[35805]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-02-26T04:56:04.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:04 smithi191 ceph-mon[36352]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-02-26T04:56:05.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.817+0000 7f6df674e700 1 -- 172.21.15.60:0/1760584787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6df0100e30 msgr2=0x7f6df0101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.817+0000 7f6df674e700 1 --2- 172.21.15.60:0/1760584787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6df0100e30 0x7f6df0101210 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009990 tx=0x7f6de0009ca0 comp rx=0 tx=0).stop 2024-02-26T04:56:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.817+0000 7f6df674e700 1 -- 172.21.15.60:0/1760584787 shutdown_connections 2024-02-26T04:56:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.817+0000 7f6df674e700 1 --2- 172.21.15.60:0/1760584787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6df0105ec0 0x7f6df010da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.817+0000 7f6df674e700 1 --2- 172.21.15.60:0/1760584787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6df0100e30 0x7f6df0101210 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.817+0000 7f6df674e700 1 --2- 172.21.15.60:0/1760584787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df01017e0 0x7f6df01057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.817+0000 7f6df674e700 1 -- 172.21.15.60:0/1760584787 >> 172.21.15.60:0/1760584787 conn(0x7f6df0076bc0 msgr2=0x7f6df0076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.817+0000 7f6df674e700 1 -- 172.21.15.60:0/1760584787 shutdown_connections 2024-02-26T04:56:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.817+0000 7f6df674e700 1 -- 172.21.15.60:0/1760584787 wait complete. 2024-02-26T04:56:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.817+0000 7f6df674e700 1 Processor -- start 2024-02-26T04:56:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.817+0000 7f6df674e700 1 -- start start 2024-02-26T04:56:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6df674e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6df0100e30 0x7f6df01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6df674e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6df01017e0 0x7f6df0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6df674e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df0105ec0 0x7f6df019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6df674e700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df019d0a0 con 0x7f6df01017e0 2024-02-26T04:56:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6df674e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df019d210 con 0x7f6df0105ec0 2024-02-26T04:56:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6df674e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df019d3d0 con 0x7f6df0100e30 2024-02-26T04:56:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6def7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6df01017e0 0x7f6df0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6def7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6df01017e0 0x7f6df0198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38814/0 (socket says 172.21.15.60:38814) 2024-02-26T04:56:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6def7fe700 1 -- 172.21.15.60:0/1930006086 learned_addr learned my addr 172.21.15.60:0/1930006086 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6deffff700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6df0100e30 0x7f6df01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6df4ceb700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df0105ec0 0x7f6df019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6def7fe700 1 -- 172.21.15.60:0/1930006086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6df0100e30 msgr2=0x7f6df01981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6def7fe700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6df0100e30 0x7f6df01981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6def7fe700 1 -- 172.21.15.60:0/1930006086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df0105ec0 msgr2=0x7f6df019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6def7fe700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df0105ec0 0x7f6df019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6def7fe700 1 -- 172.21.15.60:0/1930006086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de00095f0 con 0x7f6df01017e0 2024-02-26T04:56:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6df4ceb700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df0105ec0 0x7f6df019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:56:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6def7fe700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6df01017e0 0x7f6df0198700 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6de400d980 tx=0x7f6de400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6ded7fa700 1 -- 172.21.15.60:0/1930006086 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4009930 con 0x7f6df01017e0 2024-02-26T04:56:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6df674e700 1 -- 172.21.15.60:0/1930006086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df019d6b0 con 0x7f6df01017e0 2024-02-26T04:56:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6ded7fa700 1 -- 172.21.15.60:0/1930006086 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6de4010460 con 0x7f6df01017e0 2024-02-26T04:56:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.818+0000 7f6ded7fa700 1 -- 172.21.15.60:0/1930006086 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de400f620 con 0x7f6df01017e0 2024-02-26T04:56:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.819+0000 7f6df674e700 1 -- 172.21.15.60:0/1930006086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df01a4f10 con 0x7f6df01017e0 2024-02-26T04:56:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.819+0000 7f6df674e700 1 -- 172.21.15.60:0/1930006086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df010a160 con 0x7f6df01017e0 2024-02-26T04:56:05.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.821+0000 7f6ded7fa700 1 -- 172.21.15.60:0/1930006086 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f6de40105d0 con 0x7f6df01017e0 2024-02-26T04:56:05.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.821+0000 7f6ded7fa700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6dd80a21e0 0x7f6dd80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:05.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.821+0000 7f6deffff700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6dd80a21e0 0x7f6dd80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:05.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.821+0000 7f6ded7fa700 1 -- 172.21.15.60:0/1930006086 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f6de40cdb90 con 0x7f6df01017e0 2024-02-26T04:56:05.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.822+0000 7f6deffff700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6dd80a21e0 0x7f6dd80a46a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6de000b5c0 tx=0x7f6de0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:05.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:05.825+0000 7f6ded7fa700 1 -- 172.21.15.60:0/1930006086 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6de409a2f0 con 0x7f6df01017e0 2024-02-26T04:56:06.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:06.027+0000 7f6df674e700 1 -- 172.21.15.60:0/1930006086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6df0199550 con 0x7f6df01017e0 2024-02-26T04:56:06.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:06.028+0000 7f6ded7fa700 1 -- 172.21.15.60:0/1930006086 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24126 (secure 0 0 0) 0x7f6de4020090 con 0x7f6df01017e0 2024-02-26T04:56:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:56:06.031 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:55:32.581481+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:56:06.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:06.030+0000 7f6df674e700 1 -- 172.21.15.60:0/1930006086 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6dd80a21e0 msgr2=0x7f6dd80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:06.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:06.030+0000 7f6df674e700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6dd80a21e0 0x7f6dd80a46a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6de000b5c0 tx=0x7f6de0005960 comp rx=0 tx=0).stop 2024-02-26T04:56:06.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:06.031+0000 7f6df674e700 1 -- 172.21.15.60:0/1930006086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6df01017e0 msgr2=0x7f6df0198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:06.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:06.031+0000 7f6df674e700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6df01017e0 0x7f6df0198700 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6de400d980 tx=0x7f6de400dc90 comp rx=0 tx=0).stop 2024-02-26T04:56:06.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:06.031+0000 7f6df674e700 1 -- 172.21.15.60:0/1930006086 shutdown_connections 2024-02-26T04:56:06.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:06.031+0000 7f6df674e700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6dd80a21e0 0x7f6dd80a46a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:06.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:06.031+0000 7f6df674e700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df0105ec0 0x7f6df019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:06.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:06.031+0000 7f6df674e700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6df0100e30 0x7f6df01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:06.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:06.031+0000 7f6df674e700 1 --2- 172.21.15.60:0/1930006086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6df01017e0 0x7f6df0198700 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:06.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:06.031+0000 7f6df674e700 1 -- 172.21.15.60:0/1930006086 >> 172.21.15.60:0/1930006086 conn(0x7f6df0076bc0 msgr2=0x7f6df006a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:06.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:06.031+0000 7f6df674e700 1 -- 172.21.15.60:0/1930006086 shutdown_connections 2024-02-26T04:56:06.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:06.031+0000 7f6df674e700 1 -- 172.21.15.60:0/1930006086 wait complete. 2024-02-26T04:56:06.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:06 smithi156 ceph-mon[35805]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-02-26T04:56:06.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:06 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1930006086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:56:06.469 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:06 smithi060 ceph-mon[32382]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-02-26T04:56:06.469 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:06 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1930006086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:56:06.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:06 smithi191 ceph-mon[36352]: pgmap v312: 129 pgs: 129 active+clean; 625 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-02-26T04:56:06.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:06 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1930006086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:56:06.810 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-26T04:56:06.810 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd dump --format=json 2024-02-26T04:56:07.058 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:08.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:08 smithi156 ceph-mon[35805]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-02-26T04:56:08.460 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:08 smithi060 ceph-mon[32382]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-02-26T04:56:08.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:08 smithi191 ceph-mon[36352]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-02-26T04:56:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.850+0000 7fe4adf96700 1 -- 172.21.15.60:0/581217798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a8100c20 msgr2=0x7fe4a810eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.850+0000 7fe4adf96700 1 --2- 172.21.15.60:0/581217798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a8100c20 0x7fe4a810eb80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000b0a0 tx=0x7fe4a000b3b0 comp rx=0 tx=0).stop 2024-02-26T04:56:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.850+0000 7fe4adf96700 1 -- 172.21.15.60:0/581217798 shutdown_connections 2024-02-26T04:56:08.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.850+0000 7fe4adf96700 1 --2- 172.21.15.60:0/581217798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a8100c20 0x7fe4a810eb80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:08.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.850+0000 7fe4adf96700 1 --2- 172.21.15.60:0/581217798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4a8106ce0 0x7fe4a81070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:08.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.850+0000 7fe4adf96700 1 --2- 172.21.15.60:0/581217798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4a8100260 0x7fe4a81006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:08.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.850+0000 7fe4adf96700 1 -- 172.21.15.60:0/581217798 >> 172.21.15.60:0/581217798 conn(0x7fe4a8077170 msgr2=0x7fe4a80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:08.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.850+0000 7fe4adf96700 1 -- 172.21.15.60:0/581217798 shutdown_connections 2024-02-26T04:56:08.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.850+0000 7fe4adf96700 1 -- 172.21.15.60:0/581217798 wait complete. 2024-02-26T04:56:08.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.850+0000 7fe4adf96700 1 Processor -- start 2024-02-26T04:56:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4adf96700 1 -- start start 2024-02-26T04:56:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4adf96700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4a8100260 0x7fe4a81982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4adf96700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a8100c20 0x7fe4a8198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4adf96700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4a8106ce0 0x7fe4a819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4adf96700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a819d160 con 0x7fe4a8100c20 2024-02-26T04:56:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4adf96700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a819d2d0 con 0x7fe4a8100260 2024-02-26T04:56:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4adf96700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a819d490 con 0x7fe4a8106ce0 2024-02-26T04:56:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4a6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a8100c20 0x7fe4a8198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4a6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a8100c20 0x7fe4a8198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38842/0 (socket says 172.21.15.60:38842) 2024-02-26T04:56:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4a6ffd700 1 -- 172.21.15.60:0/2125248195 learned_addr learned my addr 172.21.15.60:0/2125248195 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4a7fff700 1 --2- 172.21.15.60:0/2125248195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4a8106ce0 0x7fe4a819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4a77fe700 1 --2- 172.21.15.60:0/2125248195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4a8100260 0x7fe4a81982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4a6ffd700 1 -- 172.21.15.60:0/2125248195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4a8106ce0 msgr2=0x7fe4a819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4a6ffd700 1 --2- 172.21.15.60:0/2125248195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4a8106ce0 0x7fe4a819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4a6ffd700 1 -- 172.21.15.60:0/2125248195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4a8100260 msgr2=0x7fe4a81982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4a6ffd700 1 --2- 172.21.15.60:0/2125248195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4a8100260 0x7fe4a81982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.851+0000 7fe4a6ffd700 1 -- 172.21.15.60:0/2125248195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a0009d00 con 0x7fe4a8100c20 2024-02-26T04:56:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.852+0000 7fe4a6ffd700 1 --2- 172.21.15.60:0/2125248195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a8100c20 0x7fe4a8198810 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00c9b0 tx=0x7fe49c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.852+0000 7fe4a4ff9700 1 -- 172.21.15.60:0/2125248195 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c007b00 con 0x7fe4a8100c20 2024-02-26T04:56:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.852+0000 7fe4a4ff9700 1 -- 172.21.15.60:0/2125248195 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe49c007c60 con 0x7fe4a8100c20 2024-02-26T04:56:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.852+0000 7fe4adf96700 1 -- 172.21.15.60:0/2125248195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a819d770 con 0x7fe4a8100c20 2024-02-26T04:56:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.852+0000 7fe4a4ff9700 1 -- 172.21.15.60:0/2125248195 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c018660 con 0x7fe4a8100c20 2024-02-26T04:56:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.852+0000 7fe4adf96700 1 -- 172.21.15.60:0/2125248195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a81a4fd0 con 0x7fe4a8100c20 2024-02-26T04:56:08.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.853+0000 7fe4adf96700 1 -- 172.21.15.60:0/2125248195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a8103580 con 0x7fe4a8100c20 2024-02-26T04:56:08.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.854+0000 7fe4a4ff9700 1 -- 172.21.15.60:0/2125248195 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe49c00f450 con 0x7fe4a8100c20 2024-02-26T04:56:08.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.855+0000 7fe4a4ff9700 1 --2- 172.21.15.60:0/2125248195 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe4900a21e0 0x7fe4900a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:08.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.855+0000 7fe4a77fe700 1 --2- 172.21.15.60:0/2125248195 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe4900a21e0 0x7fe4900a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:08.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.855+0000 7fe4a4ff9700 1 -- 172.21.15.60:0/2125248195 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe49c0ce0b0 con 0x7fe4a8100c20 2024-02-26T04:56:08.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.855+0000 7fe4a77fe700 1 --2- 172.21.15.60:0/2125248195 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe4900a21e0 0x7fe4900a46a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe498005ea0 tx=0x7fe498005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:08.859+0000 7fe4a4ff9700 1 -- 172.21.15.60:0/2125248195 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe49c096c80 con 0x7fe4a8100c20 2024-02-26T04:56:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:09.061+0000 7fe4adf96700 1 -- 172.21.15.60:0/2125248195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4a8068930 con 0x7fe4a8100c20 2024-02-26T04:56:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:09.063+0000 7fe4a4ff9700 1 -- 172.21.15.60:0/2125248195 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24126 (secure 0 0 0) 0x7fe49c09a340 con 0x7fe4a8100c20 2024-02-26T04:56:09.066 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:56:09.066 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:55:32.581481+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:56:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:09.065+0000 7fe4adf96700 1 -- 172.21.15.60:0/2125248195 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe4900a21e0 msgr2=0x7fe4900a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:09.065+0000 7fe4adf96700 1 --2- 172.21.15.60:0/2125248195 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe4900a21e0 0x7fe4900a46a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe498005ea0 tx=0x7fe498005e30 comp rx=0 tx=0).stop 2024-02-26T04:56:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:09.065+0000 7fe4adf96700 1 -- 172.21.15.60:0/2125248195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a8100c20 msgr2=0x7fe4a8198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:09.065+0000 7fe4adf96700 1 --2- 172.21.15.60:0/2125248195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a8100c20 0x7fe4a8198810 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00c9b0 tx=0x7fe49c00ccc0 comp rx=0 tx=0).stop 2024-02-26T04:56:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:09.065+0000 7fe4adf96700 1 -- 172.21.15.60:0/2125248195 shutdown_connections 2024-02-26T04:56:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:09.065+0000 7fe4adf96700 1 --2- 172.21.15.60:0/2125248195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4a8106ce0 0x7fe4a819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:09.065+0000 7fe4adf96700 1 --2- 172.21.15.60:0/2125248195 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe4900a21e0 0x7fe4900a46a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:09.065+0000 7fe4adf96700 1 --2- 172.21.15.60:0/2125248195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4a8100260 0x7fe4a81982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:09.065+0000 7fe4adf96700 1 --2- 172.21.15.60:0/2125248195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a8100c20 0x7fe4a8198810 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:09.065+0000 7fe4adf96700 1 -- 172.21.15.60:0/2125248195 >> 172.21.15.60:0/2125248195 conn(0x7fe4a8077170 msgr2=0x7fe4a8104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:09.065+0000 7fe4adf96700 1 -- 172.21.15.60:0/2125248195 shutdown_connections 2024-02-26T04:56:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:09.065+0000 7fe4adf96700 1 -- 172.21.15.60:0/2125248195 wait complete. 2024-02-26T04:56:09.420 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:09 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2125248195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:56:09.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:09 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2125248195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:56:09.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:09 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2125248195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:56:09.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-26T04:56:09.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-26T04:56:09.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-26T04:56:09.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-26T04:56:09.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-26T04:56:09.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-26T04:56:09.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-26T04:56:09.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-26T04:56:09.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-02-26T04:56:09.881 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-02-26T04:56:09.881 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-02-26T04:56:09.882 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-02-26T04:56:10.181 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:10 smithi060 ceph-mon[32382]: pgmap v314: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-02-26T04:56:10.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:10 smithi156 ceph-mon[35805]: pgmap v314: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-02-26T04:56:10.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:10 smithi191 ceph-mon[36352]: pgmap v314: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-02-26T04:56:10.570 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:10.571 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:10.582 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:10.603 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:11.332 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:12.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:12 smithi060 ceph-mon[32382]: pgmap v315: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-02-26T04:56:12.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:12 smithi191 ceph-mon[36352]: pgmap v315: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-02-26T04:56:12.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:12 smithi156 ceph-mon[35805]: pgmap v315: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-02-26T04:56:13.859 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:14.168 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:14.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:14 smithi060 ceph-mon[32382]: pgmap v316: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-26T04:56:14.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:14 smithi191 ceph-mon[36352]: pgmap v316: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-26T04:56:14.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:14 smithi156 ceph-mon[35805]: pgmap v316: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-26T04:56:16.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.003+0000 7f49f908b700 1 -- 172.21.15.60:0/1158547682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49f4100e50 msgr2=0x7f49f41012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.003+0000 7f49f908b700 1 --2- 172.21.15.60:0/1158547682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49f4100e50 0x7f49f41012d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009990 tx=0x7f49e8009ca0 comp rx=0 tx=0).stop 2024-02-26T04:56:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.003+0000 7f49f908b700 1 -- 172.21.15.60:0/1158547682 shutdown_connections 2024-02-26T04:56:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.003+0000 7f49f908b700 1 --2- 172.21.15.60:0/1158547682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49f4101810 0x7f49f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.003+0000 7f49f908b700 1 --2- 172.21.15.60:0/1158547682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49f4106d60 0x7f49f4107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.003+0000 7f49f908b700 1 --2- 172.21.15.60:0/1158547682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49f4100e50 0x7f49f41012d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.003+0000 7f49f908b700 1 -- 172.21.15.60:0/1158547682 >> 172.21.15.60:0/1158547682 conn(0x7f49f40779d0 msgr2=0x7f49f4077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.003+0000 7f49f908b700 1 -- 172.21.15.60:0/1158547682 shutdown_connections 2024-02-26T04:56:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.003+0000 7f49f908b700 1 -- 172.21.15.60:0/1158547682 wait complete. 2024-02-26T04:56:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.004+0000 7f49f908b700 1 Processor -- start 2024-02-26T04:56:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.004+0000 7f49f908b700 1 -- start start 2024-02-26T04:56:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f908b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49f4100e50 0x7f49f4198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f908b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49f4101810 0x7f49f4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f908b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49f4106d60 0x7f49f419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f908b700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f419d0e0 con 0x7f49f4106d60 2024-02-26T04:56:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f908b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f419d250 con 0x7f49f4101810 2024-02-26T04:56:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f908b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f419d410 con 0x7f49f4100e50 2024-02-26T04:56:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f359e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49f4106d60 0x7f49f419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f359e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49f4106d60 0x7f49f419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49714/0 (socket says 172.21.15.60:49714) 2024-02-26T04:56:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f359e700 1 -- 172.21.15.60:0/3201191611 learned_addr learned my addr 172.21.15.60:0/3201191611 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f259c700 1 --2- 172.21.15.60:0/3201191611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49f4101810 0x7f49f4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f2d9d700 1 --2- 172.21.15.60:0/3201191611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49f4100e50 0x7f49f4198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f359e700 1 -- 172.21.15.60:0/3201191611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49f4100e50 msgr2=0x7f49f4198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f359e700 1 --2- 172.21.15.60:0/3201191611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49f4100e50 0x7f49f4198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f359e700 1 -- 172.21.15.60:0/3201191611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49f4101810 msgr2=0x7f49f4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:16.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f359e700 1 --2- 172.21.15.60:0/3201191611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49f4101810 0x7f49f4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:16.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f359e700 1 -- 172.21.15.60:0/3201191611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e80095f0 con 0x7f49f4106d60 2024-02-26T04:56:16.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f359e700 1 --2- 172.21.15.60:0/3201191611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49f4106d60 0x7f49f419cad0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f49ec007f10 tx=0x7f49ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:16.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49dbfff700 1 -- 172.21.15.60:0/3201191611 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec017070 con 0x7f49f4106d60 2024-02-26T04:56:16.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f908b700 1 -- 172.21.15.60:0/3201191611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f419d6f0 con 0x7f49f4106d60 2024-02-26T04:56:16.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49dbfff700 1 -- 172.21.15.60:0/3201191611 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49ec00f040 con 0x7f49f4106d60 2024-02-26T04:56:16.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49f908b700 1 -- 172.21.15.60:0/3201191611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f41a4f50 con 0x7f49f4106d60 2024-02-26T04:56:16.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.005+0000 7f49dbfff700 1 -- 172.21.15.60:0/3201191611 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec005760 con 0x7f49f4106d60 2024-02-26T04:56:16.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.006+0000 7f49f908b700 1 -- 172.21.15.60:0/3201191611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49f4110ef0 con 0x7f49f4106d60 2024-02-26T04:56:16.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.008+0000 7f49dbfff700 1 -- 172.21.15.60:0/3201191611 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f49ec0058c0 con 0x7f49f4106d60 2024-02-26T04:56:16.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.008+0000 7f49dbfff700 1 --2- 172.21.15.60:0/3201191611 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49dc0a21e0 0x7f49dc0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:16.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.008+0000 7f49dbfff700 1 -- 172.21.15.60:0/3201191611 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f49ec0cdee0 con 0x7f49f4106d60 2024-02-26T04:56:16.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.008+0000 7f49f2d9d700 1 --2- 172.21.15.60:0/3201191611 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49dc0a21e0 0x7f49dc0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:16.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.008+0000 7f49dbfff700 1 --2- 172.21.15.60:0/3201191611 >> [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016] conn(0x7f49dc0acea0 0x7f49dc0af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:16.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.008+0000 7f49dbfff700 1 -- 172.21.15.60:0/3201191611 --> [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49dc0af9b0 con 0x7f49dc0acea0 2024-02-26T04:56:16.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.008+0000 7f49dbfff700 1 -- 172.21.15.60:0/3201191611 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49ec0ce2f0 con 0x7f49f4106d60 2024-02-26T04:56:16.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.008+0000 7f49f259c700 1 --2- 172.21.15.60:0/3201191611 >> [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016] conn(0x7f49dc0acea0 0x7f49dc0af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:16.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.009+0000 7f49f2d9d700 1 --2- 172.21.15.60:0/3201191611 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49dc0a21e0 0x7f49dc0a46a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009bd0 tx=0x7f49e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:16.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.009+0000 7f49f259c700 1 --2- 172.21.15.60:0/3201191611 >> [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016] conn(0x7f49dc0acea0 0x7f49dc0af2e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:16.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.009+0000 7f49dbfff700 1 -- 172.21.15.60:0/3201191611 <== osd.0 v2:172.21.15.60:6802/108541016 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f49dc0af9b0 con 0x7f49dc0acea0 2024-02-26T04:56:16.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.027+0000 7f49f908b700 1 -- 172.21.15.60:0/3201191611 --> [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f49f4104cf0 con 0x7f49dc0acea0 2024-02-26T04:56:16.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.028+0000 7f49dbfff700 1 -- 172.21.15.60:0/3201191611 <== osd.0 v2:172.21.15.60:6802/108541016 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f49f4104cf0 con 0x7f49dc0acea0 2024-02-26T04:56:16.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.028+0000 7f49f908b700 1 -- 172.21.15.60:0/3201191611 >> [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016] conn(0x7f49dc0acea0 msgr2=0x7f49dc0af2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:16.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.028+0000 7f49f908b700 1 --2- 172.21.15.60:0/3201191611 >> [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016] conn(0x7f49dc0acea0 0x7f49dc0af2e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:16.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.028+0000 7f49f908b700 1 -- 172.21.15.60:0/3201191611 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49dc0a21e0 msgr2=0x7f49dc0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:16.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.028+0000 7f49f908b700 1 --2- 172.21.15.60:0/3201191611 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49dc0a21e0 0x7f49dc0a46a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009bd0 tx=0x7f49e4009260 comp rx=0 tx=0).stop 2024-02-26T04:56:16.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.028+0000 7f49f908b700 1 -- 172.21.15.60:0/3201191611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49f4106d60 msgr2=0x7f49f419cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:16.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.028+0000 7f49f908b700 1 --2- 172.21.15.60:0/3201191611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49f4106d60 0x7f49f419cad0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f49ec007f10 tx=0x7f49ec00d3b0 comp rx=0 tx=0).stop 2024-02-26T04:56:16.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.028+0000 7f49f359e700 1 -- 172.21.15.60:0/3201191611 reap_dead start 2024-02-26T04:56:16.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.028+0000 7f49f908b700 1 -- 172.21.15.60:0/3201191611 shutdown_connections 2024-02-26T04:56:16.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.028+0000 7f49f908b700 1 -- 172.21.15.60:0/3201191611 >> 172.21.15.60:0/3201191611 conn(0x7f49f40779d0 msgr2=0x7f49f4100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:16.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.028+0000 7f49f908b700 1 -- 172.21.15.60:0/3201191611 shutdown_connections 2024-02-26T04:56:16.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:16.028+0000 7f49f908b700 1 -- 172.21.15.60:0/3201191611 wait complete. 2024-02-26T04:56:16.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:16 smithi060 ceph-mon[32382]: pgmap v317: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T04:56:16.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:16 smithi191 ceph-mon[36352]: pgmap v317: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T04:56:16.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:16 smithi156 ceph-mon[35805]: pgmap v317: 129 pgs: 129 active+clean; 637 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-26T04:56:18.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:18 smithi060 ceph-mon[32382]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T04:56:18.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:18 smithi191 ceph-mon[36352]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T04:56:18.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:18 smithi156 ceph-mon[35805]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-26T04:56:19.275 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-02-26T04:56:19.275 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 6]}]: dispatch 2024-02-26T04:56:19.275 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [1, 8]}]: dispatch 2024-02-26T04:56:19.275 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-02-26T04:56:19.276 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-02-26T04:56:19.276 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-02-26T04:56:19.276 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-02-26T04:56:19.276 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-02-26T04:56:19.276 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-02-26T04:56:19.276 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-02-26T04:56:19.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-02-26T04:56:19.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 6]}]: dispatch 2024-02-26T04:56:19.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [1, 8]}]: dispatch 2024-02-26T04:56:19.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-02-26T04:56:19.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-02-26T04:56:19.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-02-26T04:56:19.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-02-26T04:56:19.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-02-26T04:56:19.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-02-26T04:56:19.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-02-26T04:56:19.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-02-26T04:56:19.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 6]}]: dispatch 2024-02-26T04:56:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [1, 8]}]: dispatch 2024-02-26T04:56:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-02-26T04:56:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-02-26T04:56:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-02-26T04:56:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-02-26T04:56:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-02-26T04:56:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-02-26T04:56:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-02-26T04:56:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:20 smithi060 ceph-mon[32382]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-26T04:56:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-02-26T04:56:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 6]}]': finished 2024-02-26T04:56:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [1, 8]}]': finished 2024-02-26T04:56:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-02-26T04:56:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-02-26T04:56:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-02-26T04:56:20.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-02-26T04:56:20.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-02-26T04:56:20.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-02-26T04:56:20.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-02-26T04:56:20.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:20 smithi060 ceph-mon[32382]: osdmap e90: 12 total, 12 up, 12 in 2024-02-26T04:56:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:20 smithi191 ceph-mon[36352]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-26T04:56:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-02-26T04:56:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 6]}]': finished 2024-02-26T04:56:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [1, 8]}]': finished 2024-02-26T04:56:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-02-26T04:56:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-02-26T04:56:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-02-26T04:56:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-02-26T04:56:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-02-26T04:56:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-02-26T04:56:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-02-26T04:56:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:20 smithi191 ceph-mon[36352]: osdmap e90: 12 total, 12 up, 12 in 2024-02-26T04:56:20.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:20 smithi156 ceph-mon[35805]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-26T04:56:20.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-02-26T04:56:20.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 6]}]': finished 2024-02-26T04:56:20.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [1, 8]}]': finished 2024-02-26T04:56:20.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-02-26T04:56:20.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-02-26T04:56:20.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-02-26T04:56:20.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-02-26T04:56:20.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-02-26T04:56:20.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-02-26T04:56:20.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-02-26T04:56:20.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:20 smithi156 ceph-mon[35805]: osdmap e90: 12 total, 12 up, 12 in 2024-02-26T04:56:21.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:21 smithi060 ceph-mon[32382]: osdmap e91: 12 total, 12 up, 12 in 2024-02-26T04:56:21.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:21 smithi191 ceph-mon[36352]: osdmap e91: 12 total, 12 up, 12 in 2024-02-26T04:56:21.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:21 smithi156 ceph-mon[35805]: osdmap e91: 12 total, 12 up, 12 in 2024-02-26T04:56:21.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.908+0000 7f7a45539700 1 -- 172.21.15.60:0/2960575728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a4006a2c0 msgr2=0x7f7a4010eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:21.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.908+0000 7f7a45539700 1 --2- 172.21.15.60:0/2960575728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a4006a2c0 0x7f7a4010eb00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7a380099c0 tx=0x7f7a38009cd0 comp rx=0 tx=0).stop 2024-02-26T04:56:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.908+0000 7f7a45539700 1 -- 172.21.15.60:0/2960575728 shutdown_connections 2024-02-26T04:56:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.908+0000 7f7a45539700 1 --2- 172.21.15.60:0/2960575728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a4006a2c0 0x7f7a4010eb00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.908+0000 7f7a45539700 1 --2- 172.21.15.60:0/2960575728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a40107cc0 0x7f7a401080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.908+0000 7f7a45539700 1 --2- 172.21.15.60:0/2960575728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a40069920 0x7f7a40069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.908+0000 7f7a45539700 1 -- 172.21.15.60:0/2960575728 >> 172.21.15.60:0/2960575728 conn(0x7f7a40077aa0 msgr2=0x7f7a40077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.908+0000 7f7a45539700 1 -- 172.21.15.60:0/2960575728 shutdown_connections 2024-02-26T04:56:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.908+0000 7f7a45539700 1 -- 172.21.15.60:0/2960575728 wait complete. 2024-02-26T04:56:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.908+0000 7f7a45539700 1 Processor -- start 2024-02-26T04:56:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a45539700 1 -- start start 2024-02-26T04:56:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a45539700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a40069920 0x7f7a4019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a45539700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a4006a2c0 0x7f7a4019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a45539700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a40107cc0 0x7f7a4019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a45539700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4019f210 con 0x7f7a40107cc0 2024-02-26T04:56:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a45539700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4019f380 con 0x7f7a4006a2c0 2024-02-26T04:56:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a45539700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4019f540 con 0x7f7a40069920 2024-02-26T04:56:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a3f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a40107cc0 0x7f7a4019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a3f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a40107cc0 0x7f7a4019ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39482/0 (socket says 172.21.15.60:39482) 2024-02-26T04:56:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a3f7fe700 1 -- 172.21.15.60:0/3489688375 learned_addr learned my addr 172.21.15.60:0/3489688375 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a3effd700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a40069920 0x7f7a4019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a3e7fc700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a4006a2c0 0x7f7a4019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a3f7fe700 1 -- 172.21.15.60:0/3489688375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a40069920 msgr2=0x7f7a4019a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a3f7fe700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a40069920 0x7f7a4019a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a3f7fe700 1 -- 172.21.15.60:0/3489688375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a4006a2c0 msgr2=0x7f7a4019a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a3f7fe700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a4006a2c0 0x7f7a4019a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a3f7fe700 1 -- 172.21.15.60:0/3489688375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a380095f0 con 0x7f7a40107cc0 2024-02-26T04:56:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.909+0000 7f7a3effd700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a40069920 0x7f7a4019a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:56:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.910+0000 7f7a3f7fe700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a40107cc0 0x7f7a4019ec00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7a38005910 tx=0x7f7a38009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.910+0000 7f7a37fff700 1 -- 172.21.15.60:0/3489688375 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3801d070 con 0x7f7a40107cc0 2024-02-26T04:56:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.910+0000 7f7a37fff700 1 -- 172.21.15.60:0/3489688375 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a38003680 con 0x7f7a40107cc0 2024-02-26T04:56:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.910+0000 7f7a45539700 1 -- 172.21.15.60:0/3489688375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4019f7c0 con 0x7f7a40107cc0 2024-02-26T04:56:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.910+0000 7f7a37fff700 1 -- 172.21.15.60:0/3489688375 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38022c20 con 0x7f7a40107cc0 2024-02-26T04:56:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.910+0000 7f7a45539700 1 -- 172.21.15.60:0/3489688375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a401a7080 con 0x7f7a40107cc0 2024-02-26T04:56:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.911+0000 7f7a45539700 1 -- 172.21.15.60:0/3489688375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7a40111f30 con 0x7f7a40107cc0 2024-02-26T04:56:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.912+0000 7f7a37fff700 1 -- 172.21.15.60:0/3489688375 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f7a38003a70 con 0x7f7a40107cc0 2024-02-26T04:56:21.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.913+0000 7f7a37fff700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7a2c0a2260 0x7f7a2c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:21.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.913+0000 7f7a3effd700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7a2c0a2260 0x7f7a2c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:21.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.913+0000 7f7a37fff700 1 -- 172.21.15.60:0/3489688375 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f7a380cf0b0 con 0x7f7a40107cc0 2024-02-26T04:56:21.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.913+0000 7f7a37fff700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364] conn(0x7f7a2c0ad3c0 0x7f7a2c0af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:21.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.913+0000 7f7a37fff700 1 -- 172.21.15.60:0/3489688375 --> [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a2c0aff30 con 0x7f7a2c0ad3c0 2024-02-26T04:56:21.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.913+0000 7f7a37fff700 1 -- 172.21.15.60:0/3489688375 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a380d1730 con 0x7f7a40107cc0 2024-02-26T04:56:21.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.914+0000 7f7a3e7fc700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364] conn(0x7f7a2c0ad3c0 0x7f7a2c0af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:21.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.914+0000 7f7a3effd700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7a2c0a2260 0x7f7a2c0a4720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009550 tx=0x7f7a300091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:21.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.914+0000 7f7a3e7fc700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364] conn(0x7f7a2c0ad3c0 0x7f7a2c0af860 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:21.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.915+0000 7f7a37fff700 1 -- 172.21.15.60:0/3489688375 <== osd.1 v2:172.21.15.156:6800/1258051364 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7a2c0aff30 con 0x7f7a2c0ad3c0 2024-02-26T04:56:21.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.932+0000 7f7a45539700 1 -- 172.21.15.60:0/3489688375 --> [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7a4010d7e0 con 0x7f7a2c0ad3c0 2024-02-26T04:56:21.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.932+0000 7f7a37fff700 1 -- 172.21.15.60:0/3489688375 <== osd.1 v2:172.21.15.156:6800/1258051364 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7a4010d7e0 con 0x7f7a2c0ad3c0 2024-02-26T04:56:21.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.933+0000 7f7a45539700 1 -- 172.21.15.60:0/3489688375 >> [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364] conn(0x7f7a2c0ad3c0 msgr2=0x7f7a2c0af860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:21.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.933+0000 7f7a45539700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364] conn(0x7f7a2c0ad3c0 0x7f7a2c0af860 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:21.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.933+0000 7f7a45539700 1 -- 172.21.15.60:0/3489688375 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7a2c0a2260 msgr2=0x7f7a2c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:21.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.933+0000 7f7a45539700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7a2c0a2260 0x7f7a2c0a4720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009550 tx=0x7f7a300091b0 comp rx=0 tx=0).stop 2024-02-26T04:56:21.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.933+0000 7f7a45539700 1 -- 172.21.15.60:0/3489688375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a40107cc0 msgr2=0x7f7a4019ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:21.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.933+0000 7f7a45539700 1 --2- 172.21.15.60:0/3489688375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a40107cc0 0x7f7a4019ec00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7a38005910 tx=0x7f7a38009580 comp rx=0 tx=0).stop 2024-02-26T04:56:21.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.933+0000 7f7a3f7fe700 1 -- 172.21.15.60:0/3489688375 reap_dead start 2024-02-26T04:56:21.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.933+0000 7f7a45539700 1 -- 172.21.15.60:0/3489688375 shutdown_connections 2024-02-26T04:56:21.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.933+0000 7f7a45539700 1 -- 172.21.15.60:0/3489688375 >> 172.21.15.60:0/3489688375 conn(0x7f7a40077aa0 msgr2=0x7f7a4010ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:21.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.933+0000 7f7a45539700 1 -- 172.21.15.60:0/3489688375 shutdown_connections 2024-02-26T04:56:21.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:21.933+0000 7f7a45539700 1 -- 172.21.15.60:0/3489688375 wait complete. 2024-02-26T04:56:22.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:22 smithi060 ceph-mon[32382]: pgmap v322: 129 pgs: 3 activating, 126 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-02-26T04:56:22.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:22 smithi191 ceph-mon[36352]: pgmap v322: 129 pgs: 3 activating, 126 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-02-26T04:56:22.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:22 smithi156 ceph-mon[35805]: pgmap v322: 129 pgs: 3 activating, 126 active+clean; 641 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-02-26T04:56:23.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.081+0000 7fe7906c9700 1 -- 172.21.15.60:0/4155264958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe788107020 msgr2=0x7fe78810eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.081+0000 7fe7906c9700 1 --2- 172.21.15.60:0/4155264958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe788107020 0x7fe78810eba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe78000b0a0 tx=0x7fe78000b3b0 comp rx=0 tx=0).stop 2024-02-26T04:56:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.082+0000 7fe7906c9700 1 -- 172.21.15.60:0/4155264958 shutdown_connections 2024-02-26T04:56:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.082+0000 7fe7906c9700 1 --2- 172.21.15.60:0/4155264958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe788107020 0x7fe78810eba0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.082+0000 7fe7906c9700 1 --2- 172.21.15.60:0/4155264958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe788100800 0x7fe788100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.082+0000 7fe7906c9700 1 --2- 172.21.15.60:0/4155264958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7881011b0 0x7fe7881068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.082+0000 7fe7906c9700 1 -- 172.21.15.60:0/4155264958 >> 172.21.15.60:0/4155264958 conn(0x7fe788075ea0 msgr2=0x7fe7880762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.082+0000 7fe7906c9700 1 -- 172.21.15.60:0/4155264958 shutdown_connections 2024-02-26T04:56:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.082+0000 7fe7906c9700 1 -- 172.21.15.60:0/4155264958 wait complete. 2024-02-26T04:56:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.082+0000 7fe7906c9700 1 Processor -- start 2024-02-26T04:56:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.082+0000 7fe7906c9700 1 -- start start 2024-02-26T04:56:23.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.082+0000 7fe7906c9700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe788100800 0x7fe788101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:23.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.082+0000 7fe7906c9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7881011b0 0x7fe788101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:23.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.082+0000 7fe7906c9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe788107020 0x7fe788102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:23.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe7906c9700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7881a66d0 con 0x7fe788100800 2024-02-26T04:56:23.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe7906c9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7881a6840 con 0x7fe788107020 2024-02-26T04:56:23.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe7906c9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7881a6a00 con 0x7fe7881011b0 2024-02-26T04:56:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe78e465700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe788100800 0x7fe788101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe78e465700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe788100800 0x7fe788101780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39504/0 (socket says 172.21.15.60:39504) 2024-02-26T04:56:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe78e465700 1 -- 172.21.15.60:0/2423446568 learned_addr learned my addr 172.21.15.60:0/2423446568 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe78ec66700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe788107020 0x7fe788102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe78dc64700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7881011b0 0x7fe788101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe78ec66700 1 -- 172.21.15.60:0/2423446568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7881011b0 msgr2=0x7fe788101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe78ec66700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7881011b0 0x7fe788101cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe78ec66700 1 -- 172.21.15.60:0/2423446568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe788100800 msgr2=0x7fe788101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe78ec66700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe788100800 0x7fe788101780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe78ec66700 1 -- 172.21.15.60:0/2423446568 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe780009d00 con 0x7fe788107020 2024-02-26T04:56:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe78e465700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe788100800 0x7fe788101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:56:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe78ec66700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe788107020 0x7fe788102200 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe78000b0a0 tx=0x7fe78000be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.083+0000 7fe77f7fe700 1 -- 172.21.15.60:0/2423446568 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78000e040 con 0x7fe788107020 2024-02-26T04:56:23.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.084+0000 7fe77f7fe700 1 -- 172.21.15.60:0/2423446568 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe78001d470 con 0x7fe788107020 2024-02-26T04:56:23.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.084+0000 7fe7906c9700 1 -- 172.21.15.60:0/2423446568 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7881a6c80 con 0x7fe788107020 2024-02-26T04:56:23.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.084+0000 7fe77f7fe700 1 -- 172.21.15.60:0/2423446568 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7800052a0 con 0x7fe788107020 2024-02-26T04:56:23.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.084+0000 7fe7906c9700 1 -- 172.21.15.60:0/2423446568 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7881a70f0 con 0x7fe788107020 2024-02-26T04:56:23.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.084+0000 7fe7906c9700 1 -- 172.21.15.60:0/2423446568 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe788111fb0 con 0x7fe788107020 2024-02-26T04:56:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.086+0000 7fe77f7fe700 1 -- 172.21.15.60:0/2423446568 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe78001d5e0 con 0x7fe788107020 2024-02-26T04:56:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.087+0000 7fe77f7fe700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe7740a21e0 0x7fe7740a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.087+0000 7fe78e465700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe7740a21e0 0x7fe7740a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.087+0000 7fe77f7fe700 1 -- 172.21.15.60:0/2423446568 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fe7800cf810 con 0x7fe788107020 2024-02-26T04:56:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.087+0000 7fe77f7fe700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318] conn(0x7fe7740ad210 0x7fe7740af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.087+0000 7fe77f7fe700 1 -- 172.21.15.60:0/2423446568 --> [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7740afd80 con 0x7fe7740ad210 2024-02-26T04:56:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.087+0000 7fe77f7fe700 1 -- 172.21.15.60:0/2423446568 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7800cfc20 con 0x7fe788107020 2024-02-26T04:56:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.087+0000 7fe78dc64700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318] conn(0x7fe7740ad210 0x7fe7740af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.087+0000 7fe78e465700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe7740a21e0 0x7fe7740a46a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe784005950 tx=0x7fe78400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:23.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.088+0000 7fe78dc64700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318] conn(0x7fe7740ad210 0x7fe7740af6b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:23.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.089+0000 7fe77f7fe700 1 -- 172.21.15.60:0/2423446568 <== osd.7 v2:172.21.15.156:6816/2499710318 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe7740afd80 con 0x7fe7740ad210 2024-02-26T04:56:23.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.105+0000 7fe7906c9700 1 -- 172.21.15.60:0/2423446568 --> [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe788062380 con 0x7fe7740ad210 2024-02-26T04:56:23.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.105+0000 7fe77f7fe700 1 -- 172.21.15.60:0/2423446568 <== osd.7 v2:172.21.15.156:6816/2499710318 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe788062380 con 0x7fe7740ad210 2024-02-26T04:56:23.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.106+0000 7fe7906c9700 1 -- 172.21.15.60:0/2423446568 >> [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318] conn(0x7fe7740ad210 msgr2=0x7fe7740af6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:23.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.106+0000 7fe7906c9700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318] conn(0x7fe7740ad210 0x7fe7740af6b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:23.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.106+0000 7fe7906c9700 1 -- 172.21.15.60:0/2423446568 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe7740a21e0 msgr2=0x7fe7740a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:23.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.106+0000 7fe7906c9700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe7740a21e0 0x7fe7740a46a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe784005950 tx=0x7fe78400b500 comp rx=0 tx=0).stop 2024-02-26T04:56:23.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.106+0000 7fe7906c9700 1 -- 172.21.15.60:0/2423446568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe788107020 msgr2=0x7fe788102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:23.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.106+0000 7fe7906c9700 1 --2- 172.21.15.60:0/2423446568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe788107020 0x7fe788102200 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe78000b0a0 tx=0x7fe78000be80 comp rx=0 tx=0).stop 2024-02-26T04:56:23.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.106+0000 7fe78ec66700 1 -- 172.21.15.60:0/2423446568 reap_dead start 2024-02-26T04:56:23.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.106+0000 7fe7906c9700 1 -- 172.21.15.60:0/2423446568 shutdown_connections 2024-02-26T04:56:23.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.106+0000 7fe7906c9700 1 -- 172.21.15.60:0/2423446568 >> 172.21.15.60:0/2423446568 conn(0x7fe788075ea0 msgr2=0x7fe7880fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:23.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.106+0000 7fe7906c9700 1 -- 172.21.15.60:0/2423446568 shutdown_connections 2024-02-26T04:56:23.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:23.106+0000 7fe7906c9700 1 -- 172.21.15.60:0/2423446568 wait complete. 2024-02-26T04:56:23.909 INFO:teuthology.orchestra.run.smithi060.stdout:219043332141 2024-02-26T04:56:23.909 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-26T04:56:24.048 INFO:teuthology.orchestra.run.smithi060.stdout:34359738447 2024-02-26T04:56:24.049 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-26T04:56:24.433 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:24 smithi060 ceph-mon[32382]: pgmap v323: 129 pgs: 3 activating, 126 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-02-26T04:56:24.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:24 smithi191 ceph-mon[36352]: pgmap v323: 129 pgs: 3 activating, 126 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-02-26T04:56:24.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:24 smithi156 ceph-mon[35805]: pgmap v323: 129 pgs: 3 activating, 126 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-02-26T04:56:26.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:26 smithi060 ceph-mon[32382]: pgmap v324: 129 pgs: 3 activating, 126 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T04:56:26.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:26 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:56:26.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:26 smithi191 ceph-mon[36352]: pgmap v324: 129 pgs: 3 activating, 126 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T04:56:26.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:26 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:56:26.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:26 smithi156 ceph-mon[35805]: pgmap v324: 129 pgs: 3 activating, 126 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-26T04:56:26.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:26 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:56:27.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.618+0000 7ffa66dde700 1 -- 172.21.15.60:0/2190103484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa60107ca0 msgr2=0x7ffa60108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:27.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.618+0000 7ffa66dde700 1 --2- 172.21.15.60:0/2190103484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa60107ca0 0x7ffa60108080 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffa540099c0 tx=0x7ffa54009cd0 comp rx=0 tx=0).stop 2024-02-26T04:56:27.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.618+0000 7ffa66dde700 1 -- 172.21.15.60:0/2190103484 shutdown_connections 2024-02-26T04:56:27.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.618+0000 7ffa66dde700 1 --2- 172.21.15.60:0/2190103484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa60102680 0x7ffa6010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:27.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.618+0000 7ffa66dde700 1 --2- 172.21.15.60:0/2190103484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa60107ca0 0x7ffa60108080 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:27.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.618+0000 7ffa66dde700 1 --2- 172.21.15.60:0/2190103484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa60101cc0 0x7ffa60102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:27.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.618+0000 7ffa66dde700 1 -- 172.21.15.60:0/2190103484 >> 172.21.15.60:0/2190103484 conn(0x7ffa60078da0 msgr2=0x7ffa600791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.618+0000 7ffa66dde700 1 -- 172.21.15.60:0/2190103484 shutdown_connections 2024-02-26T04:56:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.618+0000 7ffa66dde700 1 -- 172.21.15.60:0/2190103484 wait complete. 2024-02-26T04:56:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.618+0000 7ffa66dde700 1 Processor -- start 2024-02-26T04:56:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.618+0000 7ffa66dde700 1 -- start start 2024-02-26T04:56:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa66dde700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa60101cc0 0x7ffa60073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa66dde700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa60102680 0x7ffa600744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa66dde700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa60107ca0 0x7ffa600749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa66dde700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa601a64b0 con 0x7ffa60102680 2024-02-26T04:56:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa66dde700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa601a6620 con 0x7ffa60101cc0 2024-02-26T04:56:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa66dde700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa601a67e0 con 0x7ffa60107ca0 2024-02-26T04:56:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa60102680 0x7ffa600744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa64b7a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa60101cc0 0x7ffa60073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa60102680 0x7ffa600744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39526/0 (socket says 172.21.15.60:39526) 2024-02-26T04:56:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa5ffff700 1 -- 172.21.15.60:0/2917798128 learned_addr learned my addr 172.21.15.60:0/2917798128 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa6537b700 1 --2- 172.21.15.60:0/2917798128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa60107ca0 0x7ffa600749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa64b7a700 1 -- 172.21.15.60:0/2917798128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa60107ca0 msgr2=0x7ffa600749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa64b7a700 1 --2- 172.21.15.60:0/2917798128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa60107ca0 0x7ffa600749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa64b7a700 1 -- 172.21.15.60:0/2917798128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa60102680 msgr2=0x7ffa600744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa64b7a700 1 --2- 172.21.15.60:0/2917798128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa60102680 0x7ffa600744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa64b7a700 1 -- 172.21.15.60:0/2917798128 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa540095f0 con 0x7ffa60101cc0 2024-02-26T04:56:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa5ffff700 1 --2- 172.21.15.60:0/2917798128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa60102680 0x7ffa600744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:56:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa64b7a700 1 --2- 172.21.15.60:0/2917798128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa60101cc0 0x7ffa60073f70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa540098e0 tx=0x7ffa5400bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa5dffb700 1 -- 172.21.15.60:0/2917798128 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5401d070 con 0x7ffa60101cc0 2024-02-26T04:56:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.619+0000 7ffa66dde700 1 -- 172.21.15.60:0/2917798128 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa601a6a60 con 0x7ffa60101cc0 2024-02-26T04:56:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.620+0000 7ffa5dffb700 1 -- 172.21.15.60:0/2917798128 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa54003680 con 0x7ffa60101cc0 2024-02-26T04:56:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.620+0000 7ffa5dffb700 1 -- 172.21.15.60:0/2917798128 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54022e00 con 0x7ffa60101cc0 2024-02-26T04:56:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.620+0000 7ffa66dde700 1 -- 172.21.15.60:0/2917798128 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa601a6ed0 con 0x7ffa60101cc0 2024-02-26T04:56:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.620+0000 7ffa66dde700 1 -- 172.21.15.60:0/2917798128 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa60111fd0 con 0x7ffa60101cc0 2024-02-26T04:56:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.622+0000 7ffa5dffb700 1 -- 172.21.15.60:0/2917798128 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ffa54022500 con 0x7ffa60101cc0 2024-02-26T04:56:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.623+0000 7ffa5dffb700 1 --2- 172.21.15.60:0/2917798128 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa500a2260 0x7ffa500a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.623+0000 7ffa5ffff700 1 --2- 172.21.15.60:0/2917798128 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa500a2260 0x7ffa500a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.623+0000 7ffa5dffb700 1 -- 172.21.15.60:0/2917798128 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ffa540cf4d0 con 0x7ffa60101cc0 2024-02-26T04:56:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.623+0000 7ffa5dffb700 1 --2- 172.21.15.60:0/2917798128 >> [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576] conn(0x7ffa500ad3c0 0x7ffa500af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.623+0000 7ffa5dffb700 1 -- 172.21.15.60:0/2917798128 --> [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa500aff30 con 0x7ffa500ad3c0 2024-02-26T04:56:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.623+0000 7ffa5dffb700 1 -- 172.21.15.60:0/2917798128 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa540cf840 con 0x7ffa60101cc0 2024-02-26T04:56:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.623+0000 7ffa6537b700 1 --2- 172.21.15.60:0/2917798128 >> [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576] conn(0x7ffa500ad3c0 0x7ffa500af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.623+0000 7ffa5ffff700 1 --2- 172.21.15.60:0/2917798128 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa500a2260 0x7ffa500a4720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffa60079330 tx=0x7ffa4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.623+0000 7ffa6537b700 1 --2- 172.21.15.60:0/2917798128 >> [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576] conn(0x7ffa500ad3c0 0x7ffa500af860 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.624+0000 7ffa5dffb700 1 -- 172.21.15.60:0/2917798128 <== osd.4 v2:172.21.15.156:6808/714533576 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffa500aff30 con 0x7ffa500ad3c0 2024-02-26T04:56:27.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.641+0000 7ffa66dde700 1 -- 172.21.15.60:0/2917798128 --> [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa60062380 con 0x7ffa500ad3c0 2024-02-26T04:56:27.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.642+0000 7ffa5dffb700 1 -- 172.21.15.60:0/2917798128 <== osd.4 v2:172.21.15.156:6808/714533576 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ffa60062380 con 0x7ffa500ad3c0 2024-02-26T04:56:27.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.642+0000 7ffa66dde700 1 -- 172.21.15.60:0/2917798128 >> [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576] conn(0x7ffa500ad3c0 msgr2=0x7ffa500af860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:27.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.642+0000 7ffa66dde700 1 --2- 172.21.15.60:0/2917798128 >> [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576] conn(0x7ffa500ad3c0 0x7ffa500af860 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:27.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.642+0000 7ffa66dde700 1 -- 172.21.15.60:0/2917798128 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa500a2260 msgr2=0x7ffa500a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:27.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.642+0000 7ffa66dde700 1 --2- 172.21.15.60:0/2917798128 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa500a2260 0x7ffa500a4720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffa60079330 tx=0x7ffa4c009260 comp rx=0 tx=0).stop 2024-02-26T04:56:27.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.642+0000 7ffa66dde700 1 -- 172.21.15.60:0/2917798128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa60101cc0 msgr2=0x7ffa60073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:27.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.642+0000 7ffa66dde700 1 --2- 172.21.15.60:0/2917798128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa60101cc0 0x7ffa60073f70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa540098e0 tx=0x7ffa5400bfd0 comp rx=0 tx=0).stop 2024-02-26T04:56:27.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.642+0000 7ffa6537b700 1 -- 172.21.15.60:0/2917798128 reap_dead start 2024-02-26T04:56:27.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.642+0000 7ffa66dde700 1 -- 172.21.15.60:0/2917798128 shutdown_connections 2024-02-26T04:56:27.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.642+0000 7ffa66dde700 1 -- 172.21.15.60:0/2917798128 >> 172.21.15.60:0/2917798128 conn(0x7ffa60078da0 msgr2=0x7ffa60100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:27.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.642+0000 7ffa66dde700 1 -- 172.21.15.60:0/2917798128 shutdown_connections 2024-02-26T04:56:27.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:27.642+0000 7ffa66dde700 1 -- 172.21.15.60:0/2917798128 wait complete. 2024-02-26T04:56:28.405 INFO:teuthology.orchestra.run.smithi060.stdout:55834574923 2024-02-26T04:56:28.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-26T04:56:28.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:28 smithi060 ceph-mon[32382]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 7 op/s 2024-02-26T04:56:28.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:28 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:56:28.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:28 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:56:28.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:28 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:56:28.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:28 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:56:28.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:28 smithi191 ceph-mon[36352]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 7 op/s 2024-02-26T04:56:28.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:28 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:56:28.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:28 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:56:28.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:28 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:56:28.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:28 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:56:28.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:28 smithi156 ceph-mon[35805]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 7 op/s 2024-02-26T04:56:28.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:28 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:56:28.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:28 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:56:28.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:28 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:56:28.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:28 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:56:28.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.977+0000 7f5d311eb700 1 -- 172.21.15.60:0/3698412700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2c06a570 msgr2=0x7f5d2c06a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:28.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.977+0000 7f5d311eb700 1 --2- 172.21.15.60:0/3698412700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2c06a570 0x7f5d2c06a950 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009990 tx=0x7f5d1c009ca0 comp rx=0 tx=0).stop 2024-02-26T04:56:28.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.977+0000 7f5d311eb700 1 -- 172.21.15.60:0/3698412700 shutdown_connections 2024-02-26T04:56:28.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.977+0000 7f5d311eb700 1 --2- 172.21.15.60:0/3698412700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d2c106fe0 0x7f5d2c10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:28.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.977+0000 7f5d311eb700 1 --2- 172.21.15.60:0/3698412700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2c06a570 0x7f5d2c06a950 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:28.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.977+0000 7f5d311eb700 1 --2- 172.21.15.60:0/3698412700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d2c06af20 0x7f5d2c1068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:28.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.977+0000 7f5d311eb700 1 -- 172.21.15.60:0/3698412700 >> 172.21.15.60:0/3698412700 conn(0x7f5d2c077ae0 msgr2=0x7f5d2c077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:28.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.977+0000 7f5d311eb700 1 -- 172.21.15.60:0/3698412700 shutdown_connections 2024-02-26T04:56:28.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.977+0000 7f5d311eb700 1 -- 172.21.15.60:0/3698412700 wait complete. 2024-02-26T04:56:28.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.978+0000 7f5d311eb700 1 Processor -- start 2024-02-26T04:56:28.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.978+0000 7f5d311eb700 1 -- start start 2024-02-26T04:56:28.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.978+0000 7f5d311eb700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2c06a570 0x7f5d2c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:28.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.978+0000 7f5d311eb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d2c06af20 0x7f5d2c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:28.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.978+0000 7f5d311eb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d2c106fe0 0x7f5d2c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:28.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.978+0000 7f5d311eb700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c19d0e0 con 0x7f5d2c06a570 2024-02-26T04:56:28.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.978+0000 7f5d311eb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c19d250 con 0x7f5d2c106fe0 2024-02-26T04:56:28.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.978+0000 7f5d311eb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c19d410 con 0x7f5d2c06af20 2024-02-26T04:56:28.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d2ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2c06a570 0x7f5d2c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:28.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d2ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2c06a570 0x7f5d2c198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39552/0 (socket says 172.21.15.60:39552) 2024-02-26T04:56:28.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d2ad9d700 1 -- 172.21.15.60:0/1608165016 learned_addr learned my addr 172.21.15.60:0/1608165016 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:28.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d2a59c700 1 --2- 172.21.15.60:0/1608165016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d2c06af20 0x7f5d2c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:28.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d2b59e700 1 --2- 172.21.15.60:0/1608165016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d2c106fe0 0x7f5d2c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:28.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d2ad9d700 1 -- 172.21.15.60:0/1608165016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d2c06af20 msgr2=0x7f5d2c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:28.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d2ad9d700 1 --2- 172.21.15.60:0/1608165016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d2c06af20 0x7f5d2c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:28.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d2ad9d700 1 -- 172.21.15.60:0/1608165016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d2c106fe0 msgr2=0x7f5d2c19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:28.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d2ad9d700 1 --2- 172.21.15.60:0/1608165016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d2c106fe0 0x7f5d2c19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:28.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d2ad9d700 1 -- 172.21.15.60:0/1608165016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c0095f0 con 0x7f5d2c06a570 2024-02-26T04:56:28.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d2ad9d700 1 --2- 172.21.15.60:0/1608165016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2c06a570 0x7f5d2c198200 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005b40 tx=0x7f5d1c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:28.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d13fff700 1 -- 172.21.15.60:0/1608165016 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c01d070 con 0x7f5d2c06a570 2024-02-26T04:56:28.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d311eb700 1 -- 172.21.15.60:0/1608165016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c19d690 con 0x7f5d2c06a570 2024-02-26T04:56:28.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d13fff700 1 -- 172.21.15.60:0/1608165016 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c003680 con 0x7f5d2c06a570 2024-02-26T04:56:28.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d13fff700 1 -- 172.21.15.60:0/1608165016 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c005440 con 0x7f5d2c06a570 2024-02-26T04:56:28.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.979+0000 7f5d311eb700 1 -- 172.21.15.60:0/1608165016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c19d950 con 0x7f5d2c06a570 2024-02-26T04:56:28.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.980+0000 7f5d311eb700 1 -- 172.21.15.60:0/1608165016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d2c111fc0 con 0x7f5d2c06a570 2024-02-26T04:56:28.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.981+0000 7f5d13fff700 1 -- 172.21.15.60:0/1608165016 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f5d1c0055a0 con 0x7f5d2c06a570 2024-02-26T04:56:28.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.982+0000 7f5d13fff700 1 --2- 172.21.15.60:0/1608165016 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5d140a2260 0x7f5d140a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:28.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.982+0000 7f5d2a59c700 1 --2- 172.21.15.60:0/1608165016 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5d140a2260 0x7f5d140a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:28.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.982+0000 7f5d13fff700 1 -- 172.21.15.60:0/1608165016 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f5d1c0cf430 con 0x7f5d2c06a570 2024-02-26T04:56:28.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.982+0000 7f5d13fff700 1 --2- 172.21.15.60:0/1608165016 >> [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804] conn(0x7f5d140ad290 0x7f5d140af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:28.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.982+0000 7f5d13fff700 1 -- 172.21.15.60:0/1608165016 --> [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d140afe00 con 0x7f5d140ad290 2024-02-26T04:56:28.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.982+0000 7f5d13fff700 1 -- 172.21.15.60:0/1608165016 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d1c0cf840 con 0x7f5d2c06a570 2024-02-26T04:56:28.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.982+0000 7f5d2b59e700 1 --2- 172.21.15.60:0/1608165016 >> [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804] conn(0x7f5d140ad290 0x7f5d140af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:28.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.982+0000 7f5d2a59c700 1 --2- 172.21.15.60:0/1608165016 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5d140a2260 0x7f5d140a4720 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c100770 tx=0x7f5d2000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:28.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.983+0000 7f5d2b59e700 1 --2- 172.21.15.60:0/1608165016 >> [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804] conn(0x7f5d140ad290 0x7f5d140af730 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:28.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:28.983+0000 7f5d13fff700 1 -- 172.21.15.60:0/1608165016 <== osd.6 v2:172.21.15.60:6818/3054430804 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5d140afe00 con 0x7f5d140ad290 2024-02-26T04:56:29.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.000+0000 7f5d311eb700 1 -- 172.21.15.60:0/1608165016 --> [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5d2c062300 con 0x7f5d140ad290 2024-02-26T04:56:29.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.000+0000 7f5d13fff700 1 -- 172.21.15.60:0/1608165016 <== osd.6 v2:172.21.15.60:6818/3054430804 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5d2c062300 con 0x7f5d140ad290 2024-02-26T04:56:29.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.000+0000 7f5d311eb700 1 -- 172.21.15.60:0/1608165016 >> [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804] conn(0x7f5d140ad290 msgr2=0x7f5d140af730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.000+0000 7f5d311eb700 1 --2- 172.21.15.60:0/1608165016 >> [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804] conn(0x7f5d140ad290 0x7f5d140af730 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.001+0000 7f5d311eb700 1 -- 172.21.15.60:0/1608165016 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5d140a2260 msgr2=0x7f5d140a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.001+0000 7f5d311eb700 1 --2- 172.21.15.60:0/1608165016 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5d140a2260 0x7f5d140a4720 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c100770 tx=0x7f5d2000b410 comp rx=0 tx=0).stop 2024-02-26T04:56:29.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.001+0000 7f5d311eb700 1 -- 172.21.15.60:0/1608165016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2c06a570 msgr2=0x7f5d2c198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.001+0000 7f5d311eb700 1 --2- 172.21.15.60:0/1608165016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2c06a570 0x7f5d2c198200 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005b40 tx=0x7f5d1c00f690 comp rx=0 tx=0).stop 2024-02-26T04:56:29.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.001+0000 7f5d2b59e700 1 -- 172.21.15.60:0/1608165016 reap_dead start 2024-02-26T04:56:29.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.001+0000 7f5d311eb700 1 -- 172.21.15.60:0/1608165016 shutdown_connections 2024-02-26T04:56:29.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.001+0000 7f5d311eb700 1 -- 172.21.15.60:0/1608165016 >> 172.21.15.60:0/1608165016 conn(0x7f5d2c077ae0 msgr2=0x7f5d2c108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:29.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.001+0000 7f5d311eb700 1 -- 172.21.15.60:0/1608165016 shutdown_connections 2024-02-26T04:56:29.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.001+0000 7f5d311eb700 1 -- 172.21.15.60:0/1608165016 wait complete. 2024-02-26T04:56:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.195+0000 7fbb2ec09700 1 -- 172.21.15.60:0/1933461587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28101f20 msgr2=0x7fbb28102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.195+0000 7fbb2ec09700 1 --2- 172.21.15.60:0/1933461587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28101f20 0x7fbb28102300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009990 tx=0x7fbb1c009ca0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.195+0000 7fbb2ec09700 1 -- 172.21.15.60:0/1933461587 shutdown_connections 2024-02-26T04:56:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.195+0000 7fbb2ec09700 1 --2- 172.21.15.60:0/1933461587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb28106ff0 0x7fbb2810ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.195+0000 7fbb2ec09700 1 --2- 172.21.15.60:0/1933461587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28101f20 0x7fbb28102300 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.195+0000 7fbb2ec09700 1 --2- 172.21.15.60:0/1933461587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb281028d0 0x7fbb28106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.195+0000 7fbb2ec09700 1 -- 172.21.15.60:0/1933461587 >> 172.21.15.60:0/1933461587 conn(0x7fbb28078dd0 msgr2=0x7fbb280791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.195+0000 7fbb2ec09700 1 -- 172.21.15.60:0/1933461587 shutdown_connections 2024-02-26T04:56:29.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.195+0000 7fbb2ec09700 1 -- 172.21.15.60:0/1933461587 wait complete. 2024-02-26T04:56:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.196+0000 7fbb2ec09700 1 Processor -- start 2024-02-26T04:56:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.196+0000 7fbb2ec09700 1 -- start start 2024-02-26T04:56:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.196+0000 7fbb2ec09700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb28101f20 0x7fbb28198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.196+0000 7fbb2ec09700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb281028d0 0x7fbb281987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.196+0000 7fbb2ec09700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28106ff0 0x7fbb2819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.196+0000 7fbb2ec09700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2819d0f0 con 0x7fbb28106ff0 2024-02-26T04:56:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.196+0000 7fbb2ec09700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2819d260 con 0x7fbb28101f20 2024-02-26T04:56:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.196+0000 7fbb2ec09700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2819d420 con 0x7fbb281028d0 2024-02-26T04:56:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.196+0000 7fbb2d1a6700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28106ff0 0x7fbb2819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.196+0000 7fbb27fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb281028d0 0x7fbb281987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.196+0000 7fbb2d1a6700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28106ff0 0x7fbb2819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39578/0 (socket says 172.21.15.60:39578) 2024-02-26T04:56:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.196+0000 7fbb27fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb281028d0 0x7fbb281987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.60:55492/0 (socket says 172.21.15.60:55492) 2024-02-26T04:56:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.196+0000 7fbb2d1a6700 1 -- 172.21.15.60:0/2645221067 learned_addr learned my addr 172.21.15.60:0/2645221067 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.196+0000 7fbb2c9a5700 1 --2- 172.21.15.60:0/2645221067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb28101f20 0x7fbb28198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.197+0000 7fbb27fff700 1 -- 172.21.15.60:0/2645221067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb28101f20 msgr2=0x7fbb28198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.197+0000 7fbb27fff700 1 --2- 172.21.15.60:0/2645221067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb28101f20 0x7fbb28198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.197+0000 7fbb27fff700 1 -- 172.21.15.60:0/2645221067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28106ff0 msgr2=0x7fbb2819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.197+0000 7fbb27fff700 1 --2- 172.21.15.60:0/2645221067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28106ff0 0x7fbb2819cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.197+0000 7fbb27fff700 1 -- 172.21.15.60:0/2645221067 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb20009c40 con 0x7fbb281028d0 2024-02-26T04:56:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.197+0000 7fbb2d1a6700 1 --2- 172.21.15.60:0/2645221067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28106ff0 0x7fbb2819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:56:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.197+0000 7fbb27fff700 1 --2- 172.21.15.60:0/2645221067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb281028d0 0x7fbb281987a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400e9c0 tx=0x7fbb1400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.197+0000 7fbb25ffb700 1 -- 172.21.15.60:0/2645221067 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400c9c0 con 0x7fbb281028d0 2024-02-26T04:56:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.197+0000 7fbb2ec09700 1 -- 172.21.15.60:0/2645221067 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c0095f0 con 0x7fbb281028d0 2024-02-26T04:56:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.197+0000 7fbb25ffb700 1 -- 172.21.15.60:0/2645221067 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb14010430 con 0x7fbb281028d0 2024-02-26T04:56:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.197+0000 7fbb25ffb700 1 -- 172.21.15.60:0/2645221067 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14018660 con 0x7fbb281028d0 2024-02-26T04:56:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.197+0000 7fbb2ec09700 1 -- 172.21.15.60:0/2645221067 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb281a4d90 con 0x7fbb281028d0 2024-02-26T04:56:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.198+0000 7fbb2ec09700 1 -- 172.21.15.60:0/2645221067 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb28112020 con 0x7fbb281028d0 2024-02-26T04:56:29.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.199+0000 7fbb25ffb700 1 -- 172.21.15.60:0/2645221067 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fbb140105a0 con 0x7fbb281028d0 2024-02-26T04:56:29.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.200+0000 7fbb25ffb700 1 --2- 172.21.15.60:0/2645221067 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbb180a21e0 0x7fbb180a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:29.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.200+0000 7fbb25ffb700 1 -- 172.21.15.60:0/2645221067 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fbb14014070 con 0x7fbb281028d0 2024-02-26T04:56:29.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.200+0000 7fbb2c9a5700 1 --2- 172.21.15.60:0/2645221067 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbb180a21e0 0x7fbb180a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:29.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.200+0000 7fbb25ffb700 1 --2- 172.21.15.60:0/2645221067 >> [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954] conn(0x7fbb180ad340 0x7fbb180af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:29.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.200+0000 7fbb25ffb700 1 -- 172.21.15.60:0/2645221067 --> [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb180afeb0 con 0x7fbb180ad340 2024-02-26T04:56:29.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.200+0000 7fbb25ffb700 1 -- 172.21.15.60:0/2645221067 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb140ced00 con 0x7fbb281028d0 2024-02-26T04:56:29.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.201+0000 7fbb2d1a6700 1 --2- 172.21.15.60:0/2645221067 >> [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954] conn(0x7fbb180ad340 0x7fbb180af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:29.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.201+0000 7fbb2c9a5700 1 --2- 172.21.15.60:0/2645221067 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbb180a21e0 0x7fbb180a46a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00b2e0 tx=0x7fbb1c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:29.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.201+0000 7fbb2d1a6700 1 --2- 172.21.15.60:0/2645221067 >> [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954] conn(0x7fbb180ad340 0x7fbb180af7e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:29.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.202+0000 7fbb25ffb700 1 -- 172.21.15.60:0/2645221067 <== osd.2 v2:172.21.15.191:6800/1607191954 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbb180afeb0 con 0x7fbb180ad340 2024-02-26T04:56:29.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.221+0000 7fbb2ec09700 1 -- 172.21.15.60:0/2645221067 --> [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb28062300 con 0x7fbb180ad340 2024-02-26T04:56:29.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.222+0000 7fbb25ffb700 1 -- 172.21.15.60:0/2645221067 <== osd.2 v2:172.21.15.191:6800/1607191954 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbb28062300 con 0x7fbb180ad340 2024-02-26T04:56:29.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.222+0000 7fbb2ec09700 1 -- 172.21.15.60:0/2645221067 >> [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954] conn(0x7fbb180ad340 msgr2=0x7fbb180af7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.222+0000 7fbb2ec09700 1 --2- 172.21.15.60:0/2645221067 >> [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954] conn(0x7fbb180ad340 0x7fbb180af7e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.222+0000 7fbb2ec09700 1 -- 172.21.15.60:0/2645221067 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbb180a21e0 msgr2=0x7fbb180a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.222+0000 7fbb2ec09700 1 --2- 172.21.15.60:0/2645221067 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbb180a21e0 0x7fbb180a46a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00b2e0 tx=0x7fbb1c005960 comp rx=0 tx=0).stop 2024-02-26T04:56:29.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.222+0000 7fbb2ec09700 1 -- 172.21.15.60:0/2645221067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb281028d0 msgr2=0x7fbb281987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.222+0000 7fbb2ec09700 1 --2- 172.21.15.60:0/2645221067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb281028d0 0x7fbb281987a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400e9c0 tx=0x7fbb1400ecd0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.222+0000 7fbb2d1a6700 1 -- 172.21.15.60:0/2645221067 reap_dead start 2024-02-26T04:56:29.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.222+0000 7fbb2ec09700 1 -- 172.21.15.60:0/2645221067 shutdown_connections 2024-02-26T04:56:29.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.222+0000 7fbb2ec09700 1 -- 172.21.15.60:0/2645221067 >> 172.21.15.60:0/2645221067 conn(0x7fbb28078dd0 msgr2=0x7fbb28106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:29.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.223+0000 7fbb2ec09700 1 -- 172.21.15.60:0/2645221067 shutdown_connections 2024-02-26T04:56:29.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.223+0000 7fbb2ec09700 1 -- 172.21.15.60:0/2645221067 wait complete. 2024-02-26T04:56:29.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.642+0000 7fd20a074700 1 -- 172.21.15.60:0/3884587814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd204107d40 msgr2=0x7fd204108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.642+0000 7fd20a074700 1 --2- 172.21.15.60:0/3884587814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd204107d40 0x7fd204108120 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009990 tx=0x7fd1f4009ca0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.642+0000 7fd20a074700 1 -- 172.21.15.60:0/3884587814 shutdown_connections 2024-02-26T04:56:29.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.642+0000 7fd20a074700 1 --2- 172.21.15.60:0/3884587814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2041026b0 0x7fd20410eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.642+0000 7fd20a074700 1 --2- 172.21.15.60:0/3884587814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd204107d40 0x7fd204108120 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.642+0000 7fd20a074700 1 --2- 172.21.15.60:0/3884587814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd204101cf0 0x7fd204102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.642+0000 7fd20a074700 1 -- 172.21.15.60:0/3884587814 >> 172.21.15.60:0/3884587814 conn(0x7fd204078dc0 msgr2=0x7fd2040791d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:29.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.642+0000 7fd20a074700 1 -- 172.21.15.60:0/3884587814 shutdown_connections 2024-02-26T04:56:29.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.642+0000 7fd20a074700 1 -- 172.21.15.60:0/3884587814 wait complete. 2024-02-26T04:56:29.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.643+0000 7fd20a074700 1 Processor -- start 2024-02-26T04:56:29.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.643+0000 7fd20a074700 1 -- start start 2024-02-26T04:56:29.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.643+0000 7fd20a074700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204101cf0 0x7fd2041980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:29.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.643+0000 7fd20a074700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2041026b0 0x7fd204198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:29.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.643+0000 7fd20a074700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd204107d40 0x7fd20419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:29.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.643+0000 7fd20a074700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20419cfd0 con 0x7fd2041026b0 2024-02-26T04:56:29.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.643+0000 7fd20a074700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20419d140 con 0x7fd204101cf0 2024-02-26T04:56:29.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.643+0000 7fd20a074700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20419d300 con 0x7fd204107d40 2024-02-26T04:56:29.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.643+0000 7fd202ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2041026b0 0x7fd204198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:29.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.643+0000 7fd202ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2041026b0 0x7fd204198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53220/0 (socket says 172.21.15.60:53220) 2024-02-26T04:56:29.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.643+0000 7fd202ffd700 1 -- 172.21.15.60:0/984170235 learned_addr learned my addr 172.21.15.60:0/984170235 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:29.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.644+0000 7fd203fff700 1 --2- 172.21.15.60:0/984170235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd204107d40 0x7fd20419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:29.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.644+0000 7fd2037fe700 1 --2- 172.21.15.60:0/984170235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204101cf0 0x7fd2041980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:29.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.644+0000 7fd202ffd700 1 -- 172.21.15.60:0/984170235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd204107d40 msgr2=0x7fd20419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.644+0000 7fd202ffd700 1 --2- 172.21.15.60:0/984170235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd204107d40 0x7fd20419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.644+0000 7fd202ffd700 1 -- 172.21.15.60:0/984170235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204101cf0 msgr2=0x7fd2041980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.644+0000 7fd202ffd700 1 --2- 172.21.15.60:0/984170235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204101cf0 0x7fd2041980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.644+0000 7fd202ffd700 1 -- 172.21.15.60:0/984170235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f40095f0 con 0x7fd2041026b0 2024-02-26T04:56:29.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.644+0000 7fd202ffd700 1 --2- 172.21.15.60:0/984170235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2041026b0 0x7fd204198630 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800e9a0 tx=0x7fd1f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:29.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.644+0000 7fd200ff9700 1 -- 172.21.15.60:0/984170235 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8009790 con 0x7fd2041026b0 2024-02-26T04:56:29.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.644+0000 7fd20a074700 1 -- 172.21.15.60:0/984170235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20419d5e0 con 0x7fd2041026b0 2024-02-26T04:56:29.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.644+0000 7fd20a074700 1 -- 172.21.15.60:0/984170235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2041a4e40 con 0x7fd2041026b0 2024-02-26T04:56:29.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.644+0000 7fd200ff9700 1 -- 172.21.15.60:0/984170235 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1f8010430 con 0x7fd2041026b0 2024-02-26T04:56:29.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.644+0000 7fd200ff9700 1 -- 172.21.15.60:0/984170235 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8018680 con 0x7fd2041026b0 2024-02-26T04:56:29.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.645+0000 7fd20a074700 1 -- 172.21.15.60:0/984170235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd2041033b0 con 0x7fd2041026b0 2024-02-26T04:56:29.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.646+0000 7fd200ff9700 1 -- 172.21.15.60:0/984170235 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fd1f8010910 con 0x7fd2041026b0 2024-02-26T04:56:29.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.647+0000 7fd200ff9700 1 --2- 172.21.15.60:0/984170235 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd1ec0a2260 0x7fd1ec0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:29.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.647+0000 7fd2037fe700 1 --2- 172.21.15.60:0/984170235 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd1ec0a2260 0x7fd1ec0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:29.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.647+0000 7fd200ff9700 1 -- 172.21.15.60:0/984170235 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fd1f8014070 con 0x7fd2041026b0 2024-02-26T04:56:29.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.647+0000 7fd200ff9700 1 --2- 172.21.15.60:0/984170235 >> [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798] conn(0x7fd1ec0ad3c0 0x7fd1ec0af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:29.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.647+0000 7fd200ff9700 1 -- 172.21.15.60:0/984170235 --> [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1ec0aff30 con 0x7fd1ec0ad3c0 2024-02-26T04:56:29.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.647+0000 7fd200ff9700 1 -- 172.21.15.60:0/984170235 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1f80ced60 con 0x7fd2041026b0 2024-02-26T04:56:29.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.647+0000 7fd2037fe700 1 --2- 172.21.15.60:0/984170235 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd1ec0a2260 0x7fd1ec0a4720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400b2e0 tx=0x7fd1f4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:29.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.648+0000 7fd203fff700 1 --2- 172.21.15.60:0/984170235 >> [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798] conn(0x7fd1ec0ad3c0 0x7fd1ec0af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:29.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.648+0000 7fd203fff700 1 --2- 172.21.15.60:0/984170235 >> [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798] conn(0x7fd1ec0ad3c0 0x7fd1ec0af860 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:29.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.649+0000 7fd200ff9700 1 -- 172.21.15.60:0/984170235 <== osd.10 v2:172.21.15.156:6824/3738247798 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd1ec0aff30 con 0x7fd1ec0ad3c0 2024-02-26T04:56:29.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.665+0000 7fd20a074700 1 -- 172.21.15.60:0/984170235 --> [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd204112030 con 0x7fd1ec0ad3c0 2024-02-26T04:56:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.666+0000 7fd200ff9700 1 -- 172.21.15.60:0/984170235 <== osd.10 v2:172.21.15.156:6824/3738247798 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd204112030 con 0x7fd1ec0ad3c0 2024-02-26T04:56:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.666+0000 7fd20a074700 1 -- 172.21.15.60:0/984170235 >> [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798] conn(0x7fd1ec0ad3c0 msgr2=0x7fd1ec0af860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.666+0000 7fd20a074700 1 --2- 172.21.15.60:0/984170235 >> [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798] conn(0x7fd1ec0ad3c0 0x7fd1ec0af860 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.666+0000 7fd20a074700 1 -- 172.21.15.60:0/984170235 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd1ec0a2260 msgr2=0x7fd1ec0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.666+0000 7fd20a074700 1 --2- 172.21.15.60:0/984170235 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd1ec0a2260 0x7fd1ec0a4720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400b2e0 tx=0x7fd1f4005960 comp rx=0 tx=0).stop 2024-02-26T04:56:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.666+0000 7fd20a074700 1 -- 172.21.15.60:0/984170235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2041026b0 msgr2=0x7fd204198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.666+0000 7fd20a074700 1 --2- 172.21.15.60:0/984170235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2041026b0 0x7fd204198630 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800e9a0 tx=0x7fd1f800ecb0 comp rx=0 tx=0).stop 2024-02-26T04:56:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.666+0000 7fd203fff700 1 -- 172.21.15.60:0/984170235 reap_dead start 2024-02-26T04:56:29.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.666+0000 7fd20a074700 1 -- 172.21.15.60:0/984170235 shutdown_connections 2024-02-26T04:56:29.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.666+0000 7fd20a074700 1 -- 172.21.15.60:0/984170235 >> 172.21.15.60:0/984170235 conn(0x7fd204078dc0 msgr2=0x7fd2041008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:29.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.666+0000 7fd20a074700 1 -- 172.21.15.60:0/984170235 shutdown_connections 2024-02-26T04:56:29.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:29.666+0000 7fd20a074700 1 -- 172.21.15.60:0/984170235 wait complete. 2024-02-26T04:56:30.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:30 smithi060 ceph-mon[32382]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 336 B/s wr, 7 op/s 2024-02-26T04:56:30.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:30 smithi191 ceph-mon[36352]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 336 B/s wr, 7 op/s 2024-02-26T04:56:30.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:30 smithi156 ceph-mon[35805]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 336 B/s wr, 7 op/s 2024-02-26T04:56:31.107 INFO:teuthology.orchestra.run.smithi060.stdout:141733920829 2024-02-26T04:56:31.107 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-26T04:56:32.318 INFO:teuthology.orchestra.run.smithi060.stdout:77309411400 2024-02-26T04:56:32.319 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-26T04:56:32.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:32 smithi060 ceph-mon[32382]: pgmap v327: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 303 B/s wr, 6 op/s 2024-02-26T04:56:32.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:32 smithi191 ceph-mon[36352]: pgmap v327: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 303 B/s wr, 6 op/s 2024-02-26T04:56:32.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:32 smithi156 ceph-mon[35805]: pgmap v327: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 303 B/s wr, 6 op/s 2024-02-26T04:56:32.760 INFO:teuthology.orchestra.run.smithi060.stdout:197568495667 2024-02-26T04:56:32.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-26T04:56:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 -- 172.21.15.60:0/3480132189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3806a370 msgr2=0x7faf3806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 --2- 172.21.15.60:0/3480132189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3806a370 0x7faf3806a750 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faf34009990 tx=0x7faf34009ca0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 -- 172.21.15.60:0/3480132189 shutdown_connections 2024-02-26T04:56:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 --2- 172.21.15.60:0/3480132189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf3810e750 0x7faf38110b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 --2- 172.21.15.60:0/3480132189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3806a370 0x7faf3806a750 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 --2- 172.21.15.60:0/3480132189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3806ac90 0x7faf3810e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 -- 172.21.15.60:0/3480132189 >> 172.21.15.60:0/3480132189 conn(0x7faf38076b90 msgr2=0x7faf38076fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 -- 172.21.15.60:0/3480132189 shutdown_connections 2024-02-26T04:56:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 -- 172.21.15.60:0/3480132189 wait complete. 2024-02-26T04:56:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 Processor -- start 2024-02-26T04:56:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 -- start start 2024-02-26T04:56:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3806a370 0x7faf3819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf3806ac90 0x7faf3819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3810e750 0x7faf3819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3819f2a0 con 0x7faf3806a370 2024-02-26T04:56:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3819f410 con 0x7faf3810e750 2024-02-26T04:56:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.126+0000 7faf40415700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3819f5d0 con 0x7faf3806ac90 2024-02-26T04:56:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf3e1b1700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3806a370 0x7faf3819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf3e1b1700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3806a370 0x7faf3819a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53240/0 (socket says 172.21.15.60:53240) 2024-02-26T04:56:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf3e1b1700 1 -- 172.21.15.60:0/3359509365 learned_addr learned my addr 172.21.15.60:0/3359509365 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf3e9b2700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3810e750 0x7faf3819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf3d9b0700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf3806ac90 0x7faf3819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf3e1b1700 1 -- 172.21.15.60:0/3359509365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf3806ac90 msgr2=0x7faf3819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf3e1b1700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf3806ac90 0x7faf3819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf3e1b1700 1 -- 172.21.15.60:0/3359509365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3810e750 msgr2=0x7faf3819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf3e1b1700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3810e750 0x7faf3819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf3e1b1700 1 -- 172.21.15.60:0/3359509365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf340095f0 con 0x7faf3806a370 2024-02-26T04:56:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf3e9b2700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf3810e750 0x7faf3819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:56:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf3d9b0700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf3806ac90 0x7faf3819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:56:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf3e1b1700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3806a370 0x7faf3819a410 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faf340098e0 tx=0x7faf3400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf2f7fe700 1 -- 172.21.15.60:0/3359509365 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3401d070 con 0x7faf3806a370 2024-02-26T04:56:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf2f7fe700 1 -- 172.21.15.60:0/3359509365 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf34003680 con 0x7faf3806a370 2024-02-26T04:56:33.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf40415700 1 -- 172.21.15.60:0/3359509365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3819f850 con 0x7faf3806a370 2024-02-26T04:56:33.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf2f7fe700 1 -- 172.21.15.60:0/3359509365 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34017650 con 0x7faf3806a370 2024-02-26T04:56:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.127+0000 7faf40415700 1 -- 172.21.15.60:0/3359509365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf381a7110 con 0x7faf3806a370 2024-02-26T04:56:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.128+0000 7faf40415700 1 -- 172.21.15.60:0/3359509365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf381141d0 con 0x7faf3806a370 2024-02-26T04:56:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.130+0000 7faf2f7fe700 1 -- 172.21.15.60:0/3359509365 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7faf34022470 con 0x7faf3806a370 2024-02-26T04:56:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.130+0000 7faf2f7fe700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7faf240a21e0 0x7faf240a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.130+0000 7faf2f7fe700 1 -- 172.21.15.60:0/3359509365 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7faf340cfa90 con 0x7faf3806a370 2024-02-26T04:56:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.130+0000 7faf3d9b0700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7faf240a21e0 0x7faf240a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.131+0000 7faf2f7fe700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354] conn(0x7faf240ad340 0x7faf240af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.131+0000 7faf2f7fe700 1 -- 172.21.15.60:0/3359509365 --> [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf240afeb0 con 0x7faf240ad340 2024-02-26T04:56:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.131+0000 7faf2f7fe700 1 -- 172.21.15.60:0/3359509365 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf340cfe00 con 0x7faf3806a370 2024-02-26T04:56:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.131+0000 7faf3e9b2700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354] conn(0x7faf240ad340 0x7faf240af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.131+0000 7faf3d9b0700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7faf240a21e0 0x7faf240a46a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf28005fd0 tx=0x7faf28005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.131+0000 7faf3e9b2700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354] conn(0x7faf240ad340 0x7faf240af7e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.132+0000 7faf2f7fe700 1 -- 172.21.15.60:0/3359509365 <== osd.11 v2:172.21.15.191:6824/277456354 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7faf240afeb0 con 0x7faf240ad340 2024-02-26T04:56:33.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.157+0000 7faf40415700 1 -- 172.21.15.60:0/3359509365 --> [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faf38062380 con 0x7faf240ad340 2024-02-26T04:56:33.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.157+0000 7faf2f7fe700 1 -- 172.21.15.60:0/3359509365 <== osd.11 v2:172.21.15.191:6824/277456354 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7faf38062380 con 0x7faf240ad340 2024-02-26T04:56:33.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.158+0000 7faf40415700 1 -- 172.21.15.60:0/3359509365 >> [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354] conn(0x7faf240ad340 msgr2=0x7faf240af7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.158+0000 7faf40415700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354] conn(0x7faf240ad340 0x7faf240af7e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.158+0000 7faf40415700 1 -- 172.21.15.60:0/3359509365 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7faf240a21e0 msgr2=0x7faf240a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.158+0000 7faf40415700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7faf240a21e0 0x7faf240a46a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf28005fd0 tx=0x7faf28005ee0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.158+0000 7faf40415700 1 -- 172.21.15.60:0/3359509365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3806a370 msgr2=0x7faf3819a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.158+0000 7faf40415700 1 --2- 172.21.15.60:0/3359509365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3806a370 0x7faf3819a410 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faf340098e0 tx=0x7faf3400f690 comp rx=0 tx=0).stop 2024-02-26T04:56:33.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.158+0000 7faf3e9b2700 1 -- 172.21.15.60:0/3359509365 reap_dead start 2024-02-26T04:56:33.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.158+0000 7faf40415700 1 -- 172.21.15.60:0/3359509365 shutdown_connections 2024-02-26T04:56:33.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.158+0000 7faf40415700 1 -- 172.21.15.60:0/3359509365 >> 172.21.15.60:0/3359509365 conn(0x7faf38076b90 msgr2=0x7faf3810fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:33.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.158+0000 7faf40415700 1 -- 172.21.15.60:0/3359509365 shutdown_connections 2024-02-26T04:56:33.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.158+0000 7faf40415700 1 -- 172.21.15.60:0/3359509365 wait complete. 2024-02-26T04:56:33.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.456+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3425257406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d8101530 msgr2=0x7ff7d810d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.456+0000 7ff7ddf04700 1 --2- 172.21.15.60:0/3425257406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d8101530 0x7ff7d810d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000b0a0 tx=0x7ff7d000b3b0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.456+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3425257406 shutdown_connections 2024-02-26T04:56:33.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.456+0000 7ff7ddf04700 1 --2- 172.21.15.60:0/3425257406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d8101530 0x7ff7d810d9c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.456+0000 7ff7ddf04700 1 --2- 172.21.15.60:0/3425257406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7d8106b80 0x7ff7d8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.456+0000 7ff7ddf04700 1 --2- 172.21.15.60:0/3425257406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d8100b70 0x7ff7d8100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.456+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3425257406 >> 172.21.15.60:0/3425257406 conn(0x7ff7d8077aa0 msgr2=0x7ff7d8077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.456+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3425257406 shutdown_connections 2024-02-26T04:56:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.456+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3425257406 wait complete. 2024-02-26T04:56:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7ddf04700 1 Processor -- start 2024-02-26T04:56:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7ddf04700 1 -- start start 2024-02-26T04:56:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7ddf04700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7d8100b70 0x7ff7d81027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7ddf04700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d8101530 0x7ff7d8102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7ddf04700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d8106b80 0x7ff7d8103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7ddf04700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d81a6580 con 0x7ff7d8101530 2024-02-26T04:56:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7ddf04700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d81a66f0 con 0x7ff7d8100b70 2024-02-26T04:56:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7ddf04700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d81a68b0 con 0x7ff7d8106b80 2024-02-26T04:56:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7d6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d8101530 0x7ff7d8102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7d6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d8101530 0x7ff7d8102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53260/0 (socket says 172.21.15.60:53260) 2024-02-26T04:56:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7d6ffd700 1 -- 172.21.15.60:0/3376523944 learned_addr learned my addr 172.21.15.60:0/3376523944 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7d7fff700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d8106b80 0x7ff7d8103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7d77fe700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7d8100b70 0x7ff7d81027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7d6ffd700 1 -- 172.21.15.60:0/3376523944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d8106b80 msgr2=0x7ff7d8103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7d6ffd700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d8106b80 0x7ff7d8103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7d6ffd700 1 -- 172.21.15.60:0/3376523944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7d8100b70 msgr2=0x7ff7d81027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7d6ffd700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7d8100b70 0x7ff7d81027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.457+0000 7ff7d6ffd700 1 -- 172.21.15.60:0/3376523944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d0009d00 con 0x7ff7d8101530 2024-02-26T04:56:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.458+0000 7ff7d77fe700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7d8100b70 0x7ff7d81027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:56:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.458+0000 7ff7d6ffd700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d8101530 0x7ff7d8102cf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00c920 tx=0x7ff7cc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.458+0000 7ff7d4ff9700 1 -- 172.21.15.60:0/3376523944 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc007b00 con 0x7ff7d8101530 2024-02-26T04:56:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.458+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3376523944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d81a6b90 con 0x7ff7d8101530 2024-02-26T04:56:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.458+0000 7ff7d4ff9700 1 -- 172.21.15.60:0/3376523944 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7cc004d10 con 0x7ff7d8101530 2024-02-26T04:56:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.458+0000 7ff7d4ff9700 1 -- 172.21.15.60:0/3376523944 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc0055c0 con 0x7ff7d8101530 2024-02-26T04:56:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.458+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3376523944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d81a70e0 con 0x7ff7d8101530 2024-02-26T04:56:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.459+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3376523944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff7d8110e90 con 0x7ff7d8101530 2024-02-26T04:56:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.460+0000 7ff7d4ff9700 1 -- 172.21.15.60:0/3376523944 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ff7cc004750 con 0x7ff7d8101530 2024-02-26T04:56:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.460+0000 7ff7d4ff9700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff7c00a21e0 0x7ff7c00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.461+0000 7ff7d77fe700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff7c00a21e0 0x7ff7c00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.461+0000 7ff7d4ff9700 1 -- 172.21.15.60:0/3376523944 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ff7cc0ce1d0 con 0x7ff7d8101530 2024-02-26T04:56:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.461+0000 7ff7d4ff9700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146] conn(0x7ff7c00ad340 0x7ff7c00af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.461+0000 7ff7d4ff9700 1 -- 172.21.15.60:0/3376523944 --> [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7c00afeb0 con 0x7ff7c00ad340 2024-02-26T04:56:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.461+0000 7ff7d4ff9700 1 -- 172.21.15.60:0/3376523944 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7cc0ce5e0 con 0x7ff7d8101530 2024-02-26T04:56:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.461+0000 7ff7d7fff700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146] conn(0x7ff7c00ad340 0x7ff7c00af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.461+0000 7ff7d77fe700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff7c00a21e0 0x7ff7c00a46a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8009b50 tx=0x7ff7c8009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.461+0000 7ff7d7fff700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146] conn(0x7ff7c00ad340 0x7ff7c00af7e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:33.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.462+0000 7ff7d4ff9700 1 -- 172.21.15.60:0/3376523944 <== osd.3 v2:172.21.15.60:6810/3123201146 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff7c00afeb0 con 0x7ff7c00ad340 2024-02-26T04:56:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.479+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3376523944 --> [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff7d8104b20 con 0x7ff7c00ad340 2024-02-26T04:56:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.479+0000 7ff7d4ff9700 1 -- 172.21.15.60:0/3376523944 <== osd.3 v2:172.21.15.60:6810/3123201146 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff7d8104b20 con 0x7ff7c00ad340 2024-02-26T04:56:33.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.479+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3376523944 >> [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146] conn(0x7ff7c00ad340 msgr2=0x7ff7c00af7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.479+0000 7ff7ddf04700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146] conn(0x7ff7c00ad340 0x7ff7c00af7e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.479+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3376523944 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff7c00a21e0 msgr2=0x7ff7c00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.479+0000 7ff7ddf04700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff7c00a21e0 0x7ff7c00a46a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8009b50 tx=0x7ff7c8009550 comp rx=0 tx=0).stop 2024-02-26T04:56:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.479+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3376523944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d8101530 msgr2=0x7ff7d8102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.479+0000 7ff7ddf04700 1 --2- 172.21.15.60:0/3376523944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d8101530 0x7ff7d8102cf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00c920 tx=0x7ff7cc00cce0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.479+0000 7ff7d7fff700 1 -- 172.21.15.60:0/3376523944 reap_dead start 2024-02-26T04:56:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.479+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3376523944 shutdown_connections 2024-02-26T04:56:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.479+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3376523944 >> 172.21.15.60:0/3376523944 conn(0x7ff7d8077aa0 msgr2=0x7ff7d80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.480+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3376523944 shutdown_connections 2024-02-26T04:56:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.480+0000 7ff7ddf04700 1 -- 172.21.15.60:0/3376523944 wait complete. 2024-02-26T04:56:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.666+0000 7f1b54888700 1 -- 172.21.15.60:0/2742194384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b50100ba0 msgr2=0x7f1b50101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.666+0000 7f1b54888700 1 --2- 172.21.15.60:0/2742194384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b50100ba0 0x7f1b50101020 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009990 tx=0x7f1b38009ca0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.666+0000 7f1b54888700 1 -- 172.21.15.60:0/2742194384 shutdown_connections 2024-02-26T04:56:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.666+0000 7f1b54888700 1 --2- 172.21.15.60:0/2742194384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b50101560 0x7f1b5010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.666+0000 7f1b54888700 1 --2- 172.21.15.60:0/2742194384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b50106bd0 0x7f1b50106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.666+0000 7f1b54888700 1 --2- 172.21.15.60:0/2742194384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b50100ba0 0x7f1b50101020 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.666+0000 7f1b54888700 1 -- 172.21.15.60:0/2742194384 >> 172.21.15.60:0/2742194384 conn(0x7f1b50076bc0 msgr2=0x7f1b50076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.666+0000 7f1b54888700 1 -- 172.21.15.60:0/2742194384 shutdown_connections 2024-02-26T04:56:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.666+0000 7f1b54888700 1 -- 172.21.15.60:0/2742194384 wait complete. 2024-02-26T04:56:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b54888700 1 Processor -- start 2024-02-26T04:56:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b54888700 1 -- start start 2024-02-26T04:56:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b54888700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b50100ba0 0x7f1b501980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b54888700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b50101560 0x7f1b50198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b54888700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b50106bd0 0x7f1b5019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b54888700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5019cfd0 con 0x7f1b50106bd0 2024-02-26T04:56:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b54888700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5019d140 con 0x7f1b50100ba0 2024-02-26T04:56:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b54888700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5019d300 con 0x7f1b50101560 2024-02-26T04:56:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b4ed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b50106bd0 0x7f1b5019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b4ed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b50106bd0 0x7f1b5019c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53290/0 (socket says 172.21.15.60:53290) 2024-02-26T04:56:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b4ed9d700 1 -- 172.21.15.60:0/1143578046 learned_addr learned my addr 172.21.15.60:0/1143578046 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b4e59c700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b50100ba0 0x7f1b501980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b4dd9b700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b50101560 0x7f1b50198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b4ed9d700 1 -- 172.21.15.60:0/1143578046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b50101560 msgr2=0x7f1b50198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.667+0000 7f1b4ed9d700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b50101560 0x7f1b50198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.668+0000 7f1b4ed9d700 1 -- 172.21.15.60:0/1143578046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b50100ba0 msgr2=0x7f1b501980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.668+0000 7f1b4ed9d700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b50100ba0 0x7f1b501980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.668+0000 7f1b4ed9d700 1 -- 172.21.15.60:0/1143578046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b380095f0 con 0x7f1b50106bd0 2024-02-26T04:56:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.668+0000 7f1b4dd9b700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b50101560 0x7f1b50198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:56:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.668+0000 7f1b4ed9d700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b50106bd0 0x7f1b5019c9c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000c900 tx=0x7f1b4000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.668+0000 7f1b477fe700 1 -- 172.21.15.60:0/1143578046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b40007b00 con 0x7f1b50106bd0 2024-02-26T04:56:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.668+0000 7f1b54888700 1 -- 172.21.15.60:0/1143578046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5019d5e0 con 0x7f1b50106bd0 2024-02-26T04:56:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.668+0000 7f1b477fe700 1 -- 172.21.15.60:0/1143578046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b40007c60 con 0x7f1b50106bd0 2024-02-26T04:56:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.668+0000 7f1b477fe700 1 -- 172.21.15.60:0/1143578046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b40018660 con 0x7f1b50106bd0 2024-02-26T04:56:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.668+0000 7f1b54888700 1 -- 172.21.15.60:0/1143578046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b501a4e40 con 0x7f1b50106bd0 2024-02-26T04:56:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.669+0000 7f1b54888700 1 -- 172.21.15.60:0/1143578046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1b50110ea0 con 0x7f1b50106bd0 2024-02-26T04:56:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.670+0000 7f1b477fe700 1 -- 172.21.15.60:0/1143578046 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f1b4000f450 con 0x7f1b50106bd0 2024-02-26T04:56:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.671+0000 7f1b477fe700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1b3c0a21e0 0x7f1b3c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.671+0000 7f1b477fe700 1 -- 172.21.15.60:0/1143578046 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f1b400ce5c0 con 0x7f1b50106bd0 2024-02-26T04:56:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.671+0000 7f1b4e59c700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1b3c0a21e0 0x7f1b3c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.671+0000 7f1b477fe700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343] conn(0x7f1b3c0ad340 0x7f1b3c0af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.671+0000 7f1b477fe700 1 -- 172.21.15.60:0/1143578046 --> [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b3c0afeb0 con 0x7f1b3c0ad340 2024-02-26T04:56:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.671+0000 7f1b477fe700 1 -- 172.21.15.60:0/1143578046 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b400ce9d0 con 0x7f1b50106bd0 2024-02-26T04:56:33.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.671+0000 7f1b4dd9b700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343] conn(0x7f1b3c0ad340 0x7f1b3c0af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.671+0000 7f1b4e59c700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1b3c0a21e0 0x7f1b3c0a46a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b480076e0 tx=0x7f1b48007670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:33.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.671+0000 7f1b4dd9b700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343] conn(0x7f1b3c0ad340 0x7f1b3c0af7e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:33.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.672+0000 7f1b477fe700 1 -- 172.21.15.60:0/1143578046 <== osd.9 v2:172.21.15.60:6826/1562445343 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1b3c0afeb0 con 0x7f1b3c0ad340 2024-02-26T04:56:33.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.689+0000 7f1b54888700 1 -- 172.21.15.60:0/1143578046 --> [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1b50104ba0 con 0x7f1b3c0ad340 2024-02-26T04:56:33.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.689+0000 7f1b477fe700 1 -- 172.21.15.60:0/1143578046 <== osd.9 v2:172.21.15.60:6826/1562445343 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1b50104ba0 con 0x7f1b3c0ad340 2024-02-26T04:56:33.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.690+0000 7f1b54888700 1 -- 172.21.15.60:0/1143578046 >> [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343] conn(0x7f1b3c0ad340 msgr2=0x7f1b3c0af7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.690+0000 7f1b54888700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343] conn(0x7f1b3c0ad340 0x7f1b3c0af7e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.690+0000 7f1b54888700 1 -- 172.21.15.60:0/1143578046 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1b3c0a21e0 msgr2=0x7f1b3c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.690+0000 7f1b54888700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1b3c0a21e0 0x7f1b3c0a46a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b480076e0 tx=0x7f1b48007670 comp rx=0 tx=0).stop 2024-02-26T04:56:33.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.690+0000 7f1b54888700 1 -- 172.21.15.60:0/1143578046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b50106bd0 msgr2=0x7f1b5019c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.690+0000 7f1b54888700 1 --2- 172.21.15.60:0/1143578046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b50106bd0 0x7f1b5019c9c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000c900 tx=0x7f1b4000ccc0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.690+0000 7f1b4ed9d700 1 -- 172.21.15.60:0/1143578046 reap_dead start 2024-02-26T04:56:33.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.690+0000 7f1b54888700 1 -- 172.21.15.60:0/1143578046 shutdown_connections 2024-02-26T04:56:33.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.690+0000 7f1b54888700 1 -- 172.21.15.60:0/1143578046 >> 172.21.15.60:0/1143578046 conn(0x7f1b50076bc0 msgr2=0x7f1b500ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:33.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.690+0000 7f1b54888700 1 -- 172.21.15.60:0/1143578046 shutdown_connections 2024-02-26T04:56:33.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.690+0000 7f1b54888700 1 -- 172.21.15.60:0/1143578046 wait complete. 2024-02-26T04:56:33.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.909+0000 7f105d0f0700 1 -- 172.21.15.60:0/1776302555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1058105ef0 msgr2=0x7f105810dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.909+0000 7f105d0f0700 1 --2- 172.21.15.60:0/1776302555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1058105ef0 0x7f105810dac0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1050009a10 tx=0x7f1050009d20 comp rx=0 tx=0).stop 2024-02-26T04:56:33.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.909+0000 7f105d0f0700 1 -- 172.21.15.60:0/1776302555 shutdown_connections 2024-02-26T04:56:33.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.909+0000 7f105d0f0700 1 --2- 172.21.15.60:0/1776302555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1058105ef0 0x7f105810dac0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.909+0000 7f105d0f0700 1 --2- 172.21.15.60:0/1776302555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10581015d0 0x7f10581019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.909+0000 7f105d0f0700 1 --2- 172.21.15.60:0/1776302555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1058101f80 0x7f1058105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.909+0000 7f105d0f0700 1 -- 172.21.15.60:0/1776302555 >> 172.21.15.60:0/1776302555 conn(0x7f1058077170 msgr2=0x7f10580fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:33.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.909+0000 7f105d0f0700 1 -- 172.21.15.60:0/1776302555 shutdown_connections 2024-02-26T04:56:33.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.909+0000 7f105d0f0700 1 -- 172.21.15.60:0/1776302555 wait complete. 2024-02-26T04:56:33.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.910+0000 7f105d0f0700 1 Processor -- start 2024-02-26T04:56:33.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.910+0000 7f105d0f0700 1 -- start start 2024-02-26T04:56:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.910+0000 7f105d0f0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10581015d0 0x7f1058195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.910+0000 7f105d0f0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1058101f80 0x7f1058196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.910+0000 7f105d0f0700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1058105ef0 0x7f105819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.910+0000 7f105d0f0700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105819ae50 con 0x7f1058105ef0 2024-02-26T04:56:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.910+0000 7f105d0f0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105819afc0 con 0x7f10581015d0 2024-02-26T04:56:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.910+0000 7f105d0f0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105819b180 con 0x7f1058101f80 2024-02-26T04:56:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.910+0000 7f105759e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1058105ef0 0x7f105819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.910+0000 7f105759e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1058105ef0 0x7f105819a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53300/0 (socket says 172.21.15.60:53300) 2024-02-26T04:56:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.910+0000 7f105759e700 1 -- 172.21.15.60:0/2940347768 learned_addr learned my addr 172.21.15.60:0/2940347768 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.910+0000 7f1056d9d700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10581015d0 0x7f1058195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.911+0000 7f105659c700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1058101f80 0x7f1058196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.911+0000 7f105759e700 1 -- 172.21.15.60:0/2940347768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1058101f80 msgr2=0x7f1058196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.911+0000 7f105759e700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1058101f80 0x7f1058196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.911+0000 7f105759e700 1 -- 172.21.15.60:0/2940347768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10581015d0 msgr2=0x7f1058195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.911+0000 7f105759e700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10581015d0 0x7f1058195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.911+0000 7f105759e700 1 -- 172.21.15.60:0/2940347768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10500095f0 con 0x7f1058105ef0 2024-02-26T04:56:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.911+0000 7f105659c700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1058101f80 0x7f1058196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:56:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.911+0000 7f105759e700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1058105ef0 0x7f105819a840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1050009960 tx=0x7f1050003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.911+0000 7f104ffff700 1 -- 172.21.15.60:0/2940347768 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105000bb60 con 0x7f1058105ef0 2024-02-26T04:56:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.911+0000 7f105d0f0700 1 -- 172.21.15.60:0/2940347768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105819b400 con 0x7f1058105ef0 2024-02-26T04:56:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.911+0000 7f104ffff700 1 -- 172.21.15.60:0/2940347768 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1050004640 con 0x7f1058105ef0 2024-02-26T04:56:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.911+0000 7f104ffff700 1 -- 172.21.15.60:0/2940347768 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050016690 con 0x7f1058105ef0 2024-02-26T04:56:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.911+0000 7f105d0f0700 1 -- 172.21.15.60:0/2940347768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10581a2cc0 con 0x7f1058105ef0 2024-02-26T04:56:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.912+0000 7f105d0f0700 1 -- 172.21.15.60:0/2940347768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f105810fe80 con 0x7f1058105ef0 2024-02-26T04:56:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.913+0000 7f104ffff700 1 -- 172.21.15.60:0/2940347768 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f105000e4e0 con 0x7f1058105ef0 2024-02-26T04:56:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.914+0000 7f104ffff700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f10440a21e0 0x7f10440a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.914+0000 7f1056d9d700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f10440a21e0 0x7f10440a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.914+0000 7f104ffff700 1 -- 172.21.15.60:0/2940347768 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f10500d1d40 con 0x7f1058105ef0 2024-02-26T04:56:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.914+0000 7f104ffff700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461] conn(0x7f10440ad340 0x7f10440af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.914+0000 7f104ffff700 1 -- 172.21.15.60:0/2940347768 --> [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f10440afeb0 con 0x7f10440ad340 2024-02-26T04:56:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.914+0000 7f104ffff700 1 -- 172.21.15.60:0/2940347768 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10500d2150 con 0x7f1058105ef0 2024-02-26T04:56:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.914+0000 7f1056d9d700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f10440a21e0 0x7f10440a46a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1048009550 tx=0x7f10480091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.915+0000 7f105659c700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461] conn(0x7f10440ad340 0x7f10440af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.915+0000 7f105659c700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461] conn(0x7f10440ad340 0x7f10440af7e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:33.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.916+0000 7f104ffff700 1 -- 172.21.15.60:0/2940347768 <== osd.8 v2:172.21.15.191:6816/3417694461 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f10440afeb0 con 0x7f10440ad340 2024-02-26T04:56:33.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.933+0000 7f105d0f0700 1 -- 172.21.15.60:0/2940347768 --> [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1058062380 con 0x7f10440ad340 2024-02-26T04:56:33.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.933+0000 7f104ffff700 1 -- 172.21.15.60:0/2940347768 <== osd.8 v2:172.21.15.191:6816/3417694461 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1058062380 con 0x7f10440ad340 2024-02-26T04:56:33.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.933+0000 7f105d0f0700 1 -- 172.21.15.60:0/2940347768 >> [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461] conn(0x7f10440ad340 msgr2=0x7f10440af7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.933+0000 7f105d0f0700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461] conn(0x7f10440ad340 0x7f10440af7e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.933+0000 7f105d0f0700 1 -- 172.21.15.60:0/2940347768 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f10440a21e0 msgr2=0x7f10440a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.933+0000 7f105d0f0700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f10440a21e0 0x7f10440a46a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1048009550 tx=0x7f10480091f0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.933+0000 7f105d0f0700 1 -- 172.21.15.60:0/2940347768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1058105ef0 msgr2=0x7f105819a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:33.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.933+0000 7f105d0f0700 1 --2- 172.21.15.60:0/2940347768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1058105ef0 0x7f105819a840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1050009960 tx=0x7f1050003af0 comp rx=0 tx=0).stop 2024-02-26T04:56:33.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.933+0000 7f105759e700 1 -- 172.21.15.60:0/2940347768 reap_dead start 2024-02-26T04:56:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.933+0000 7f105d0f0700 1 -- 172.21.15.60:0/2940347768 shutdown_connections 2024-02-26T04:56:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.933+0000 7f105d0f0700 1 -- 172.21.15.60:0/2940347768 >> 172.21.15.60:0/2940347768 conn(0x7f1058077170 msgr2=0x7f10581045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.934+0000 7f105d0f0700 1 -- 172.21.15.60:0/2940347768 shutdown_connections 2024-02-26T04:56:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:33.934+0000 7f105d0f0700 1 -- 172.21.15.60:0/2940347768 wait complete. 2024-02-26T04:56:34.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:34 smithi060 ceph-mon[32382]: pgmap v328: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-26T04:56:34.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:34 smithi191 ceph-mon[36352]: pgmap v328: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-26T04:56:34.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:34 smithi156 ceph-mon[35805]: pgmap v328: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-26T04:56:34.967 INFO:teuthology.orchestra.run.smithi060.stdout:107374182468 2024-02-26T04:56:34.968 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-26T04:56:35.471 INFO:teuthology.orchestra.run.smithi060.stdout:287762808869 2024-02-26T04:56:35.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-02-26T04:56:36.119 INFO:teuthology.orchestra.run.smithi060.stdout:343597383706 2024-02-26T04:56:36.120 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-02-26T04:56:36.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:36 smithi060 ceph-mon[32382]: pgmap v329: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T04:56:36.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.371+0000 7fb7e0b62700 1 -- 172.21.15.60:0/793389924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7dc106b80 msgr2=0x7fb7dc106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:36.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.371+0000 7fb7e0b62700 1 --2- 172.21.15.60:0/793389924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7dc106b80 0x7fb7dc106f60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009990 tx=0x7fb7cc009ca0 comp rx=0 tx=0).stop 2024-02-26T04:56:36.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.371+0000 7fb7e0b62700 1 -- 172.21.15.60:0/793389924 shutdown_connections 2024-02-26T04:56:36.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.371+0000 7fb7e0b62700 1 --2- 172.21.15.60:0/793389924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7dc101530 0x7fb7dc10d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:36.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.371+0000 7fb7e0b62700 1 --2- 172.21.15.60:0/793389924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7dc106b80 0x7fb7dc106f60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:36.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.371+0000 7fb7e0b62700 1 --2- 172.21.15.60:0/793389924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7dc100b70 0x7fb7dc100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.371+0000 7fb7e0b62700 1 -- 172.21.15.60:0/793389924 >> 172.21.15.60:0/793389924 conn(0x7fb7dc077aa0 msgr2=0x7fb7dc077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.371+0000 7fb7e0b62700 1 -- 172.21.15.60:0/793389924 shutdown_connections 2024-02-26T04:56:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.371+0000 7fb7e0b62700 1 -- 172.21.15.60:0/793389924 wait complete. 2024-02-26T04:56:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.371+0000 7fb7e0b62700 1 Processor -- start 2024-02-26T04:56:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7e0b62700 1 -- start start 2024-02-26T04:56:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7e0b62700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7dc100b70 0x7fb7dc1027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7e0b62700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7dc101530 0x7fb7dc102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7e0b62700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7dc106b80 0x7fb7dc103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7e0b62700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc1a6590 con 0x7fb7dc100b70 2024-02-26T04:56:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7e0b62700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc1a6700 con 0x7fb7dc106b80 2024-02-26T04:56:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7e0b62700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc1a68c0 con 0x7fb7dc101530 2024-02-26T04:56:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7da59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7dc100b70 0x7fb7dc1027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7da59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7dc100b70 0x7fb7dc1027b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53312/0 (socket says 172.21.15.60:53312) 2024-02-26T04:56:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7da59c700 1 -- 172.21.15.60:0/703180081 learned_addr learned my addr 172.21.15.60:0/703180081 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7dad9d700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7dc106b80 0x7fb7dc103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7d9d9b700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7dc101530 0x7fb7dc102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7da59c700 1 -- 172.21.15.60:0/703180081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7dc101530 msgr2=0x7fb7dc102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7da59c700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7dc101530 0x7fb7dc102cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7da59c700 1 -- 172.21.15.60:0/703180081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7dc106b80 msgr2=0x7fb7dc103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7da59c700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7dc106b80 0x7fb7dc103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7da59c700 1 -- 172.21.15.60:0/703180081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc0095f0 con 0x7fb7dc100b70 2024-02-26T04:56:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7d9d9b700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7dc101530 0x7fb7dc102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:56:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.372+0000 7fb7da59c700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7dc100b70 0x7fb7dc1027b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc005910 tx=0x7fb7cc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.373+0000 7fb7cb7fe700 1 -- 172.21.15.60:0/703180081 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc01d070 con 0x7fb7dc100b70 2024-02-26T04:56:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.373+0000 7fb7e0b62700 1 -- 172.21.15.60:0/703180081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7dc1a6b40 con 0x7fb7dc100b70 2024-02-26T04:56:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.373+0000 7fb7cb7fe700 1 -- 172.21.15.60:0/703180081 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7cc003680 con 0x7fb7dc100b70 2024-02-26T04:56:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.373+0000 7fb7cb7fe700 1 -- 172.21.15.60:0/703180081 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc017650 con 0x7fb7dc100b70 2024-02-26T04:56:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.373+0000 7fb7e0b62700 1 -- 172.21.15.60:0/703180081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7dc1a6fb0 con 0x7fb7dc100b70 2024-02-26T04:56:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.374+0000 7fb7e0b62700 1 -- 172.21.15.60:0/703180081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7dc110e60 con 0x7fb7dc100b70 2024-02-26T04:56:36.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.375+0000 7fb7cb7fe700 1 -- 172.21.15.60:0/703180081 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fb7cc022470 con 0x7fb7dc100b70 2024-02-26T04:56:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.375+0000 7fb7cb7fe700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb7c40a21e0 0x7fb7c40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.375+0000 7fb7d9d9b700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb7c40a21e0 0x7fb7c40a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.375+0000 7fb7cb7fe700 1 -- 172.21.15.60:0/703180081 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fb7cc0cfc30 con 0x7fb7dc100b70 2024-02-26T04:56:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.376+0000 7fb7cb7fe700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417] conn(0x7fb7c40ad340 0x7fb7c40af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.376+0000 7fb7cb7fe700 1 -- 172.21.15.60:0/703180081 --> [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7c40afeb0 con 0x7fb7c40ad340 2024-02-26T04:56:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.376+0000 7fb7cb7fe700 1 -- 172.21.15.60:0/703180081 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7cc0d0040 con 0x7fb7dc100b70 2024-02-26T04:56:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.376+0000 7fb7d9d9b700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb7c40a21e0 0x7fb7c40a46a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc078030 tx=0x7fb7d0005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.376+0000 7fb7dad9d700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417] conn(0x7fb7c40ad340 0x7fb7c40af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.376+0000 7fb7dad9d700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417] conn(0x7fb7c40ad340 0x7fb7c40af7e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:36.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.377+0000 7fb7cb7fe700 1 -- 172.21.15.60:0/703180081 <== osd.5 v2:172.21.15.191:6808/1681496417 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb7c40afeb0 con 0x7fb7c40ad340 2024-02-26T04:56:36.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.395+0000 7fb7e0b62700 1 -- 172.21.15.60:0/703180081 --> [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb7dc104bd0 con 0x7fb7c40ad340 2024-02-26T04:56:36.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.396+0000 7fb7cb7fe700 1 -- 172.21.15.60:0/703180081 <== osd.5 v2:172.21.15.191:6808/1681496417 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb7dc104bd0 con 0x7fb7c40ad340 2024-02-26T04:56:36.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.396+0000 7fb7e0b62700 1 -- 172.21.15.60:0/703180081 >> [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417] conn(0x7fb7c40ad340 msgr2=0x7fb7c40af7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:36.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.396+0000 7fb7e0b62700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417] conn(0x7fb7c40ad340 0x7fb7c40af7e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:36.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.397+0000 7fb7e0b62700 1 -- 172.21.15.60:0/703180081 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb7c40a21e0 msgr2=0x7fb7c40a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:36.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.397+0000 7fb7e0b62700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb7c40a21e0 0x7fb7c40a46a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc078030 tx=0x7fb7d0005f50 comp rx=0 tx=0).stop 2024-02-26T04:56:36.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.397+0000 7fb7e0b62700 1 -- 172.21.15.60:0/703180081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7dc100b70 msgr2=0x7fb7dc1027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:36.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.397+0000 7fb7e0b62700 1 --2- 172.21.15.60:0/703180081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7dc100b70 0x7fb7dc1027b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc005910 tx=0x7fb7cc00f690 comp rx=0 tx=0).stop 2024-02-26T04:56:36.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.397+0000 7fb7dad9d700 1 -- 172.21.15.60:0/703180081 reap_dead start 2024-02-26T04:56:36.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.397+0000 7fb7e0b62700 1 -- 172.21.15.60:0/703180081 shutdown_connections 2024-02-26T04:56:36.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.397+0000 7fb7e0b62700 1 -- 172.21.15.60:0/703180081 >> 172.21.15.60:0/703180081 conn(0x7fb7dc077aa0 msgr2=0x7fb7dc0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:36.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.397+0000 7fb7e0b62700 1 -- 172.21.15.60:0/703180081 shutdown_connections 2024-02-26T04:56:36.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:36.397+0000 7fb7e0b62700 1 -- 172.21.15.60:0/703180081 wait complete. 2024-02-26T04:56:36.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:36 smithi191 ceph-mon[36352]: pgmap v329: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T04:56:36.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:36 smithi156 ceph-mon[35805]: pgmap v329: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-26T04:56:37.147 INFO:teuthology.orchestra.run.smithi060.stdout:253403070506 2024-02-26T04:56:37.147 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-02-26T04:56:37.727 INFO:teuthology.orchestra.run.smithi060.stdout:163208757306 2024-02-26T04:56:37.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-26T04:56:38.197 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:38.197 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:38.198 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:38.198 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:38.200 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:38.202 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:38.261 INFO:teuthology.orchestra.run.smithi060.stdout:309237645342 2024-02-26T04:56:38.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-02-26T04:56:38.262 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:38.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:38 smithi060 ceph-mon[32382]: pgmap v330: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T04:56:38.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:38 smithi191 ceph-mon[36352]: pgmap v330: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T04:56:38.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:38 smithi156 ceph-mon[35805]: pgmap v330: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-02-26T04:56:40.499 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:40 smithi156 ceph-mon[35805]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:56:40.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:40 smithi191 ceph-mon[36352]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:56:40.518 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:40 smithi060 ceph-mon[32382]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:56:42.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:42 smithi191 ceph-mon[36352]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:56:42.526 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:42 smithi060 ceph-mon[32382]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:56:42.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:42 smithi156 ceph-mon[35805]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:56:44.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:44 smithi060 ceph-mon[32382]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-02-26T04:56:44.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:44 smithi191 ceph-mon[36352]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-02-26T04:56:44.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:44 smithi156 ceph-mon[35805]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-02-26T04:56:45.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.768+0000 7ffb70936700 1 -- 172.21.15.60:0/1278151370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c106ce0 msgr2=0x7ffb6c1070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:45.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.768+0000 7ffb70936700 1 --2- 172.21.15.60:0/1278151370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c106ce0 0x7ffb6c1070c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009990 tx=0x7ffb5c009ca0 comp rx=0 tx=0).stop 2024-02-26T04:56:45.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.768+0000 7ffb70936700 1 -- 172.21.15.60:0/1278151370 shutdown_connections 2024-02-26T04:56:45.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.768+0000 7ffb70936700 1 --2- 172.21.15.60:0/1278151370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb6c100c20 0x7ffb6c10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:45.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.768+0000 7ffb70936700 1 --2- 172.21.15.60:0/1278151370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c106ce0 0x7ffb6c1070c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.768+0000 7ffb70936700 1 --2- 172.21.15.60:0/1278151370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb6c100260 0x7ffb6c1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.768+0000 7ffb70936700 1 -- 172.21.15.60:0/1278151370 >> 172.21.15.60:0/1278151370 conn(0x7ffb6c077170 msgr2=0x7ffb6c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.769+0000 7ffb70936700 1 -- 172.21.15.60:0/1278151370 shutdown_connections 2024-02-26T04:56:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.769+0000 7ffb70936700 1 -- 172.21.15.60:0/1278151370 wait complete. 2024-02-26T04:56:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.769+0000 7ffb70936700 1 Processor -- start 2024-02-26T04:56:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.769+0000 7ffb70936700 1 -- start start 2024-02-26T04:56:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.769+0000 7ffb70936700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c100260 0x7ffb6c101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.769+0000 7ffb70936700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb6c100c20 0x7ffb6c101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.769+0000 7ffb70936700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb6c106ce0 0x7ffb6c101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.769+0000 7ffb70936700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c1a6650 con 0x7ffb6c100260 2024-02-26T04:56:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.769+0000 7ffb70936700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c1a67c0 con 0x7ffb6c106ce0 2024-02-26T04:56:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.769+0000 7ffb70936700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c1a6980 con 0x7ffb6c100c20 2024-02-26T04:56:45.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb6a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c100260 0x7ffb6c101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb6a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c100260 0x7ffb6c101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54838/0 (socket says 172.21.15.60:54838) 2024-02-26T04:56:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb6a59c700 1 -- 172.21.15.60:0/2120141479 learned_addr learned my addr 172.21.15.60:0/2120141479 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb69d9b700 1 --2- 172.21.15.60:0/2120141479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb6c100c20 0x7ffb6c101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb6ad9d700 1 --2- 172.21.15.60:0/2120141479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb6c106ce0 0x7ffb6c101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb6a59c700 1 -- 172.21.15.60:0/2120141479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb6c100c20 msgr2=0x7ffb6c101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb6a59c700 1 --2- 172.21.15.60:0/2120141479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb6c100c20 0x7ffb6c101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb6a59c700 1 -- 172.21.15.60:0/2120141479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb6c106ce0 msgr2=0x7ffb6c101ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb6a59c700 1 --2- 172.21.15.60:0/2120141479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb6c106ce0 0x7ffb6c101ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb6a59c700 1 -- 172.21.15.60:0/2120141479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c0095f0 con 0x7ffb6c100260 2024-02-26T04:56:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb6a59c700 1 --2- 172.21.15.60:0/2120141479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c100260 0x7ffb6c101440 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c000c00 tx=0x7ffb5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb637fe700 1 -- 172.21.15.60:0/2120141479 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c01d070 con 0x7ffb6c100260 2024-02-26T04:56:45.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb70936700 1 -- 172.21.15.60:0/2120141479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c1a6c00 con 0x7ffb6c100260 2024-02-26T04:56:45.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb637fe700 1 -- 172.21.15.60:0/2120141479 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb5c022470 con 0x7ffb6c100260 2024-02-26T04:56:45.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.770+0000 7ffb637fe700 1 -- 172.21.15.60:0/2120141479 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c017650 con 0x7ffb6c100260 2024-02-26T04:56:45.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.771+0000 7ffb70936700 1 -- 172.21.15.60:0/2120141479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c1a7070 con 0x7ffb6c100260 2024-02-26T04:56:45.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.771+0000 7ffb70936700 1 -- 172.21.15.60:0/2120141479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c103580 con 0x7ffb6c100260 2024-02-26T04:56:45.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.773+0000 7ffb637fe700 1 -- 172.21.15.60:0/2120141479 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ffb5c0225e0 con 0x7ffb6c100260 2024-02-26T04:56:45.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.773+0000 7ffb637fe700 1 --2- 172.21.15.60:0/2120141479 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffb580a21e0 0x7ffb580a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:45.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.774+0000 7ffb69d9b700 1 --2- 172.21.15.60:0/2120141479 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffb580a21e0 0x7ffb580a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:45.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.774+0000 7ffb637fe700 1 -- 172.21.15.60:0/2120141479 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ffb5c0cf820 con 0x7ffb6c100260 2024-02-26T04:56:45.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.774+0000 7ffb69d9b700 1 --2- 172.21.15.60:0/2120141479 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffb580a21e0 0x7ffb580a46a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0ff760 tx=0x7ffb5400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:45.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.778+0000 7ffb637fe700 1 -- 172.21.15.60:0/2120141479 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffb5c0983c0 con 0x7ffb6c100260 2024-02-26T04:56:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.989+0000 7ffb70936700 1 -- 172.21.15.60:0/2120141479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffb6c111e40 con 0x7ffb6c100260 2024-02-26T04:56:45.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.989+0000 7ffb637fe700 1 -- 172.21.15.60:0/2120141479 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffb5c027020 con 0x7ffb6c100260 2024-02-26T04:56:45.992 INFO:teuthology.orchestra.run.smithi060.stdout:34359738453 2024-02-26T04:56:45.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.991+0000 7ffb70936700 1 -- 172.21.15.60:0/2120141479 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffb580a21e0 msgr2=0x7ffb580a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:45.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.991+0000 7ffb70936700 1 --2- 172.21.15.60:0/2120141479 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffb580a21e0 0x7ffb580a46a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0ff760 tx=0x7ffb5400b410 comp rx=0 tx=0).stop 2024-02-26T04:56:45.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.991+0000 7ffb70936700 1 -- 172.21.15.60:0/2120141479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c100260 msgr2=0x7ffb6c101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:45.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.991+0000 7ffb70936700 1 --2- 172.21.15.60:0/2120141479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c100260 0x7ffb6c101440 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c000c00 tx=0x7ffb5c004730 comp rx=0 tx=0).stop 2024-02-26T04:56:45.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.991+0000 7ffb70936700 1 -- 172.21.15.60:0/2120141479 shutdown_connections 2024-02-26T04:56:45.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.991+0000 7ffb70936700 1 --2- 172.21.15.60:0/2120141479 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffb580a21e0 0x7ffb580a46a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:45.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.991+0000 7ffb70936700 1 --2- 172.21.15.60:0/2120141479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb6c106ce0 0x7ffb6c101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:45.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.991+0000 7ffb70936700 1 --2- 172.21.15.60:0/2120141479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c100260 0x7ffb6c101440 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:45.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.991+0000 7ffb70936700 1 --2- 172.21.15.60:0/2120141479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb6c100c20 0x7ffb6c101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:45.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.991+0000 7ffb70936700 1 -- 172.21.15.60:0/2120141479 >> 172.21.15.60:0/2120141479 conn(0x7ffb6c077170 msgr2=0x7ffb6c104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:45.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.992+0000 7ffb70936700 1 -- 172.21.15.60:0/2120141479 shutdown_connections 2024-02-26T04:56:45.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:45.992+0000 7ffb70936700 1 -- 172.21.15.60:0/2120141479 wait complete. 2024-02-26T04:56:46.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:46 smithi191 ceph-mon[36352]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-02-26T04:56:46.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:46 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2120141479' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-26T04:56:46.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:46 smithi156 ceph-mon[35805]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-02-26T04:56:46.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:46 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2120141479' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-26T04:56:46.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:46 smithi060 ceph-mon[32382]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-02-26T04:56:46.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:46 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2120141479' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-26T04:56:46.777 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738447 got 34359738453 for osd.0 2024-02-26T04:56:46.777 DEBUG:teuthology.parallel:result is None 2024-02-26T04:56:48.020 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.023+0000 7fbe10f02700 1 -- 172.21.15.60:0/1615747765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe0c06af20 msgr2=0x7fbe0c1068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.023+0000 7fbe10f02700 1 --2- 172.21.15.60:0/1615747765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe0c06af20 0x7fbe0c1068b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009990 tx=0x7fbe00009ca0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.024+0000 7fbe10f02700 1 -- 172.21.15.60:0/1615747765 shutdown_connections 2024-02-26T04:56:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.024+0000 7fbe10f02700 1 --2- 172.21.15.60:0/1615747765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe0c106fe0 0x7fbe0c10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.024+0000 7fbe10f02700 1 --2- 172.21.15.60:0/1615747765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe0c06a570 0x7fbe0c06a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.024+0000 7fbe10f02700 1 --2- 172.21.15.60:0/1615747765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe0c06af20 0x7fbe0c1068b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.024+0000 7fbe10f02700 1 -- 172.21.15.60:0/1615747765 >> 172.21.15.60:0/1615747765 conn(0x7fbe0c077ae0 msgr2=0x7fbe0c077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.024+0000 7fbe10f02700 1 -- 172.21.15.60:0/1615747765 shutdown_connections 2024-02-26T04:56:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.024+0000 7fbe10f02700 1 -- 172.21.15.60:0/1615747765 wait complete. 2024-02-26T04:56:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.025+0000 7fbe10f02700 1 Processor -- start 2024-02-26T04:56:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.025+0000 7fbe10f02700 1 -- start start 2024-02-26T04:56:48.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.025+0000 7fbe10f02700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe0c06a570 0x7fbe0c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:48.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.025+0000 7fbe10f02700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe0c06af20 0x7fbe0c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:48.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.025+0000 7fbe10f02700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe0c106fe0 0x7fbe0c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:48.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.025+0000 7fbe10f02700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c19d110 con 0x7fbe0c06a570 2024-02-26T04:56:48.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.025+0000 7fbe10f02700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c19d280 con 0x7fbe0c106fe0 2024-02-26T04:56:48.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.025+0000 7fbe10f02700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c19d440 con 0x7fbe0c06af20 2024-02-26T04:56:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.025+0000 7fbe0a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe0c06a570 0x7fbe0c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.025+0000 7fbe0a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe0c06a570 0x7fbe0c198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54874/0 (socket says 172.21.15.60:54874) 2024-02-26T04:56:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.025+0000 7fbe0a59c700 1 -- 172.21.15.60:0/2774363908 learned_addr learned my addr 172.21.15.60:0/2774363908 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.025+0000 7fbe09d9b700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe0c06af20 0x7fbe0c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.025+0000 7fbe0ad9d700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe0c106fe0 0x7fbe0c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.026+0000 7fbe09d9b700 1 -- 172.21.15.60:0/2774363908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe0c106fe0 msgr2=0x7fbe0c19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.026+0000 7fbe09d9b700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe0c106fe0 0x7fbe0c19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.026+0000 7fbe09d9b700 1 -- 172.21.15.60:0/2774363908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe0c06a570 msgr2=0x7fbe0c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.026+0000 7fbe09d9b700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe0c06a570 0x7fbe0c198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.026+0000 7fbe09d9b700 1 -- 172.21.15.60:0/2774363908 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe000095f0 con 0x7fbe0c06af20 2024-02-26T04:56:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.026+0000 7fbe0a59c700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe0c06a570 0x7fbe0c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:56:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.026+0000 7fbe09d9b700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe0c06af20 0x7fbe0c198770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe000098e0 tx=0x7fbe00004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:48.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.026+0000 7fbdfb7fe700 1 -- 172.21.15.60:0/2774363908 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0001d070 con 0x7fbe0c06af20 2024-02-26T04:56:48.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.026+0000 7fbe10f02700 1 -- 172.21.15.60:0/2774363908 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0c19d6c0 con 0x7fbe0c06af20 2024-02-26T04:56:48.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.026+0000 7fbdfb7fe700 1 -- 172.21.15.60:0/2774363908 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe0000b610 con 0x7fbe0c06af20 2024-02-26T04:56:48.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.026+0000 7fbdfb7fe700 1 -- 172.21.15.60:0/2774363908 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00021e80 con 0x7fbe0c06af20 2024-02-26T04:56:48.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.026+0000 7fbe10f02700 1 -- 172.21.15.60:0/2774363908 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0c1a4f80 con 0x7fbe0c06af20 2024-02-26T04:56:48.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.028+0000 7fbe10f02700 1 -- 172.21.15.60:0/2774363908 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0c10a1d0 con 0x7fbe0c06af20 2024-02-26T04:56:48.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.029+0000 7fbdfb7fe700 1 -- 172.21.15.60:0/2774363908 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fbe00021580 con 0x7fbe0c06af20 2024-02-26T04:56:48.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.029+0000 7fbdfb7fe700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbdf40a2260 0x7fbdf40a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:48.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.030+0000 7fbdfb7fe700 1 -- 172.21.15.60:0/2774363908 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fbe000cf1a0 con 0x7fbe0c06af20 2024-02-26T04:56:48.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.030+0000 7fbe0a59c700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbdf40a2260 0x7fbdf40a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:48.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.030+0000 7fbe0a59c700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbdf40a2260 0x7fbdf40a4720 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc005950 tx=0x7fbdfc00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:48.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.033+0000 7fbdfb7fe700 1 -- 172.21.15.60:0/2774363908 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe00097d40 con 0x7fbe0c06af20 2024-02-26T04:56:48.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.267+0000 7fbe10f02700 1 -- 172.21.15.60:0/2774363908 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbe0c199590 con 0x7fbe0c06af20 2024-02-26T04:56:48.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.267+0000 7fbdfb7fe700 1 -- 172.21.15.60:0/2774363908 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbe0009b400 con 0x7fbe0c06af20 2024-02-26T04:56:48.270 INFO:teuthology.orchestra.run.smithi060.stdout:55834574928 2024-02-26T04:56:48.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.270+0000 7fbe10f02700 1 -- 172.21.15.60:0/2774363908 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbdf40a2260 msgr2=0x7fbdf40a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:48.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.270+0000 7fbe10f02700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbdf40a2260 0x7fbdf40a4720 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc005950 tx=0x7fbdfc00b500 comp rx=0 tx=0).stop 2024-02-26T04:56:48.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.270+0000 7fbe10f02700 1 -- 172.21.15.60:0/2774363908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe0c06af20 msgr2=0x7fbe0c198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:48.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.270+0000 7fbe10f02700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe0c06af20 0x7fbe0c198770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe000098e0 tx=0x7fbe00004730 comp rx=0 tx=0).stop 2024-02-26T04:56:48.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.270+0000 7fbe10f02700 1 -- 172.21.15.60:0/2774363908 shutdown_connections 2024-02-26T04:56:48.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.270+0000 7fbe10f02700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fbdf40a2260 0x7fbdf40a4720 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.270+0000 7fbe10f02700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe0c106fe0 0x7fbe0c19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.270+0000 7fbe10f02700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe0c06a570 0x7fbe0c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.270+0000 7fbe10f02700 1 --2- 172.21.15.60:0/2774363908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe0c06af20 0x7fbe0c198770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.270+0000 7fbe10f02700 1 -- 172.21.15.60:0/2774363908 >> 172.21.15.60:0/2774363908 conn(0x7fbe0c077ae0 msgr2=0x7fbe0c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:48.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.270+0000 7fbe10f02700 1 -- 172.21.15.60:0/2774363908 shutdown_connections 2024-02-26T04:56:48.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.270+0000 7fbe10f02700 1 -- 172.21.15.60:0/2774363908 wait complete. 2024-02-26T04:56:48.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:48 smithi060 ceph-mon[32382]: pgmap v335: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:56:48.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:48 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1981082084' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-26T04:56:48.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:48 smithi191 ceph-mon[36352]: pgmap v335: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:56:48.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:48 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1981082084' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-26T04:56:48.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:48 smithi156 ceph-mon[35805]: pgmap v335: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:56:48.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:48 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1981082084' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-26T04:56:48.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.904+0000 7f27e8f5b700 1 -- 172.21.15.60:0/982492854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27e4100b50 msgr2=0x7f27e4100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.904+0000 7f27e8f5b700 1 --2- 172.21.15.60:0/982492854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27e4100b50 0x7f27e4100fd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009990 tx=0x7f27cc009ca0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.904+0000 7f27e8f5b700 1 -- 172.21.15.60:0/982492854 shutdown_connections 2024-02-26T04:56:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.904+0000 7f27e8f5b700 1 --2- 172.21.15.60:0/982492854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27e4101510 0x7f27e410d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.904+0000 7f27e8f5b700 1 --2- 172.21.15.60:0/982492854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e4105ab0 0x7f27e4105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.904+0000 7f27e8f5b700 1 --2- 172.21.15.60:0/982492854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27e4100b50 0x7f27e4100fd0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.904+0000 7f27e8f5b700 1 -- 172.21.15.60:0/982492854 >> 172.21.15.60:0/982492854 conn(0x7f27e4077b80 msgr2=0x7f27e4077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.904+0000 7f27e8f5b700 1 -- 172.21.15.60:0/982492854 shutdown_connections 2024-02-26T04:56:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.904+0000 7f27e8f5b700 1 -- 172.21.15.60:0/982492854 wait complete. 2024-02-26T04:56:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.905+0000 7f27e8f5b700 1 Processor -- start 2024-02-26T04:56:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.905+0000 7f27e8f5b700 1 -- start start 2024-02-26T04:56:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.905+0000 7f27e8f5b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27e4100b50 0x7f27e41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e8f5b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27e4101510 0x7f27e4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e8f5b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e4105ab0 0x7f27e419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e8f5b700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e419d0a0 con 0x7f27e4100b50 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e8f5b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e419d210 con 0x7f27e4105ab0 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e8f5b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e419d3d0 con 0x7f27e4101510 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27e4100b50 0x7f27e41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27e4100b50 0x7f27e41981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54856/0 (socket says 172.21.15.60:54856) 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e259c700 1 -- 172.21.15.60:0/1981082084 learned_addr learned my addr 172.21.15.60:0/1981082084 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e1d9b700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27e4101510 0x7f27e4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e2d9d700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e4105ab0 0x7f27e419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e259c700 1 -- 172.21.15.60:0/1981082084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27e4101510 msgr2=0x7f27e4198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e259c700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27e4101510 0x7f27e4198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e259c700 1 -- 172.21.15.60:0/1981082084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e4105ab0 msgr2=0x7f27e419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e259c700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e4105ab0 0x7f27e419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e259c700 1 -- 172.21.15.60:0/1981082084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc0095f0 con 0x7f27e4100b50 2024-02-26T04:56:48.838 INFO:teuthology.orchestra.run.smithi060.stdout:219043332145 2024-02-26T04:56:48.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e1d9b700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27e4101510 0x7f27e4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:56:48.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e259c700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27e4100b50 0x7f27e41981c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00d980 tx=0x7f27dc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:48.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27db7fe700 1 -- 172.21.15.60:0/1981082084 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc00bb60 con 0x7f27e4100b50 2024-02-26T04:56:48.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27db7fe700 1 -- 172.21.15.60:0/1981082084 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27dc007640 con 0x7f27e4100b50 2024-02-26T04:56:48.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e8f5b700 1 -- 172.21.15.60:0/1981082084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e419d6b0 con 0x7f27e4100b50 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27db7fe700 1 -- 172.21.15.60:0/1981082084 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc005020 con 0x7f27e4100b50 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.906+0000 7f27e8f5b700 1 -- 172.21.15.60:0/1981082084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e41a4f10 con 0x7f27e4100b50 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.907+0000 7f27e8f5b700 1 -- 172.21.15.60:0/1981082084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e4109070 con 0x7f27e4100b50 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.908+0000 7f27db7fe700 1 -- 172.21.15.60:0/1981082084 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f27dc004020 con 0x7f27e4100b50 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.909+0000 7f27db7fe700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f27d00a21e0 0x7f27d00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.909+0000 7f27e1d9b700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f27d00a21e0 0x7f27d00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.909+0000 7f27db7fe700 1 -- 172.21.15.60:0/1981082084 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f27dc0cdfa0 con 0x7f27e4100b50 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.910+0000 7f27e1d9b700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f27d00a21e0 0x7f27d00a46a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f27cc005b40 tx=0x7f27cc005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:47.914+0000 7f27db7fe700 1 -- 172.21.15.60:0/1981082084 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f27dc096ab0 con 0x7f27e4100b50 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.150+0000 7f27e8f5b700 1 -- 172.21.15.60:0/1981082084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f27e4199520 con 0x7f27e4100b50 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.150+0000 7f27db7fe700 1 -- 172.21.15.60:0/1981082084 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f27dc016020 con 0x7f27e4100b50 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.152+0000 7f27e8f5b700 1 -- 172.21.15.60:0/1981082084 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f27d00a21e0 msgr2=0x7f27d00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.152+0000 7f27e8f5b700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f27d00a21e0 0x7f27d00a46a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f27cc005b40 tx=0x7f27cc005a90 comp rx=0 tx=0).stop 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.152+0000 7f27e8f5b700 1 -- 172.21.15.60:0/1981082084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27e4100b50 msgr2=0x7f27e41981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.152+0000 7f27e8f5b700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27e4100b50 0x7f27e41981c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00d980 tx=0x7f27dc00dd40 comp rx=0 tx=0).stop 2024-02-26T04:56:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.152+0000 7f27e8f5b700 1 -- 172.21.15.60:0/1981082084 shutdown_connections 2024-02-26T04:56:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.152+0000 7f27e8f5b700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f27d00a21e0 0x7f27d00a46a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.152+0000 7f27e8f5b700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e4105ab0 0x7f27e419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.152+0000 7f27e8f5b700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27e4100b50 0x7f27e41981c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.152+0000 7f27e8f5b700 1 --2- 172.21.15.60:0/1981082084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27e4101510 0x7f27e4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.152+0000 7f27e8f5b700 1 -- 172.21.15.60:0/1981082084 >> 172.21.15.60:0/1981082084 conn(0x7f27e4077b80 msgr2=0x7f27e40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.152+0000 7f27e8f5b700 1 -- 172.21.15.60:0/1981082084 shutdown_connections 2024-02-26T04:56:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:48.152+0000 7f27e8f5b700 1 -- 172.21.15.60:0/1981082084 wait complete. 2024-02-26T04:56:49.432 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:49 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2774363908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-26T04:56:49.460 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332141 got 219043332145 for osd.7 2024-02-26T04:56:49.460 DEBUG:teuthology.parallel:result is None 2024-02-26T04:56:49.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:49 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2774363908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-26T04:56:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.627+0000 7fb575132700 1 -- 172.21.15.60:0/3668702244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5701011b0 msgr2=0x7fb570104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.627+0000 7fb575132700 1 --2- 172.21.15.60:0/3668702244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5701011b0 0x7fb570104fa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb5640099c0 tx=0x7fb564009cd0 comp rx=0 tx=0).stop 2024-02-26T04:56:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.627+0000 7fb575132700 1 -- 172.21.15.60:0/3668702244 shutdown_connections 2024-02-26T04:56:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.627+0000 7fb575132700 1 --2- 172.21.15.60:0/3668702244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5701056d0 0x7fb57010d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.627+0000 7fb575132700 1 --2- 172.21.15.60:0/3668702244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb570100800 0x7fb570100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.627+0000 7fb575132700 1 --2- 172.21.15.60:0/3668702244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5701011b0 0x7fb570104fa0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.627+0000 7fb575132700 1 -- 172.21.15.60:0/3668702244 >> 172.21.15.60:0/3668702244 conn(0x7fb570075ea0 msgr2=0x7fb5700762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:49.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.627+0000 7fb575132700 1 -- 172.21.15.60:0/3668702244 shutdown_connections 2024-02-26T04:56:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.627+0000 7fb575132700 1 -- 172.21.15.60:0/3668702244 wait complete. 2024-02-26T04:56:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.628+0000 7fb575132700 1 Processor -- start 2024-02-26T04:56:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.628+0000 7fb575132700 1 -- start start 2024-02-26T04:56:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb575132700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb570100800 0x7fb570106160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb575132700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5701011b0 0x7fb5701066a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb575132700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5701056d0 0x7fb570106be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb575132700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5701a64f0 con 0x7fb570100800 2024-02-26T04:56:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb575132700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5701a6660 con 0x7fb5701056d0 2024-02-26T04:56:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb575132700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5701a6820 con 0x7fb5701011b0 2024-02-26T04:56:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb56ed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb570100800 0x7fb570106160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb56ed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb570100800 0x7fb570106160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57114/0 (socket says 172.21.15.60:57114) 2024-02-26T04:56:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb56ed9d700 1 -- 172.21.15.60:0/3921357010 learned_addr learned my addr 172.21.15.60:0/3921357010 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb56f59e700 1 --2- 172.21.15.60:0/3921357010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5701056d0 0x7fb570106be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb56e59c700 1 --2- 172.21.15.60:0/3921357010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5701011b0 0x7fb5701066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb56ed9d700 1 -- 172.21.15.60:0/3921357010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5701011b0 msgr2=0x7fb5701066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb56ed9d700 1 --2- 172.21.15.60:0/3921357010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5701011b0 0x7fb5701066a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb56ed9d700 1 -- 172.21.15.60:0/3921357010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5701056d0 msgr2=0x7fb570106be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb56ed9d700 1 --2- 172.21.15.60:0/3921357010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5701056d0 0x7fb570106be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb56ed9d700 1 -- 172.21.15.60:0/3921357010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5640095f0 con 0x7fb570100800 2024-02-26T04:56:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb56ed9d700 1 --2- 172.21.15.60:0/3921357010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb570100800 0x7fb570106160 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb56000c9b0 tx=0x7fb56000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb557fff700 1 -- 172.21.15.60:0/3921357010 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560007b00 con 0x7fb570100800 2024-02-26T04:56:49.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.629+0000 7fb575132700 1 -- 172.21.15.60:0/3921357010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5701a6b00 con 0x7fb570100800 2024-02-26T04:56:49.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.630+0000 7fb557fff700 1 -- 172.21.15.60:0/3921357010 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb56000ce50 con 0x7fb570100800 2024-02-26T04:56:49.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.630+0000 7fb557fff700 1 -- 172.21.15.60:0/3921357010 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560018660 con 0x7fb570100800 2024-02-26T04:56:49.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.630+0000 7fb575132700 1 -- 172.21.15.60:0/3921357010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5701a7020 con 0x7fb570100800 2024-02-26T04:56:49.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.631+0000 7fb575132700 1 -- 172.21.15.60:0/3921357010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb570068930 con 0x7fb570100800 2024-02-26T04:56:49.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.632+0000 7fb557fff700 1 -- 172.21.15.60:0/3921357010 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fb56000f450 con 0x7fb570100800 2024-02-26T04:56:49.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.632+0000 7fb557fff700 1 --2- 172.21.15.60:0/3921357010 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb5580a21e0 0x7fb5580a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:49.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.632+0000 7fb56e59c700 1 --2- 172.21.15.60:0/3921357010 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb5580a21e0 0x7fb5580a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:49.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.633+0000 7fb557fff700 1 -- 172.21.15.60:0/3921357010 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fb5600ce380 con 0x7fb570100800 2024-02-26T04:56:49.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.633+0000 7fb56e59c700 1 --2- 172.21.15.60:0/3921357010 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb5580a21e0 0x7fb5580a46a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb564000c00 tx=0x7fb564005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:49.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.636+0000 7fb557fff700 1 -- 172.21.15.60:0/3921357010 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb560096ec0 con 0x7fb570100800 2024-02-26T04:56:49.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:49 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2774363908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-26T04:56:49.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.843+0000 7fb575132700 1 -- 172.21.15.60:0/3921357010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb570110460 con 0x7fb570100800 2024-02-26T04:56:49.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.843+0000 7fb557fff700 1 -- 172.21.15.60:0/3921357010 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb56009a580 con 0x7fb570100800 2024-02-26T04:56:49.846 INFO:teuthology.orchestra.run.smithi060.stdout:141733920833 2024-02-26T04:56:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.845+0000 7fb575132700 1 -- 172.21.15.60:0/3921357010 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb5580a21e0 msgr2=0x7fb5580a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.845+0000 7fb575132700 1 --2- 172.21.15.60:0/3921357010 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb5580a21e0 0x7fb5580a46a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb564000c00 tx=0x7fb564005fb0 comp rx=0 tx=0).stop 2024-02-26T04:56:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.845+0000 7fb575132700 1 -- 172.21.15.60:0/3921357010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb570100800 msgr2=0x7fb570106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.845+0000 7fb575132700 1 --2- 172.21.15.60:0/3921357010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb570100800 0x7fb570106160 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb56000c9b0 tx=0x7fb56000ccc0 comp rx=0 tx=0).stop 2024-02-26T04:56:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.845+0000 7fb575132700 1 -- 172.21.15.60:0/3921357010 shutdown_connections 2024-02-26T04:56:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.845+0000 7fb575132700 1 --2- 172.21.15.60:0/3921357010 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb5580a21e0 0x7fb5580a46a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.845+0000 7fb575132700 1 --2- 172.21.15.60:0/3921357010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5701056d0 0x7fb570106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.845+0000 7fb575132700 1 --2- 172.21.15.60:0/3921357010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb570100800 0x7fb570106160 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.845+0000 7fb575132700 1 --2- 172.21.15.60:0/3921357010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5701011b0 0x7fb5701066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.845+0000 7fb575132700 1 -- 172.21.15.60:0/3921357010 >> 172.21.15.60:0/3921357010 conn(0x7fb570075ea0 msgr2=0x7fb5700ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.845+0000 7fb575132700 1 -- 172.21.15.60:0/3921357010 shutdown_connections 2024-02-26T04:56:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:49.845+0000 7fb575132700 1 -- 172.21.15.60:0/3921357010 wait complete. 2024-02-26T04:56:50.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:50 smithi060 ceph-mon[32382]: pgmap v336: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:56:50.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:50 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3921357010' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-26T04:56:50.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:50 smithi191 ceph-mon[36352]: pgmap v336: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:56:50.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:50 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3921357010' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-26T04:56:50.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:50 smithi156 ceph-mon[35805]: pgmap v336: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:56:50.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:50 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3921357010' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-26T04:56:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.857+0000 7f20472e8700 1 -- 172.21.15.60:0/3964576761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2040106b80 msgr2=0x7f2040106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.857+0000 7f20472e8700 1 --2- 172.21.15.60:0/3964576761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2040106b80 0x7f2040106f60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f203000b0a0 tx=0x7f203000b3b0 comp rx=0 tx=0).stop 2024-02-26T04:56:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.857+0000 7f20472e8700 1 -- 172.21.15.60:0/3964576761 shutdown_connections 2024-02-26T04:56:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.857+0000 7f20472e8700 1 --2- 172.21.15.60:0/3964576761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2040101530 0x7f204010d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.857+0000 7f20472e8700 1 --2- 172.21.15.60:0/3964576761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2040106b80 0x7f2040106f60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.857+0000 7f20472e8700 1 --2- 172.21.15.60:0/3964576761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2040100b70 0x7f2040100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.857+0000 7f20472e8700 1 -- 172.21.15.60:0/3964576761 >> 172.21.15.60:0/3964576761 conn(0x7f2040077aa0 msgr2=0x7f2040077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.857+0000 7f20472e8700 1 -- 172.21.15.60:0/3964576761 shutdown_connections 2024-02-26T04:56:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.857+0000 7f20472e8700 1 -- 172.21.15.60:0/3964576761 wait complete. 2024-02-26T04:56:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f20472e8700 1 Processor -- start 2024-02-26T04:56:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f20472e8700 1 -- start start 2024-02-26T04:56:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f20472e8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2040100b70 0x7f2040198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f20472e8700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2040101530 0x7f20401987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f20472e8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2040106b80 0x7f204019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f20472e8700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204019d170 con 0x7f2040101530 2024-02-26T04:56:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f20472e8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204019d2e0 con 0x7f2040106b80 2024-02-26T04:56:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f20472e8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204019d4a0 con 0x7f2040100b70 2024-02-26T04:56:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f2044883700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2040101530 0x7f20401987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f2044883700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2040101530 0x7f20401987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57144/0 (socket says 172.21.15.60:57144) 2024-02-26T04:56:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f2044883700 1 -- 172.21.15.60:0/1814201008 learned_addr learned my addr 172.21.15.60:0/1814201008 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f2045084700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2040100b70 0x7f2040198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f2045885700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2040106b80 0x7f204019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f2044883700 1 -- 172.21.15.60:0/1814201008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2040100b70 msgr2=0x7f2040198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f2044883700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2040100b70 0x7f2040198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f2044883700 1 -- 172.21.15.60:0/1814201008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2040106b80 msgr2=0x7f204019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f2044883700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2040106b80 0x7f204019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:51.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f2044883700 1 -- 172.21.15.60:0/1814201008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2030009d00 con 0x7f2040101530 2024-02-26T04:56:51.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.858+0000 7f2045885700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2040106b80 0x7f204019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:56:51.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.859+0000 7f2044883700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2040101530 0x7f20401987d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f203400c9b0 tx=0x7f203400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:51.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.859+0000 7f203e7fc700 1 -- 172.21.15.60:0/1814201008 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034007b00 con 0x7f2040101530 2024-02-26T04:56:51.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.859+0000 7f20472e8700 1 -- 172.21.15.60:0/1814201008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204019d780 con 0x7f2040101530 2024-02-26T04:56:51.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.859+0000 7f203e7fc700 1 -- 172.21.15.60:0/1814201008 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2034007c60 con 0x7f2040101530 2024-02-26T04:56:51.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.859+0000 7f203e7fc700 1 -- 172.21.15.60:0/1814201008 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034018730 con 0x7f2040101530 2024-02-26T04:56:51.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.859+0000 7f20472e8700 1 -- 172.21.15.60:0/1814201008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20401a4fe0 con 0x7f2040101530 2024-02-26T04:56:51.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.860+0000 7f20472e8700 1 -- 172.21.15.60:0/1814201008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2040068930 con 0x7f2040101530 2024-02-26T04:56:51.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.861+0000 7f203e7fc700 1 -- 172.21.15.60:0/1814201008 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f2034018890 con 0x7f2040101530 2024-02-26T04:56:51.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.862+0000 7f203e7fc700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f20280a21e0 0x7f20280a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:51.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.862+0000 7f2045084700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f20280a21e0 0x7f20280a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:51.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.862+0000 7f203e7fc700 1 -- 172.21.15.60:0/1814201008 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f20340ce5f0 con 0x7f2040101530 2024-02-26T04:56:51.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.862+0000 7f2045084700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f20280a21e0 0x7f20280a46a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2030006130 tx=0x7f2030006080 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:51.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:51.865+0000 7f203e7fc700 1 -- 172.21.15.60:0/1814201008 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f20340970a0 con 0x7f2040101530 2024-02-26T04:56:52.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.067+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/1883552602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f78101f50 msgr2=0x7f7f78102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:52.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.067+0000 7f7f7cbb1700 1 --2- 172.21.15.60:0/1883552602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f78101f50 0x7f7f78102330 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009990 tx=0x7f7f68009ca0 comp rx=0 tx=0).stop 2024-02-26T04:56:52.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.067+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/1883552602 shutdown_connections 2024-02-26T04:56:52.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.067+0000 7f7f7cbb1700 1 --2- 172.21.15.60:0/1883552602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f78106f80 0x7f7f7810eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:52.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.067+0000 7f7f7cbb1700 1 --2- 172.21.15.60:0/1883552602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f78101f50 0x7f7f78102330 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:52.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.067+0000 7f7f7cbb1700 1 --2- 172.21.15.60:0/1883552602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f78102900 0x7f7f781068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:52.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.067+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/1883552602 >> 172.21.15.60:0/1883552602 conn(0x7f7f78078d00 msgr2=0x7f7f78079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:52.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.067+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/1883552602 shutdown_connections 2024-02-26T04:56:52.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.067+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/1883552602 wait complete. 2024-02-26T04:56:52.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.068+0000 7f7f7cbb1700 1 Processor -- start 2024-02-26T04:56:52.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.068+0000 7f7f7cbb1700 1 -- start start 2024-02-26T04:56:52.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.068+0000 7f7f7cbb1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f78101f50 0x7f7f78073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:52.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.068+0000 7f7f7cbb1700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f78102900 0x7f7f780744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:52.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.068+0000 7f7f7cbb1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f78106f80 0x7f7f780749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:52.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.068+0000 7f7f7cbb1700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f781a65b0 con 0x7f7f78102900 2024-02-26T04:56:52.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.068+0000 7f7f7cbb1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f781a6720 con 0x7f7f78106f80 2024-02-26T04:56:52.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.068+0000 7f7f7cbb1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f781a68e0 con 0x7f7f78101f50 2024-02-26T04:56:52.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.068+0000 7f7f75d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f78102900 0x7f7f780744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:52.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.068+0000 7f7f75d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f78102900 0x7f7f780744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57158/0 (socket says 172.21.15.60:57158) 2024-02-26T04:56:52.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.068+0000 7f7f75d9b700 1 -- 172.21.15.60:0/2516574355 learned_addr learned my addr 172.21.15.60:0/2516574355 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:52.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.068+0000 7f7f7659c700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f78101f50 0x7f7f78073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:52.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.068+0000 7f7f76d9d700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f78106f80 0x7f7f780749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:52.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.069+0000 7f7f75d9b700 1 -- 172.21.15.60:0/2516574355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f78101f50 msgr2=0x7f7f78073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:52.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.069+0000 7f7f75d9b700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f78101f50 0x7f7f78073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:52.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.069+0000 7f7f75d9b700 1 -- 172.21.15.60:0/2516574355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f78106f80 msgr2=0x7f7f780749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:52.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.069+0000 7f7f75d9b700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f78106f80 0x7f7f780749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:52.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.069+0000 7f7f75d9b700 1 -- 172.21.15.60:0/2516574355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f680095f0 con 0x7f7f78102900 2024-02-26T04:56:52.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.069+0000 7f7f76d9d700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f78106f80 0x7f7f780749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:56:52.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.069+0000 7f7f75d9b700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f78102900 0x7f7f780744b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000d980 tx=0x7f7f6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:52.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.069+0000 7f7f6f7fe700 1 -- 172.21.15.60:0/2516574355 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60009930 con 0x7f7f78102900 2024-02-26T04:56:52.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.069+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/2516574355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f781a6bc0 con 0x7f7f78102900 2024-02-26T04:56:52.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.069+0000 7f7f6f7fe700 1 -- 172.21.15.60:0/2516574355 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f60010460 con 0x7f7f78102900 2024-02-26T04:56:52.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.069+0000 7f7f6f7fe700 1 -- 172.21.15.60:0/2516574355 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6000f620 con 0x7f7f78102900 2024-02-26T04:56:52.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.069+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/2516574355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f781a7110 con 0x7f7f78102900 2024-02-26T04:56:52.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.070+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/2516574355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7810a210 con 0x7f7f78102900 2024-02-26T04:56:52.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.071+0000 7f7f6f7fe700 1 -- 172.21.15.60:0/2516574355 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f7f600105d0 con 0x7f7f78102900 2024-02-26T04:56:52.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.072+0000 7f20472e8700 1 -- 172.21.15.60:0/1814201008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2040110ba0 con 0x7f2040101530 2024-02-26T04:56:52.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.072+0000 7f7f6f7fe700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7f640a21e0 0x7f7f640a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:52.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.072+0000 7f7f7659c700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7f640a21e0 0x7f7f640a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:52.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.072+0000 7f7f6f7fe700 1 -- 172.21.15.60:0/2516574355 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f7f600cdca0 con 0x7f7f78102900 2024-02-26T04:56:52.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.073+0000 7f7f7659c700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7f640a21e0 0x7f7f640a46a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800b5c0 tx=0x7f7f68005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:52.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.076+0000 7f203e7fc700 1 -- 172.21.15.60:0/1814201008 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f203409a760 con 0x7f2040101530 2024-02-26T04:56:52.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.077+0000 7f7f6f7fe700 1 -- 172.21.15.60:0/2516574355 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7f600967b0 con 0x7f7f78102900 2024-02-26T04:56:52.079 INFO:teuthology.orchestra.run.smithi060.stdout:197568495671 2024-02-26T04:56:52.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.079+0000 7f20472e8700 1 -- 172.21.15.60:0/1814201008 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f20280a21e0 msgr2=0x7f20280a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:52.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.079+0000 7f20472e8700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f20280a21e0 0x7f20280a46a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2030006130 tx=0x7f2030006080 comp rx=0 tx=0).stop 2024-02-26T04:56:52.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.079+0000 7f20472e8700 1 -- 172.21.15.60:0/1814201008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2040101530 msgr2=0x7f20401987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:52.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.079+0000 7f20472e8700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2040101530 0x7f20401987d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f203400c9b0 tx=0x7f203400cd70 comp rx=0 tx=0).stop 2024-02-26T04:56:52.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.079+0000 7f20472e8700 1 -- 172.21.15.60:0/1814201008 shutdown_connections 2024-02-26T04:56:52.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.079+0000 7f20472e8700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f20280a21e0 0x7f20280a46a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:52.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.079+0000 7f20472e8700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2040106b80 0x7f204019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:52.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.079+0000 7f20472e8700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2040100b70 0x7f2040198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:52.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.079+0000 7f20472e8700 1 --2- 172.21.15.60:0/1814201008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2040101530 0x7f20401987d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:52.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.079+0000 7f20472e8700 1 -- 172.21.15.60:0/1814201008 >> 172.21.15.60:0/1814201008 conn(0x7f2040077aa0 msgr2=0x7f20400ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:52.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.080+0000 7f20472e8700 1 -- 172.21.15.60:0/1814201008 shutdown_connections 2024-02-26T04:56:52.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.080+0000 7f20472e8700 1 -- 172.21.15.60:0/1814201008 wait complete. 2024-02-26T04:56:52.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.292+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/2516574355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7f780754a0 con 0x7f7f78102900 2024-02-26T04:56:52.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.292+0000 7f7f6f7fe700 1 -- 172.21.15.60:0/2516574355 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f60099e70 con 0x7f7f78102900 2024-02-26T04:56:52.295 INFO:teuthology.orchestra.run.smithi060.stdout:107374182471 2024-02-26T04:56:52.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.294+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/2516574355 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7f640a21e0 msgr2=0x7f7f640a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:52.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.294+0000 7f7f7cbb1700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7f640a21e0 0x7f7f640a46a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800b5c0 tx=0x7f7f68005960 comp rx=0 tx=0).stop 2024-02-26T04:56:52.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.294+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/2516574355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f78102900 msgr2=0x7f7f780744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:52.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.294+0000 7f7f7cbb1700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f78102900 0x7f7f780744b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000d980 tx=0x7f7f6000dc90 comp rx=0 tx=0).stop 2024-02-26T04:56:52.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.294+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/2516574355 shutdown_connections 2024-02-26T04:56:52.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.294+0000 7f7f7cbb1700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7f640a21e0 0x7f7f640a46a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:52.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.294+0000 7f7f7cbb1700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f78106f80 0x7f7f780749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:52.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.294+0000 7f7f7cbb1700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f78101f50 0x7f7f78073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:52.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.294+0000 7f7f7cbb1700 1 --2- 172.21.15.60:0/2516574355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f78102900 0x7f7f780744b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:52.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.294+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/2516574355 >> 172.21.15.60:0/2516574355 conn(0x7f7f78078d00 msgr2=0x7f7f78101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:52.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.294+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/2516574355 shutdown_connections 2024-02-26T04:56:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:52.294+0000 7f7f7cbb1700 1 -- 172.21.15.60:0/2516574355 wait complete. 2024-02-26T04:56:52.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:52 smithi060 ceph-mon[32382]: pgmap v337: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:56:52.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:52 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1814201008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-26T04:56:52.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:52 smithi191 ceph-mon[36352]: pgmap v337: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:56:52.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:52 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1814201008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-26T04:56:52.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:52 smithi156 ceph-mon[35805]: pgmap v337: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:56:52.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:52 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1814201008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-26T04:56:52.988 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920829 got 141733920833 for osd.4 2024-02-26T04:56:52.988 DEBUG:teuthology.parallel:result is None 2024-02-26T04:56:53.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.213+0000 7f8adf284700 1 -- 172.21.15.60:0/2161605719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad81056d0 msgr2=0x7f8ad810d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.213+0000 7f8adf284700 1 --2- 172.21.15.60:0/2161605719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad81056d0 0x7f8ad810d210 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000b0d0 tx=0x7f8ad000b3e0 comp rx=0 tx=0).stop 2024-02-26T04:56:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.213+0000 7f8adf284700 1 -- 172.21.15.60:0/2161605719 shutdown_connections 2024-02-26T04:56:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.213+0000 7f8adf284700 1 --2- 172.21.15.60:0/2161605719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad81056d0 0x7f8ad810d210 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.213+0000 7f8adf284700 1 --2- 172.21.15.60:0/2161605719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ad8100800 0x7f8ad8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.214+0000 7f8adf284700 1 --2- 172.21.15.60:0/2161605719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ad81011b0 0x7f8ad8104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.214+0000 7f8adf284700 1 -- 172.21.15.60:0/2161605719 >> 172.21.15.60:0/2161605719 conn(0x7f8ad8075ea0 msgr2=0x7f8ad80762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.214+0000 7f8adf284700 1 -- 172.21.15.60:0/2161605719 shutdown_connections 2024-02-26T04:56:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.214+0000 7f8adf284700 1 -- 172.21.15.60:0/2161605719 wait complete. 2024-02-26T04:56:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.214+0000 7f8adf284700 1 Processor -- start 2024-02-26T04:56:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.214+0000 7f8adf284700 1 -- start start 2024-02-26T04:56:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.214+0000 7f8adf284700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ad8100800 0x7f8ad8101fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.214+0000 7f8adf284700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad81011b0 0x7f8ad8102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.214+0000 7f8adf284700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ad81056d0 0x7f8ad8102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.214+0000 7f8adf284700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81a6650 con 0x7f8ad81011b0 2024-02-26T04:56:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.214+0000 7f8adf284700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81a67c0 con 0x7f8ad81056d0 2024-02-26T04:56:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.214+0000 7f8adf284700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81a6980 con 0x7f8ad8100800 2024-02-26T04:56:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8adc81f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad81011b0 0x7f8ad8102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8adc81f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad81011b0 0x7f8ad8102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57172/0 (socket says 172.21.15.60:57172) 2024-02-26T04:56:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8adc81f700 1 -- 172.21.15.60:0/2545083689 learned_addr learned my addr 172.21.15.60:0/2545083689 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8add020700 1 --2- 172.21.15.60:0/2545083689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ad8100800 0x7f8ad8101fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8add821700 1 --2- 172.21.15.60:0/2545083689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ad81056d0 0x7f8ad8102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8adc81f700 1 -- 172.21.15.60:0/2545083689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ad8100800 msgr2=0x7f8ad8101fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8adc81f700 1 --2- 172.21.15.60:0/2545083689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ad8100800 0x7f8ad8101fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8adc81f700 1 -- 172.21.15.60:0/2545083689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ad81056d0 msgr2=0x7f8ad8102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8adc81f700 1 --2- 172.21.15.60:0/2545083689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ad81056d0 0x7f8ad8102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8adc81f700 1 -- 172.21.15.60:0/2545083689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad0009d00 con 0x7f8ad81011b0 2024-02-26T04:56:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8adc81f700 1 --2- 172.21.15.60:0/2545083689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad81011b0 0x7f8ad8102510 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400d950 tx=0x7f8ad400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8ace7fc700 1 -- 172.21.15.60:0/2545083689 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4009930 con 0x7f8ad81011b0 2024-02-26T04:56:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8ace7fc700 1 -- 172.21.15.60:0/2545083689 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ad4010460 con 0x7f8ad81011b0 2024-02-26T04:56:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8adf284700 1 -- 172.21.15.60:0/2545083689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad81a6c60 con 0x7f8ad81011b0 2024-02-26T04:56:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8ace7fc700 1 -- 172.21.15.60:0/2545083689 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400f620 con 0x7f8ad81011b0 2024-02-26T04:56:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.215+0000 7f8adf284700 1 -- 172.21.15.60:0/2545083689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad81a7160 con 0x7f8ad81011b0 2024-02-26T04:56:53.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.216+0000 7f8adf284700 1 -- 172.21.15.60:0/2545083689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad8109fa0 con 0x7f8ad81011b0 2024-02-26T04:56:53.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.217+0000 7f8ace7fc700 1 -- 172.21.15.60:0/2545083689 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f8ad40108f0 con 0x7f8ad81011b0 2024-02-26T04:56:53.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.218+0000 7f8ace7fc700 1 --2- 172.21.15.60:0/2545083689 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8ac40a21e0 0x7f8ac40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:53.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.218+0000 7f8add020700 1 --2- 172.21.15.60:0/2545083689 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8ac40a21e0 0x7f8ac40a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:53.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.219+0000 7f8ace7fc700 1 -- 172.21.15.60:0/2545083689 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f8ad40cdcf0 con 0x7f8ad81011b0 2024-02-26T04:56:53.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.219+0000 7f8add020700 1 --2- 172.21.15.60:0/2545083689 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8ac40a21e0 0x7f8ac40a46a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8005950 tx=0x7f8ac800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.222+0000 7f8ace7fc700 1 -- 172.21.15.60:0/2545083689 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ad40967a0 con 0x7f8ad81011b0 2024-02-26T04:56:53.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.423+0000 7f8adf284700 1 -- 172.21.15.60:0/2545083689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8ad81103b0 con 0x7f8ad81011b0 2024-02-26T04:56:53.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.424+0000 7f8ace7fc700 1 -- 172.21.15.60:0/2545083689 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8ad4021020 con 0x7f8ad81011b0 2024-02-26T04:56:53.427 INFO:teuthology.orchestra.run.smithi060.stdout:77309411405 2024-02-26T04:56:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.426+0000 7f8adf284700 1 -- 172.21.15.60:0/2545083689 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8ac40a21e0 msgr2=0x7f8ac40a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.426+0000 7f8adf284700 1 --2- 172.21.15.60:0/2545083689 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8ac40a21e0 0x7f8ac40a46a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8005950 tx=0x7f8ac800b410 comp rx=0 tx=0).stop 2024-02-26T04:56:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.426+0000 7f8adf284700 1 -- 172.21.15.60:0/2545083689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad81011b0 msgr2=0x7f8ad8102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.426+0000 7f8adf284700 1 --2- 172.21.15.60:0/2545083689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad81011b0 0x7f8ad8102510 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400d950 tx=0x7f8ad400dc60 comp rx=0 tx=0).stop 2024-02-26T04:56:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.426+0000 7f8adf284700 1 -- 172.21.15.60:0/2545083689 shutdown_connections 2024-02-26T04:56:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.426+0000 7f8adf284700 1 --2- 172.21.15.60:0/2545083689 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8ac40a21e0 0x7f8ac40a46a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.426+0000 7f8adf284700 1 --2- 172.21.15.60:0/2545083689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ad81056d0 0x7f8ad8102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.426+0000 7f8adf284700 1 --2- 172.21.15.60:0/2545083689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ad8100800 0x7f8ad8101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.426+0000 7f8adf284700 1 --2- 172.21.15.60:0/2545083689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ad81011b0 0x7f8ad8102510 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.426+0000 7f8adf284700 1 -- 172.21.15.60:0/2545083689 >> 172.21.15.60:0/2545083689 conn(0x7f8ad8075ea0 msgr2=0x7f8ad81070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.426+0000 7f8adf284700 1 -- 172.21.15.60:0/2545083689 shutdown_connections 2024-02-26T04:56:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:53.426+0000 7f8adf284700 1 -- 172.21.15.60:0/2545083689 wait complete. 2024-02-26T04:56:53.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:53 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2516574355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-26T04:56:53.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:53 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2516574355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-26T04:56:53.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:53 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2516574355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-26T04:56:54.110 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574923 got 55834574928 for osd.1 2024-02-26T04:56:54.110 DEBUG:teuthology.parallel:result is None 2024-02-26T04:56:54.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:54 smithi060 ceph-mon[32382]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:56:54.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:54 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2545083689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-26T04:56:54.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:54 smithi156 ceph-mon[35805]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:56:54.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:54 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2545083689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-26T04:56:54.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:54 smithi191 ceph-mon[36352]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:56:54.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:54 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2545083689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-26T04:56:55.182 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182468 got 107374182471 for osd.3 2024-02-26T04:56:55.182 DEBUG:teuthology.parallel:result is None 2024-02-26T04:56:55.577 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:55.581 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495667 got 197568495671 for osd.6 2024-02-26T04:56:55.581 DEBUG:teuthology.parallel:result is None 2024-02-26T04:56:56.263 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411405 for osd.2 2024-02-26T04:56:56.263 DEBUG:teuthology.parallel:result is None 2024-02-26T04:56:56.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:56 smithi060 ceph-mon[32382]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:56:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:56:56.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:56 smithi156 ceph-mon[35805]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:56:56.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:56 smithi191 ceph-mon[36352]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:56:58.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.269+0000 7f643bfc8700 1 -- 172.21.15.60:0/2746764803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64341005b0 msgr2=0x7f6434100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:58.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.269+0000 7f643bfc8700 1 --2- 172.21.15.60:0/2746764803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64341005b0 0x7f6434100990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6430009990 tx=0x7f6430009ca0 comp rx=0 tx=0).stop 2024-02-26T04:56:58.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.270+0000 7f643bfc8700 1 -- 172.21.15.60:0/2746764803 shutdown_connections 2024-02-26T04:56:58.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.270+0000 7f643bfc8700 1 --2- 172.21.15.60:0/2746764803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6434112a40 0x7f6434114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:58.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.270+0000 7f643bfc8700 1 --2- 172.21.15.60:0/2746764803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64341005b0 0x7f6434100990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:58.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.270+0000 7f643bfc8700 1 --2- 172.21.15.60:0/2746764803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6434100f60 0x7f6434112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:58.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.270+0000 7f643bfc8700 1 -- 172.21.15.60:0/2746764803 >> 172.21.15.60:0/2746764803 conn(0x7f6434075ed0 msgr2=0x7f64340762e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:58.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.270+0000 7f643bfc8700 1 -- 172.21.15.60:0/2746764803 shutdown_connections 2024-02-26T04:56:58.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.270+0000 7f643bfc8700 1 -- 172.21.15.60:0/2746764803 wait complete. 2024-02-26T04:56:58.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.270+0000 7f643bfc8700 1 Processor -- start 2024-02-26T04:56:58.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.270+0000 7f643bfc8700 1 -- start start 2024-02-26T04:56:58.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f643bfc8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64341005b0 0x7f64341a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:58.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f643bfc8700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6434100f60 0x7f64341a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:58.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f643bfc8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6434112a40 0x7f643419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f643bfc8700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643419ec80 con 0x7f6434100f60 2024-02-26T04:56:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f643bfc8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643419edc0 con 0x7f64341005b0 2024-02-26T04:56:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f643bfc8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643419ef80 con 0x7f6434112a40 2024-02-26T04:56:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f6439563700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6434100f60 0x7f64341a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f6439563700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6434100f60 0x7f64341a49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57182/0 (socket says 172.21.15.60:57182) 2024-02-26T04:56:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f6439563700 1 -- 172.21.15.60:0/877759779 learned_addr learned my addr 172.21.15.60:0/877759779 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:56:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f6439d64700 1 --2- 172.21.15.60:0/877759779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64341005b0 0x7f64341a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f643a565700 1 --2- 172.21.15.60:0/877759779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6434112a40 0x7f643419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f6439563700 1 -- 172.21.15.60:0/877759779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6434112a40 msgr2=0x7f643419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f6439563700 1 --2- 172.21.15.60:0/877759779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6434112a40 0x7f643419e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f6439563700 1 -- 172.21.15.60:0/877759779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64341005b0 msgr2=0x7f64341a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f6439563700 1 --2- 172.21.15.60:0/877759779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64341005b0 0x7f64341a4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f6439563700 1 -- 172.21.15.60:0/877759779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64300095f0 con 0x7f6434100f60 2024-02-26T04:56:58.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f6439563700 1 --2- 172.21.15.60:0/877759779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6434100f60 0x7f64341a49a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f642400c920 tx=0x7f642400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:58.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.271+0000 7f642affd700 1 -- 172.21.15.60:0/877759779 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424004a30 con 0x7f6434100f60 2024-02-26T04:56:58.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.272+0000 7f642affd700 1 -- 172.21.15.60:0/877759779 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6424005500 con 0x7f6434100f60 2024-02-26T04:56:58.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.272+0000 7f643bfc8700 1 -- 172.21.15.60:0/877759779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643419f260 con 0x7f6434100f60 2024-02-26T04:56:58.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.272+0000 7f642affd700 1 -- 172.21.15.60:0/877759779 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424007c80 con 0x7f6434100f60 2024-02-26T04:56:58.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.272+0000 7f643bfc8700 1 -- 172.21.15.60:0/877759779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64341ab500 con 0x7f6434100f60 2024-02-26T04:56:58.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.273+0000 7f643bfc8700 1 -- 172.21.15.60:0/877759779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6434068930 con 0x7f6434100f60 2024-02-26T04:56:58.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.274+0000 7f642affd700 1 -- 172.21.15.60:0/877759779 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f6424004b90 con 0x7f6434100f60 2024-02-26T04:56:58.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.274+0000 7f642affd700 1 --2- 172.21.15.60:0/877759779 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f64200a21e0 0x7f64200a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:56:58.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.274+0000 7f6439d64700 1 --2- 172.21.15.60:0/877759779 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f64200a21e0 0x7f64200a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:56:58.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.275+0000 7f642affd700 1 -- 172.21.15.60:0/877759779 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f64240cdd20 con 0x7f6434100f60 2024-02-26T04:56:58.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.275+0000 7f6439d64700 1 --2- 172.21.15.60:0/877759779 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f64200a21e0 0x7f64200a46a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f643000b5c0 tx=0x7f6430005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:56:58.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.279+0000 7f642affd700 1 -- 172.21.15.60:0/877759779 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6424096830 con 0x7f6434100f60 2024-02-26T04:56:58.385 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:58 smithi060 ceph-mon[32382]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:56:58.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.486+0000 7f643bfc8700 1 -- 172.21.15.60:0/877759779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f643419fb50 con 0x7f6434100f60 2024-02-26T04:56:58.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.487+0000 7f642affd700 1 -- 172.21.15.60:0/877759779 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f6424099ef0 con 0x7f6434100f60 2024-02-26T04:56:58.490 INFO:teuthology.orchestra.run.smithi060.stdout:343597383710 2024-02-26T04:56:58.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.489+0000 7f643bfc8700 1 -- 172.21.15.60:0/877759779 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f64200a21e0 msgr2=0x7f64200a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:58.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.489+0000 7f643bfc8700 1 --2- 172.21.15.60:0/877759779 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f64200a21e0 0x7f64200a46a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f643000b5c0 tx=0x7f6430005960 comp rx=0 tx=0).stop 2024-02-26T04:56:58.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.489+0000 7f643bfc8700 1 -- 172.21.15.60:0/877759779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6434100f60 msgr2=0x7f64341a49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:56:58.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.489+0000 7f643bfc8700 1 --2- 172.21.15.60:0/877759779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6434100f60 0x7f64341a49a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f642400c920 tx=0x7f642400cc30 comp rx=0 tx=0).stop 2024-02-26T04:56:58.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.489+0000 7f643bfc8700 1 -- 172.21.15.60:0/877759779 shutdown_connections 2024-02-26T04:56:58.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.489+0000 7f643bfc8700 1 --2- 172.21.15.60:0/877759779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6434112a40 0x7f643419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:58.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.489+0000 7f643bfc8700 1 --2- 172.21.15.60:0/877759779 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f64200a21e0 0x7f64200a46a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:58.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.489+0000 7f643bfc8700 1 --2- 172.21.15.60:0/877759779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64341005b0 0x7f64341a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:58.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.489+0000 7f643bfc8700 1 --2- 172.21.15.60:0/877759779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6434100f60 0x7f64341a49a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:56:58.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.489+0000 7f643bfc8700 1 -- 172.21.15.60:0/877759779 >> 172.21.15.60:0/877759779 conn(0x7f6434075ed0 msgr2=0x7f64341104f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:56:58.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.489+0000 7f643bfc8700 1 -- 172.21.15.60:0/877759779 shutdown_connections 2024-02-26T04:56:58.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:56:58.489+0000 7f643bfc8700 1 -- 172.21.15.60:0/877759779 wait complete. 2024-02-26T04:56:58.561 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:58 smithi191 ceph-mon[36352]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:56:58.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:58 smithi156 ceph-mon[35805]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:56:59.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:56:59 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/877759779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-26T04:56:59.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:56:59 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/877759779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-26T04:56:59.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:56:59 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/877759779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-26T04:57:00.457 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:00 smithi060 ceph-mon[32382]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:00.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:00 smithi156 ceph-mon[35805]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:00.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:00 smithi191 ceph-mon[36352]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:01.476 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383706 got 343597383710 for osd.11 2024-02-26T04:57:01.477 DEBUG:teuthology.parallel:result is None 2024-02-26T04:57:02.650 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:02 smithi060 ceph-mon[32382]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:57:02.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:02 smithi156 ceph-mon[35805]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:57:02.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:02 smithi191 ceph-mon[36352]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:57:02.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.893+0000 7fd048d20700 1 -- 172.21.15.60:0/1837436791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0441026e0 msgr2=0x7fd04410ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:02.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.893+0000 7fd048d20700 1 --2- 172.21.15.60:0/1837436791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0441026e0 0x7fd04410ead0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00b0a0 tx=0x7fd03c00b3b0 comp rx=0 tx=0).stop 2024-02-26T04:57:02.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.893+0000 7fd048d20700 1 -- 172.21.15.60:0/1837436791 shutdown_connections 2024-02-26T04:57:02.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.893+0000 7fd048d20700 1 --2- 172.21.15.60:0/1837436791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0441026e0 0x7fd04410ead0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:02.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.893+0000 7fd048d20700 1 --2- 172.21.15.60:0/1837436791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd044107d90 0x7fd044108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:02.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.893+0000 7fd048d20700 1 --2- 172.21.15.60:0/1837436791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd044101d20 0x7fd0441021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:02.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.893+0000 7fd048d20700 1 -- 172.21.15.60:0/1837436791 >> 172.21.15.60:0/1837436791 conn(0x7fd044078d40 msgr2=0x7fd044079150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:02.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.893+0000 7fd048d20700 1 -- 172.21.15.60:0/1837436791 shutdown_connections 2024-02-26T04:57:02.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.893+0000 7fd048d20700 1 -- 172.21.15.60:0/1837436791 wait complete. 2024-02-26T04:57:02.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.894+0000 7fd048d20700 1 Processor -- start 2024-02-26T04:57:02.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.894+0000 7fd048d20700 1 -- start start 2024-02-26T04:57:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.894+0000 7fd048d20700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd044101d20 0x7fd044198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.894+0000 7fd048d20700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0441026e0 0x7fd044198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.894+0000 7fd048d20700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd044107d90 0x7fd04419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.894+0000 7fd048d20700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04419d0e0 con 0x7fd044107d90 2024-02-26T04:57:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.894+0000 7fd048d20700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04419d250 con 0x7fd044101d20 2024-02-26T04:57:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.894+0000 7fd048d20700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04419d410 con 0x7fd0441026e0 2024-02-26T04:57:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd042d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd044107d90 0x7fd04419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd042d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd044107d90 0x7fd04419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34310/0 (socket says 172.21.15.60:34310) 2024-02-26T04:57:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd042d9d700 1 -- 172.21.15.60:0/4124263193 learned_addr learned my addr 172.21.15.60:0/4124263193 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd041d9b700 1 --2- 172.21.15.60:0/4124263193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0441026e0 0x7fd044198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd04259c700 1 --2- 172.21.15.60:0/4124263193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd044101d20 0x7fd044198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd042d9d700 1 -- 172.21.15.60:0/4124263193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0441026e0 msgr2=0x7fd044198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd042d9d700 1 --2- 172.21.15.60:0/4124263193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0441026e0 0x7fd044198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd042d9d700 1 -- 172.21.15.60:0/4124263193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd044101d20 msgr2=0x7fd044198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd042d9d700 1 --2- 172.21.15.60:0/4124263193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd044101d20 0x7fd044198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd042d9d700 1 -- 172.21.15.60:0/4124263193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c009d00 con 0x7fd044107d90 2024-02-26T04:57:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd042d9d700 1 --2- 172.21.15.60:0/4124263193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd044107d90 0x7fd04419cad0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00b070 tx=0x7fd03c004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd0337fe700 1 -- 172.21.15.60:0/4124263193 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c00e040 con 0x7fd044107d90 2024-02-26T04:57:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd0337fe700 1 -- 172.21.15.60:0/4124263193 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd03c007850 con 0x7fd044107d90 2024-02-26T04:57:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd048d20700 1 -- 172.21.15.60:0/4124263193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04419d690 con 0x7fd044107d90 2024-02-26T04:57:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd0337fe700 1 -- 172.21.15.60:0/4124263193 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c01b850 con 0x7fd044107d90 2024-02-26T04:57:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.895+0000 7fd048d20700 1 -- 172.21.15.60:0/4124263193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0441a4f50 con 0x7fd044107d90 2024-02-26T04:57:02.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.896+0000 7fd048d20700 1 -- 172.21.15.60:0/4124263193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd044068930 con 0x7fd044107d90 2024-02-26T04:57:02.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.899+0000 7fd0337fe700 1 -- 172.21.15.60:0/4124263193 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fd03c01da80 con 0x7fd044107d90 2024-02-26T04:57:02.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.899+0000 7fd0337fe700 1 --2- 172.21.15.60:0/4124263193 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd02c0a21e0 0x7fd02c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:02.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.899+0000 7fd0337fe700 1 -- 172.21.15.60:0/4124263193 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fd03c0d0090 con 0x7fd044107d90 2024-02-26T04:57:02.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.899+0000 7fd04259c700 1 --2- 172.21.15.60:0/4124263193 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd02c0a21e0 0x7fd02c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:02.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.900+0000 7fd04259c700 1 --2- 172.21.15.60:0/4124263193 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd02c0a21e0 0x7fd02c0a46a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd034005950 tx=0x7fd03400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:02.904+0000 7fd0337fe700 1 -- 172.21.15.60:0/4124263193 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd03c098c30 con 0x7fd044107d90 2024-02-26T04:57:03.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:03.118+0000 7fd048d20700 1 -- 172.21.15.60:0/4124263193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fd04419d910 con 0x7fd044107d90 2024-02-26T04:57:03.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:03.118+0000 7fd0337fe700 1 -- 172.21.15.60:0/4124263193 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fd03c09c2f0 con 0x7fd044107d90 2024-02-26T04:57:03.122 INFO:teuthology.orchestra.run.smithi060.stdout:309237645349 2024-02-26T04:57:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:03.121+0000 7fd048d20700 1 -- 172.21.15.60:0/4124263193 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd02c0a21e0 msgr2=0x7fd02c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:03.121+0000 7fd048d20700 1 --2- 172.21.15.60:0/4124263193 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd02c0a21e0 0x7fd02c0a46a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd034005950 tx=0x7fd03400b410 comp rx=0 tx=0).stop 2024-02-26T04:57:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:03.121+0000 7fd048d20700 1 -- 172.21.15.60:0/4124263193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd044107d90 msgr2=0x7fd04419cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:03.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:03.121+0000 7fd048d20700 1 --2- 172.21.15.60:0/4124263193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd044107d90 0x7fd04419cad0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00b070 tx=0x7fd03c004e40 comp rx=0 tx=0).stop 2024-02-26T04:57:03.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:03.121+0000 7fd048d20700 1 -- 172.21.15.60:0/4124263193 shutdown_connections 2024-02-26T04:57:03.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:03.121+0000 7fd048d20700 1 --2- 172.21.15.60:0/4124263193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd044107d90 0x7fd04419cad0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:03.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:03.121+0000 7fd048d20700 1 --2- 172.21.15.60:0/4124263193 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd02c0a21e0 0x7fd02c0a46a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:03.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:03.121+0000 7fd048d20700 1 --2- 172.21.15.60:0/4124263193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd044101d20 0x7fd044198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:03.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:03.121+0000 7fd048d20700 1 --2- 172.21.15.60:0/4124263193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0441026e0 0x7fd044198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:03.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:03.121+0000 7fd048d20700 1 -- 172.21.15.60:0/4124263193 >> 172.21.15.60:0/4124263193 conn(0x7fd044078d40 msgr2=0x7fd0441006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:03.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:03.121+0000 7fd048d20700 1 -- 172.21.15.60:0/4124263193 shutdown_connections 2024-02-26T04:57:03.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:03.121+0000 7fd048d20700 1 -- 172.21.15.60:0/4124263193 wait complete. 2024-02-26T04:57:04.048 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645342 got 309237645349 for osd.10 2024-02-26T04:57:04.048 DEBUG:teuthology.parallel:result is None 2024-02-26T04:57:04.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:03 smithi191 ceph-mon[36352]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:57:04.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:03 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:57:04.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:03 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:57:04.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:03 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4124263193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-26T04:57:04.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:03 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:57:04.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:03 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:57:04.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:03 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:57:04.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:03 smithi060 ceph-mon[32382]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:57:04.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:03 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:57:04.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:03 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:57:04.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:03 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4124263193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-26T04:57:04.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:03 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:57:04.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:03 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:57:04.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:03 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:57:04.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.266+0000 7ffa65b64700 1 -- 172.21.15.60:0/1180807085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa6006a410 msgr2=0x7ffa6006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:04.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.266+0000 7ffa65b64700 1 --2- 172.21.15.60:0/1180807085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa6006a410 0x7ffa6006a7f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009990 tx=0x7ffa50009ca0 comp rx=0 tx=0).stop 2024-02-26T04:57:04.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.266+0000 7ffa65b64700 1 -- 172.21.15.60:0/1180807085 shutdown_connections 2024-02-26T04:57:04.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.266+0000 7ffa65b64700 1 --2- 172.21.15.60:0/1180807085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa60106ff0 0x7ffa6010eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.266+0000 7ffa65b64700 1 --2- 172.21.15.60:0/1180807085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa6006a410 0x7ffa6006a7f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.266+0000 7ffa65b64700 1 --2- 172.21.15.60:0/1180807085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa6006adc0 0x7ffa601068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.266+0000 7ffa65b64700 1 -- 172.21.15.60:0/1180807085 >> 172.21.15.60:0/1180807085 conn(0x7ffa60076bc0 msgr2=0x7ffa60076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.266+0000 7ffa65b64700 1 -- 172.21.15.60:0/1180807085 shutdown_connections 2024-02-26T04:57:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.266+0000 7ffa65b64700 1 -- 172.21.15.60:0/1180807085 wait complete. 2024-02-26T04:57:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.267+0000 7ffa65b64700 1 Processor -- start 2024-02-26T04:57:04.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.267+0000 7ffa65b64700 1 -- start start 2024-02-26T04:57:04.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa65b64700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa6006a410 0x7ffa60100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:04.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa65b64700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa6006adc0 0x7ffa601011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:04.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa65b64700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa60106ff0 0x7ffa601016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:04.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa65b64700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa601a6610 con 0x7ffa6006adc0 2024-02-26T04:57:04.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa65b64700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa601a6780 con 0x7ffa6006a410 2024-02-26T04:57:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa65b64700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa601a6940 con 0x7ffa60106ff0 2024-02-26T04:57:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa5effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa6006adc0 0x7ffa601011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa5effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa6006adc0 0x7ffa601011b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34322/0 (socket says 172.21.15.60:34322) 2024-02-26T04:57:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa5effd700 1 -- 172.21.15.60:0/2386187385 learned_addr learned my addr 172.21.15.60:0/2386187385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa5ffff700 1 --2- 172.21.15.60:0/2386187385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa60106ff0 0x7ffa601016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa5f7fe700 1 --2- 172.21.15.60:0/2386187385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa6006a410 0x7ffa60100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa5effd700 1 -- 172.21.15.60:0/2386187385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa60106ff0 msgr2=0x7ffa601016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa5effd700 1 --2- 172.21.15.60:0/2386187385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa60106ff0 0x7ffa601016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa5effd700 1 -- 172.21.15.60:0/2386187385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa6006a410 msgr2=0x7ffa60100c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa5effd700 1 --2- 172.21.15.60:0/2386187385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa6006a410 0x7ffa60100c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa5effd700 1 -- 172.21.15.60:0/2386187385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa500095f0 con 0x7ffa6006adc0 2024-02-26T04:57:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.268+0000 7ffa5effd700 1 --2- 172.21.15.60:0/2386187385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa6006adc0 0x7ffa601011b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400c8c0 tx=0x7ffa5400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.269+0000 7ffa5cff9700 1 -- 172.21.15.60:0/2386187385 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54004220 con 0x7ffa6006adc0 2024-02-26T04:57:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.269+0000 7ffa65b64700 1 -- 172.21.15.60:0/2386187385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa601a6c20 con 0x7ffa6006adc0 2024-02-26T04:57:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.269+0000 7ffa5cff9700 1 -- 172.21.15.60:0/2386187385 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa54004d10 con 0x7ffa6006adc0 2024-02-26T04:57:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.269+0000 7ffa5cff9700 1 -- 172.21.15.60:0/2386187385 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54003680 con 0x7ffa6006adc0 2024-02-26T04:57:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.269+0000 7ffa65b64700 1 -- 172.21.15.60:0/2386187385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa601a7120 con 0x7ffa6006adc0 2024-02-26T04:57:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.270+0000 7ffa65b64700 1 -- 172.21.15.60:0/2386187385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6010a1c0 con 0x7ffa6006adc0 2024-02-26T04:57:04.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.272+0000 7ffa5cff9700 1 -- 172.21.15.60:0/2386187385 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ffa54004750 con 0x7ffa6006adc0 2024-02-26T04:57:04.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.272+0000 7ffa5cff9700 1 --2- 172.21.15.60:0/2386187385 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa480a21e0 0x7ffa480a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:04.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.272+0000 7ffa5f7fe700 1 --2- 172.21.15.60:0/2386187385 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa480a21e0 0x7ffa480a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:04.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.272+0000 7ffa5cff9700 1 -- 172.21.15.60:0/2386187385 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ffa540ce020 con 0x7ffa6006adc0 2024-02-26T04:57:04.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.273+0000 7ffa5f7fe700 1 --2- 172.21.15.60:0/2386187385 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa480a21e0 0x7ffa480a46a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000b5c0 tx=0x7ffa50005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:04.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.276+0000 7ffa5cff9700 1 -- 172.21.15.60:0/2386187385 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffa54096b30 con 0x7ffa6006adc0 2024-02-26T04:57:04.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:03 smithi156 ceph-mon[35805]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:57:04.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:03 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:57:04.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:03 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:57:04.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:03 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4124263193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-26T04:57:04.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:03 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:57:04.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:03 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:57:04.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:03 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:57:04.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.481+0000 7ffa65b64700 1 -- 172.21.15.60:0/2386187385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ffa60068930 con 0x7ffa6006adc0 2024-02-26T04:57:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.481+0000 7ffa5cff9700 1 -- 172.21.15.60:0/2386187385 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa5401d090 con 0x7ffa6006adc0 2024-02-26T04:57:04.484 INFO:teuthology.orchestra.run.smithi060.stdout:163208757311 2024-02-26T04:57:04.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.484+0000 7ffa65b64700 1 -- 172.21.15.60:0/2386187385 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa480a21e0 msgr2=0x7ffa480a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:04.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.484+0000 7ffa65b64700 1 --2- 172.21.15.60:0/2386187385 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa480a21e0 0x7ffa480a46a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000b5c0 tx=0x7ffa50005960 comp rx=0 tx=0).stop 2024-02-26T04:57:04.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.484+0000 7ffa65b64700 1 -- 172.21.15.60:0/2386187385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa6006adc0 msgr2=0x7ffa601011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:04.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.484+0000 7ffa65b64700 1 --2- 172.21.15.60:0/2386187385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa6006adc0 0x7ffa601011b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400c8c0 tx=0x7ffa5400cbd0 comp rx=0 tx=0).stop 2024-02-26T04:57:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.484+0000 7ffa65b64700 1 -- 172.21.15.60:0/2386187385 shutdown_connections 2024-02-26T04:57:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.484+0000 7ffa65b64700 1 --2- 172.21.15.60:0/2386187385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa60106ff0 0x7ffa601016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.484+0000 7ffa65b64700 1 --2- 172.21.15.60:0/2386187385 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa480a21e0 0x7ffa480a46a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.484+0000 7ffa65b64700 1 --2- 172.21.15.60:0/2386187385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa6006a410 0x7ffa60100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.484+0000 7ffa65b64700 1 --2- 172.21.15.60:0/2386187385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa6006adc0 0x7ffa601011b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.484+0000 7ffa65b64700 1 -- 172.21.15.60:0/2386187385 >> 172.21.15.60:0/2386187385 conn(0x7ffa60076bc0 msgr2=0x7ffa600fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.484+0000 7ffa65b64700 1 -- 172.21.15.60:0/2386187385 shutdown_connections 2024-02-26T04:57:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:04.484+0000 7ffa65b64700 1 -- 172.21.15.60:0/2386187385 wait complete. 2024-02-26T04:57:05.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:04 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2386187385' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-26T04:57:05.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:04 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2386187385' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-26T04:57:05.318 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757306 got 163208757311 for osd.5 2024-02-26T04:57:05.319 DEBUG:teuthology.parallel:result is None 2024-02-26T04:57:05.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:04 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2386187385' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-26T04:57:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.544+0000 7f7477c55700 1 -- 172.21.15.60:0/463740056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7470105e90 msgr2=0x7f747010da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.544+0000 7f7477c55700 1 --2- 172.21.15.60:0/463740056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7470105e90 0x7f747010da60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f746800b0a0 tx=0x7f746800b3b0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.544+0000 7f7477c55700 1 -- 172.21.15.60:0/463740056 shutdown_connections 2024-02-26T04:57:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.544+0000 7f7477c55700 1 --2- 172.21.15.60:0/463740056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7470105e90 0x7f747010da60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.544+0000 7f7477c55700 1 --2- 172.21.15.60:0/463740056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7470100e00 0x7f74701011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.544+0000 7f7477c55700 1 --2- 172.21.15.60:0/463740056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74701017b0 0x7f74701057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.544+0000 7f7477c55700 1 -- 172.21.15.60:0/463740056 >> 172.21.15.60:0/463740056 conn(0x7f7470076b90 msgr2=0x7f7470076fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.544+0000 7f7477c55700 1 -- 172.21.15.60:0/463740056 shutdown_connections 2024-02-26T04:57:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.544+0000 7f7477c55700 1 -- 172.21.15.60:0/463740056 wait complete. 2024-02-26T04:57:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.544+0000 7f7477c55700 1 Processor -- start 2024-02-26T04:57:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f7477c55700 1 -- start start 2024-02-26T04:57:05.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f7477c55700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7470100e00 0x7f7470199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:05.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f7477c55700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74701017b0 0x7f7470199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:05.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f7477c55700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7470105e90 0x7f74701a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:05.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f7477c55700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7470199f70 con 0x7f7470100e00 2024-02-26T04:57:05.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f7477c55700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74701a46f0 con 0x7f7470105e90 2024-02-26T04:57:05.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f7477c55700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74701a4860 con 0x7f74701017b0 2024-02-26T04:57:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f74759f1700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7470100e00 0x7f7470199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f74759f1700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7470100e00 0x7f7470199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34348/0 (socket says 172.21.15.60:34348) 2024-02-26T04:57:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f74759f1700 1 -- 172.21.15.60:0/1873828802 learned_addr learned my addr 172.21.15.60:0/1873828802 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f74751f0700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74701017b0 0x7f7470199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f74761f2700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7470105e90 0x7f74701a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f74759f1700 1 -- 172.21.15.60:0/1873828802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74701017b0 msgr2=0x7f7470199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f74759f1700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74701017b0 0x7f7470199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f74759f1700 1 -- 172.21.15.60:0/1873828802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7470105e90 msgr2=0x7f74701a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f74759f1700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7470105e90 0x7f74701a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f74759f1700 1 -- 172.21.15.60:0/1873828802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7468009d00 con 0x7f7470100e00 2024-02-26T04:57:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f74761f2700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7470105e90 0x7f74701a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f74759f1700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7470100e00 0x7f7470199130 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f746c00d980 tx=0x7f746c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.545+0000 7f7466ffd700 1 -- 172.21.15.60:0/1873828802 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c004a30 con 0x7f7470100e00 2024-02-26T04:57:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.546+0000 7f7477c55700 1 -- 172.21.15.60:0/1873828802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74701a4b40 con 0x7f7470100e00 2024-02-26T04:57:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.546+0000 7f7466ffd700 1 -- 172.21.15.60:0/1873828802 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f746c005500 con 0x7f7470100e00 2024-02-26T04:57:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.546+0000 7f7466ffd700 1 -- 172.21.15.60:0/1873828802 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c009d30 con 0x7f7470100e00 2024-02-26T04:57:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.546+0000 7f7477c55700 1 -- 172.21.15.60:0/1873828802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74701a5040 con 0x7f7470100e00 2024-02-26T04:57:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.547+0000 7f7477c55700 1 -- 172.21.15.60:0/1873828802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7470109140 con 0x7f7470100e00 2024-02-26T04:57:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.548+0000 7f7466ffd700 1 -- 172.21.15.60:0/1873828802 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f746c004cb0 con 0x7f7470100e00 2024-02-26T04:57:05.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.549+0000 7f7466ffd700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f745c0a21e0 0x7f745c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:05.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.549+0000 7f74751f0700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f745c0a21e0 0x7f745c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:05.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.549+0000 7f7466ffd700 1 -- 172.21.15.60:0/1873828802 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f746c0ce3d0 con 0x7f7470100e00 2024-02-26T04:57:05.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.549+0000 7f74751f0700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f745c0a21e0 0x7f745c0a46a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f74701073c0 tx=0x7f746000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:05.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.552+0000 7f7466ffd700 1 -- 172.21.15.60:0/1873828802 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f746c096ee0 con 0x7f7470100e00 2024-02-26T04:57:05.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.756+0000 7f7477c55700 1 -- 172.21.15.60:0/1873828802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f747006a9c0 con 0x7f7470100e00 2024-02-26T04:57:05.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.757+0000 7f7466ffd700 1 -- 172.21.15.60:0/1873828802 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f746c017070 con 0x7f7470100e00 2024-02-26T04:57:05.759 INFO:teuthology.orchestra.run.smithi060.stdout:253403070513 2024-02-26T04:57:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.759+0000 7f7477c55700 1 -- 172.21.15.60:0/1873828802 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f745c0a21e0 msgr2=0x7f745c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.759+0000 7f7477c55700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f745c0a21e0 0x7f745c0a46a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f74701073c0 tx=0x7f746000b410 comp rx=0 tx=0).stop 2024-02-26T04:57:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.759+0000 7f7477c55700 1 -- 172.21.15.60:0/1873828802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7470100e00 msgr2=0x7f7470199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.759+0000 7f7477c55700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7470100e00 0x7f7470199130 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f746c00d980 tx=0x7f746c00dc90 comp rx=0 tx=0).stop 2024-02-26T04:57:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.759+0000 7f7477c55700 1 -- 172.21.15.60:0/1873828802 shutdown_connections 2024-02-26T04:57:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.759+0000 7f7477c55700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f745c0a21e0 0x7f745c0a46a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.759+0000 7f7477c55700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7470105e90 0x7f74701a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.759+0000 7f7477c55700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7470100e00 0x7f7470199130 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.759+0000 7f7477c55700 1 --2- 172.21.15.60:0/1873828802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74701017b0 0x7f7470199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.759+0000 7f7477c55700 1 -- 172.21.15.60:0/1873828802 >> 172.21.15.60:0/1873828802 conn(0x7f7470076b90 msgr2=0x7f74700fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.759+0000 7f7477c55700 1 -- 172.21.15.60:0/1873828802 shutdown_connections 2024-02-26T04:57:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.759+0000 7f7477c55700 1 -- 172.21.15.60:0/1873828802 wait complete. 2024-02-26T04:57:05.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.776+0000 7f8aad031700 1 -- 172.21.15.60:0/1952607532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa8100260 msgr2=0x7f8aa81006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:05.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.776+0000 7f8aad031700 1 --2- 172.21.15.60:0/1952607532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa8100260 0x7f8aa81006e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009990 tx=0x7f8a90009ca0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.776+0000 7f8aad031700 1 -- 172.21.15.60:0/1952607532 shutdown_connections 2024-02-26T04:57:05.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.776+0000 7f8aad031700 1 --2- 172.21.15.60:0/1952607532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8aa8100c20 0x7f8aa810eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.776+0000 7f8aad031700 1 --2- 172.21.15.60:0/1952607532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aa8106ce0 0x7f8aa81070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.776+0000 7f8aad031700 1 --2- 172.21.15.60:0/1952607532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa8100260 0x7f8aa81006e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.776+0000 7f8aad031700 1 -- 172.21.15.60:0/1952607532 >> 172.21.15.60:0/1952607532 conn(0x7f8aa8077170 msgr2=0x7f8aa80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:05.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.776+0000 7f8aad031700 1 -- 172.21.15.60:0/1952607532 shutdown_connections 2024-02-26T04:57:05.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.776+0000 7f8aad031700 1 -- 172.21.15.60:0/1952607532 wait complete. 2024-02-26T04:57:05.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.777+0000 7f8aad031700 1 Processor -- start 2024-02-26T04:57:05.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.777+0000 7f8aad031700 1 -- start start 2024-02-26T04:57:05.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.777+0000 7f8aad031700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa8100260 0x7f8aa81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:05.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.777+0000 7f8aad031700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aa8100c20 0x7f8aa81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:05.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.777+0000 7f8aad031700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8aa8106ce0 0x7f8aa819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:05.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.777+0000 7f8aad031700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa819d130 con 0x7f8aa8100260 2024-02-26T04:57:05.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.777+0000 7f8aad031700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa819d2a0 con 0x7f8aa8100c20 2024-02-26T04:57:05.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.777+0000 7f8aad031700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa819d460 con 0x7f8aa8106ce0 2024-02-26T04:57:05.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.777+0000 7f8aa6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa8100260 0x7f8aa81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:05.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.777+0000 7f8aa6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa8100260 0x7f8aa81982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34368/0 (socket says 172.21.15.60:34368) 2024-02-26T04:57:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.777+0000 7f8aa6d9d700 1 -- 172.21.15.60:0/2094939238 learned_addr learned my addr 172.21.15.60:0/2094939238 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.778+0000 7f8aa659c700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aa8100c20 0x7f8aa81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.778+0000 7f8aa759e700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8aa8106ce0 0x7f8aa819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.778+0000 7f8aa6d9d700 1 -- 172.21.15.60:0/2094939238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8aa8106ce0 msgr2=0x7f8aa819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.778+0000 7f8aa6d9d700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8aa8106ce0 0x7f8aa819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.778+0000 7f8aa6d9d700 1 -- 172.21.15.60:0/2094939238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aa8100c20 msgr2=0x7f8aa81987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.778+0000 7f8aa6d9d700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aa8100c20 0x7f8aa81987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.778+0000 7f8aa6d9d700 1 -- 172.21.15.60:0/2094939238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a900095f0 con 0x7f8aa8100260 2024-02-26T04:57:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.778+0000 7f8aa759e700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8aa8106ce0 0x7f8aa819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.778+0000 7f8aa6d9d700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa8100260 0x7f8aa81982a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800d950 tx=0x7f8a9800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.778+0000 7f8a9ffff700 1 -- 172.21.15.60:0/2094939238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98009940 con 0x7f8aa8100260 2024-02-26T04:57:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.778+0000 7f8aad031700 1 -- 172.21.15.60:0/2094939238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa819d740 con 0x7f8aa8100260 2024-02-26T04:57:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.778+0000 7f8a9ffff700 1 -- 172.21.15.60:0/2094939238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a98010460 con 0x7f8aa8100260 2024-02-26T04:57:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.778+0000 7f8a9ffff700 1 -- 172.21.15.60:0/2094939238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800f620 con 0x7f8aa8100260 2024-02-26T04:57:05.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.778+0000 7f8aad031700 1 -- 172.21.15.60:0/2094939238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa81a4fa0 con 0x7f8aa8100260 2024-02-26T04:57:05.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.779+0000 7f8aad031700 1 -- 172.21.15.60:0/2094939238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa810a2d0 con 0x7f8aa8100260 2024-02-26T04:57:05.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.781+0000 7f8a9ffff700 1 -- 172.21.15.60:0/2094939238 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f8a980105d0 con 0x7f8aa8100260 2024-02-26T04:57:05.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.781+0000 7f8a9ffff700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a940a21e0 0x7f8a940a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:05.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.781+0000 7f8a9ffff700 1 -- 172.21.15.60:0/2094939238 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f8a980ce090 con 0x7f8aa8100260 2024-02-26T04:57:05.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.781+0000 7f8aa659c700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a940a21e0 0x7f8a940a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:05.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.782+0000 7f8aa659c700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a940a21e0 0x7f8a940a46a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000b5c0 tx=0x7f8a90005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:05.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.785+0000 7f8a9ffff700 1 -- 172.21.15.60:0/2094939238 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8a9809a6d0 con 0x7f8aa8100260 2024-02-26T04:57:05.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.988+0000 7f8aad031700 1 -- 172.21.15.60:0/2094939238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f8aa8068990 con 0x7f8aa8100260 2024-02-26T04:57:05.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.989+0000 7f8a9ffff700 1 -- 172.21.15.60:0/2094939238 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a9809a260 con 0x7f8aa8100260 2024-02-26T04:57:05.991 INFO:teuthology.orchestra.run.smithi060.stdout:287762808875 2024-02-26T04:57:05.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.991+0000 7f8aad031700 1 -- 172.21.15.60:0/2094939238 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a940a21e0 msgr2=0x7f8a940a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:05.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.991+0000 7f8aad031700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a940a21e0 0x7f8a940a46a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000b5c0 tx=0x7f8a90005960 comp rx=0 tx=0).stop 2024-02-26T04:57:05.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.991+0000 7f8aad031700 1 -- 172.21.15.60:0/2094939238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa8100260 msgr2=0x7f8aa81982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:05.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.991+0000 7f8aad031700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa8100260 0x7f8aa81982a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800d950 tx=0x7f8a9800dc60 comp rx=0 tx=0).stop 2024-02-26T04:57:05.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.991+0000 7f8aad031700 1 -- 172.21.15.60:0/2094939238 shutdown_connections 2024-02-26T04:57:05.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.991+0000 7f8aad031700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8aa8106ce0 0x7f8aa819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.991+0000 7f8aad031700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa8100260 0x7f8aa81982a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.991+0000 7f8aad031700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a940a21e0 0x7f8a940a46a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.991+0000 7f8aad031700 1 --2- 172.21.15.60:0/2094939238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aa8100c20 0x7f8aa81987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:05.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.991+0000 7f8aad031700 1 -- 172.21.15.60:0/2094939238 >> 172.21.15.60:0/2094939238 conn(0x7f8aa8077170 msgr2=0x7f8aa8104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:05.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.991+0000 7f8aad031700 1 -- 172.21.15.60:0/2094939238 shutdown_connections 2024-02-26T04:57:05.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:05.991+0000 7f8aad031700 1 -- 172.21.15.60:0/2094939238 wait complete. 2024-02-26T04:57:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:05 smithi191 ceph-mon[36352]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:05 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1873828802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-26T04:57:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:05 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2094939238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-26T04:57:06.355 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:05 smithi060 ceph-mon[32382]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:06.355 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:05 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1873828802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-26T04:57:06.355 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:05 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2094939238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-26T04:57:06.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:05 smithi156 ceph-mon[35805]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:06.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:05 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1873828802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-26T04:57:06.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:05 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2094939238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-26T04:57:06.716 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808869 got 287762808875 for osd.9 2024-02-26T04:57:06.717 DEBUG:teuthology.parallel:result is None 2024-02-26T04:57:07.174 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070506 got 253403070513 for osd.8 2024-02-26T04:57:07.174 DEBUG:teuthology.parallel:result is None 2024-02-26T04:57:07.174 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-26T04:57:07.175 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph pg dump --format=json 2024-02-26T04:57:07.428 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:57:08.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:07 smithi060 ceph-mon[32382]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:57:08.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:07 smithi191 ceph-mon[36352]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:57:08.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:07 smithi156 ceph-mon[35805]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:57:09.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.228+0000 7fdd01ee9700 1 -- 172.21.15.60:0/861479006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcfc100b70 msgr2=0x7fdcfc100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:09.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.228+0000 7fdd01ee9700 1 --2- 172.21.15.60:0/861479006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcfc100b70 0x7fdcfc100ff0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009990 tx=0x7fdce4009ca0 comp rx=0 tx=0).stop 2024-02-26T04:57:09.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.228+0000 7fdd01ee9700 1 -- 172.21.15.60:0/861479006 shutdown_connections 2024-02-26T04:57:09.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.228+0000 7fdd01ee9700 1 --2- 172.21.15.60:0/861479006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcfc101530 0x7fdcfc10d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:09.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.228+0000 7fdd01ee9700 1 --2- 172.21.15.60:0/861479006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcfc106b80 0x7fdcfc106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:09.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.228+0000 7fdd01ee9700 1 --2- 172.21.15.60:0/861479006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcfc100b70 0x7fdcfc100ff0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:09.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.228+0000 7fdd01ee9700 1 -- 172.21.15.60:0/861479006 >> 172.21.15.60:0/861479006 conn(0x7fdcfc077aa0 msgr2=0x7fdcfc077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:09.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.228+0000 7fdd01ee9700 1 -- 172.21.15.60:0/861479006 shutdown_connections 2024-02-26T04:57:09.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.228+0000 7fdd01ee9700 1 -- 172.21.15.60:0/861479006 wait complete. 2024-02-26T04:57:09.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.229+0000 7fdd01ee9700 1 Processor -- start 2024-02-26T04:57:09.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.229+0000 7fdd01ee9700 1 -- start start 2024-02-26T04:57:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.229+0000 7fdd01ee9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcfc100b70 0x7fdcfc198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdd01ee9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcfc101530 0x7fdcfc198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdd01ee9700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcfc106b80 0x7fdcfc19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdd01ee9700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc19cff0 con 0x7fdcfc106b80 2024-02-26T04:57:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdd01ee9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc19d160 con 0x7fdcfc100b70 2024-02-26T04:57:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdd01ee9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc19d320 con 0x7fdcfc101530 2024-02-26T04:57:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdcfbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcfc106b80 0x7fdcfc19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdcfbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcfc106b80 0x7fdcfc19c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34388/0 (socket says 172.21.15.60:34388) 2024-02-26T04:57:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdcfbfff700 1 -- 172.21.15.60:0/198511381 learned_addr learned my addr 172.21.15.60:0/198511381 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdcfb7fe700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcfc100b70 0x7fdcfc198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdcfaffd700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcfc101530 0x7fdcfc198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdcfbfff700 1 -- 172.21.15.60:0/198511381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcfc101530 msgr2=0x7fdcfc198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdcfbfff700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcfc101530 0x7fdcfc198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdcfbfff700 1 -- 172.21.15.60:0/198511381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcfc100b70 msgr2=0x7fdcfc198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdcfbfff700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcfc100b70 0x7fdcfc198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdcfbfff700 1 -- 172.21.15.60:0/198511381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce40095f0 con 0x7fdcfc106b80 2024-02-26T04:57:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdcfb7fe700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcfc100b70 0x7fdcfc198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:57:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdcfbfff700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcfc106b80 0x7fdcfc19c9e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00d980 tx=0x7fdcec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.230+0000 7fdcf8ff9700 1 -- 172.21.15.60:0/198511381 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec004220 con 0x7fdcfc106b80 2024-02-26T04:57:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.231+0000 7fdd01ee9700 1 -- 172.21.15.60:0/198511381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc19d600 con 0x7fdcfc106b80 2024-02-26T04:57:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.231+0000 7fdcf8ff9700 1 -- 172.21.15.60:0/198511381 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdcec0075e0 con 0x7fdcfc106b80 2024-02-26T04:57:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.231+0000 7fdcf8ff9700 1 -- 172.21.15.60:0/198511381 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec00f690 con 0x7fdcfc106b80 2024-02-26T04:57:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.231+0000 7fdd01ee9700 1 -- 172.21.15.60:0/198511381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc1a4e60 con 0x7fdcfc106b80 2024-02-26T04:57:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.232+0000 7fdd01ee9700 1 -- 172.21.15.60:0/198511381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc109020 con 0x7fdcfc106b80 2024-02-26T04:57:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.233+0000 7fdcf8ff9700 1 -- 172.21.15.60:0/198511381 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fdcec004410 con 0x7fdcfc106b80 2024-02-26T04:57:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.233+0000 7fdcf8ff9700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdce80a21e0 0x7fdce80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.234+0000 7fdcf8ff9700 1 -- 172.21.15.60:0/198511381 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fdcec0ce470 con 0x7fdcfc106b80 2024-02-26T04:57:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.234+0000 7fdcfb7fe700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdce80a21e0 0x7fdce80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.234+0000 7fdcfb7fe700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdce80a21e0 0x7fdce80a46a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009550 tx=0x7fdcf40091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:09.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.237+0000 7fdcf8ff9700 1 -- 172.21.15.60:0/198511381 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdcec096f20 con 0x7fdcfc106b80 2024-02-26T04:57:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.435+0000 7fdd01ee9700 1 -- 172.21.15.60:0/198511381 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdcfc199510 con 0x7fdce80a21e0 2024-02-26T04:57:09.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.448+0000 7fdcf8ff9700 1 -- 172.21.15.60:0/198511381 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427955 (secure 0 0 0) 0x7fdcfc199510 con 0x7fdce80a21e0 2024-02-26T04:57:09.457 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:09.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.452+0000 7fdd01ee9700 1 -- 172.21.15.60:0/198511381 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdce80a21e0 msgr2=0x7fdce80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:09.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.452+0000 7fdd01ee9700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdce80a21e0 0x7fdce80a46a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009550 tx=0x7fdcf40091b0 comp rx=0 tx=0).stop 2024-02-26T04:57:09.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.452+0000 7fdd01ee9700 1 -- 172.21.15.60:0/198511381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcfc106b80 msgr2=0x7fdcfc19c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.452+0000 7fdd01ee9700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcfc106b80 0x7fdcfc19c9e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00d980 tx=0x7fdcec00dc90 comp rx=0 tx=0).stop 2024-02-26T04:57:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.452+0000 7fdd01ee9700 1 -- 172.21.15.60:0/198511381 shutdown_connections 2024-02-26T04:57:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.452+0000 7fdd01ee9700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdcfc106b80 0x7fdcfc19c9e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.452+0000 7fdd01ee9700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fdce80a21e0 0x7fdce80a46a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.452+0000 7fdd01ee9700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdcfc100b70 0x7fdcfc198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.452+0000 7fdd01ee9700 1 --2- 172.21.15.60:0/198511381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcfc101530 0x7fdcfc198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.452+0000 7fdd01ee9700 1 -- 172.21.15.60:0/198511381 >> 172.21.15.60:0/198511381 conn(0x7fdcfc077aa0 msgr2=0x7fdcfc0ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.453+0000 7fdd01ee9700 1 -- 172.21.15.60:0/198511381 shutdown_connections 2024-02-26T04:57:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:09.453+0000 7fdd01ee9700 1 -- 172.21.15.60:0/198511381 wait complete. 2024-02-26T04:57:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-02-26T04:57:10.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:10 smithi191 ceph-mon[36352]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:10.320 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:10 smithi060 ceph-mon[32382]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:10.354 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-02-26T04:57:08.332022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657568,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":583,"num_read_kb":409,"num_write":343,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386324,"kb_used_data":12152,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555500,"statfs":{"total":1151940427776,"available":1151544832000,"internally_reserved":0,"allocated":12443648,"data_stored":7958318,"data_compressed":358926,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003457"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508738+0000","last_change":"2024-02-26T04:55:28.208832+0000","last_active":"2024-02-26T04:56:20.508738+0000","last_peered":"2024-02-26T04:56:20.508738+0000","last_clean":"2024-02-26T04:56:20.508738+0000","last_became_active":"2024-02-26T04:55:28.208454+0000","last_became_peered":"2024-02-26T04:55:28.208454+0000","last_unstale":"2024-02-26T04:56:20.508738+0000","last_undegraded":"2024-02-26T04:56:20.508738+0000","last_fullsized":"2024-02-26T04:56:20.508738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:55:54.132384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201989+0000","last_change":"2024-02-26T04:55:28.203876+0000","last_active":"2024-02-26T04:56:20.201989+0000","last_peered":"2024-02-26T04:56:20.201989+0000","last_clean":"2024-02-26T04:56:20.201989+0000","last_became_active":"2024-02-26T04:55:28.203399+0000","last_became_peered":"2024-02-26T04:55:28.203399+0000","last_unstale":"2024-02-26T04:56:20.201989+0000","last_undegraded":"2024-02-26T04:56:20.201989+0000","last_fullsized":"2024-02-26T04:56:20.201989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:31:40.927929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200636+0000","last_change":"2024-02-26T04:55:28.211388+0000","last_active":"2024-02-26T04:56:20.200636+0000","last_peered":"2024-02-26T04:56:20.200636+0000","last_clean":"2024-02-26T04:56:20.200636+0000","last_became_active":"2024-02-26T04:55:28.209007+0000","last_became_peered":"2024-02-26T04:55:28.209007+0000","last_unstale":"2024-02-26T04:56:20.200636+0000","last_undegraded":"2024-02-26T04:56:20.200636+0000","last_fullsized":"2024-02-26T04:56:20.200636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:37:01.197589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.198030+0000","last_change":"2024-02-26T04:55:28.216378+0000","last_active":"2024-02-26T04:56:20.198030+0000","last_peered":"2024-02-26T04:56:20.198030+0000","last_clean":"2024-02-26T04:56:20.198030+0000","last_became_active":"2024-02-26T04:55:28.216074+0000","last_became_peered":"2024-02-26T04:55:28.216074+0000","last_unstale":"2024-02-26T04:56:20.198030+0000","last_undegraded":"2024-02-26T04:56:20.198030+0000","last_fullsized":"2024-02-26T04:56:20.198030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:15:20.460699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197626+0000","last_change":"2024-02-26T04:55:28.213026+0000","last_active":"2024-02-26T04:56:20.197626+0000","last_peered":"2024-02-26T04:56:20.197626+0000","last_clean":"2024-02-26T04:56:20.197626+0000","last_became_active":"2024-02-26T04:55:28.212706+0000","last_became_peered":"2024-02-26T04:55:28.212706+0000","last_unstale":"2024-02-26T04:56:20.197626+0000","last_undegraded":"2024-02-26T04:56:20.197626+0000","last_fullsized":"2024-02-26T04:56:20.197626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:57:35.961956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.206795+0000","last_change":"2024-02-26T04:55:28.202123+0000","last_active":"2024-02-26T04:56:20.206795+0000","last_peered":"2024-02-26T04:56:20.206795+0000","last_clean":"2024-02-26T04:56:20.206795+0000","last_became_active":"2024-02-26T04:55:28.201733+0000","last_became_peered":"2024-02-26T04:55:28.201733+0000","last_unstale":"2024-02-26T04:56:20.206795+0000","last_undegraded":"2024-02-26T04:56:20.206795+0000","last_fullsized":"2024-02-26T04:56:20.206795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:43:11.300006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508346+0000","last_change":"2024-02-26T04:56:20.508346+0000","last_active":"2024-02-26T04:56:20.508346+0000","last_peered":"2024-02-26T04:56:20.508346+0000","last_clean":"2024-02-26T04:56:20.508346+0000","last_became_active":"2024-02-26T04:56:20.507941+0000","last_became_peered":"2024-02-26T04:56:20.507941+0000","last_unstale":"2024-02-26T04:56:20.508346+0000","last_undegraded":"2024-02-26T04:56:20.508346+0000","last_fullsized":"2024-02-26T04:56:20.508346+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:35:27.425477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197071+0000","last_change":"2024-02-26T04:55:28.214631+0000","last_active":"2024-02-26T04:56:20.197071+0000","last_peered":"2024-02-26T04:56:20.197071+0000","last_clean":"2024-02-26T04:56:20.197071+0000","last_became_active":"2024-02-26T04:55:28.214290+0000","last_became_peered":"2024-02-26T04:55:28.214290+0000","last_unstale":"2024-02-26T04:56:20.197071+0000","last_undegraded":"2024-02-26T04:56:20.197071+0000","last_fullsized":"2024-02-26T04:56:20.197071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:30:11.494833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508783+0000","last_change":"2024-02-26T04:56:20.508783+0000","last_active":"2024-02-26T04:56:20.508783+0000","last_peered":"2024-02-26T04:56:20.508783+0000","last_clean":"2024-02-26T04:56:20.508783+0000","last_became_active":"2024-02-26T04:56:20.508372+0000","last_became_peered":"2024-02-26T04:56:20.508372+0000","last_unstale":"2024-02-26T04:56:20.508783+0000","last_undegraded":"2024-02-26T04:56:20.508783+0000","last_fullsized":"2024-02-26T04:56:20.508783+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:32.415022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.207364+0000","last_change":"2024-02-26T04:55:28.204176+0000","last_active":"2024-02-26T04:56:20.207364+0000","last_peered":"2024-02-26T04:56:20.207364+0000","last_clean":"2024-02-26T04:56:20.207364+0000","last_became_active":"2024-02-26T04:55:28.203821+0000","last_became_peered":"2024-02-26T04:55:28.203821+0000","last_unstale":"2024-02-26T04:56:20.207364+0000","last_undegraded":"2024-02-26T04:56:20.207364+0000","last_fullsized":"2024-02-26T04:56:20.207364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:24:03.132650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199686+0000","last_change":"2024-02-26T04:55:28.215958+0000","last_active":"2024-02-26T04:56:20.199686+0000","last_peered":"2024-02-26T04:56:20.199686+0000","last_clean":"2024-02-26T04:56:20.199686+0000","last_became_active":"2024-02-26T04:55:28.215594+0000","last_became_peered":"2024-02-26T04:55:28.215594+0000","last_unstale":"2024-02-26T04:56:20.199686+0000","last_undegraded":"2024-02-26T04:56:20.199686+0000","last_fullsized":"2024-02-26T04:56:20.199686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:45:47.915379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197890+0000","last_change":"2024-02-26T04:55:28.218392+0000","last_active":"2024-02-26T04:56:20.197890+0000","last_peered":"2024-02-26T04:56:20.197890+0000","last_clean":"2024-02-26T04:56:20.197890+0000","last_became_active":"2024-02-26T04:55:28.217933+0000","last_became_peered":"2024-02-26T04:55:28.217933+0000","last_unstale":"2024-02-26T04:56:20.197890+0000","last_undegraded":"2024-02-26T04:56:20.197890+0000","last_fullsized":"2024-02-26T04:56:20.197890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:30:38.880649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507627+0000","last_change":"2024-02-26T04:55:28.197099+0000","last_active":"2024-02-26T04:56:20.507627+0000","last_peered":"2024-02-26T04:56:20.507627+0000","last_clean":"2024-02-26T04:56:20.507627+0000","last_became_active":"2024-02-26T04:55:28.196755+0000","last_became_peered":"2024-02-26T04:55:28.196755+0000","last_unstale":"2024-02-26T04:56:20.507627+0000","last_undegraded":"2024-02-26T04:56:20.507627+0000","last_fullsized":"2024-02-26T04:56:20.507627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:37:31.685845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199949+0000","last_change":"2024-02-26T04:55:28.212064+0000","last_active":"2024-02-26T04:56:20.199949+0000","last_peered":"2024-02-26T04:56:20.199949+0000","last_clean":"2024-02-26T04:56:20.199949+0000","last_became_active":"2024-02-26T04:55:28.208742+0000","last_became_peered":"2024-02-26T04:55:28.208742+0000","last_unstale":"2024-02-26T04:56:20.199949+0000","last_undegraded":"2024-02-26T04:56:20.199949+0000","last_fullsized":"2024-02-26T04:56:20.199949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:38:19.742468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.511014+0000","last_change":"2024-02-26T04:56:20.511014+0000","last_active":"2024-02-26T04:56:20.511014+0000","last_peered":"2024-02-26T04:56:20.511014+0000","last_clean":"2024-02-26T04:56:20.511014+0000","last_became_active":"2024-02-26T04:56:20.510463+0000","last_became_peered":"2024-02-26T04:56:20.510463+0000","last_unstale":"2024-02-26T04:56:20.511014+0000","last_undegraded":"2024-02-26T04:56:20.511014+0000","last_fullsized":"2024-02-26T04:56:20.511014+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:44:59.574588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506436+0000","last_change":"2024-02-26T04:55:28.222810+0000","last_active":"2024-02-26T04:56:20.506436+0000","last_peered":"2024-02-26T04:56:20.506436+0000","last_clean":"2024-02-26T04:56:20.506436+0000","last_became_active":"2024-02-26T04:55:28.222341+0000","last_became_peered":"2024-02-26T04:55:28.222341+0000","last_unstale":"2024-02-26T04:56:20.506436+0000","last_undegraded":"2024-02-26T04:56:20.506436+0000","last_fullsized":"2024-02-26T04:56:20.506436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:59:21.627517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508612+0000","last_change":"2024-02-26T04:55:28.208377+0000","last_active":"2024-02-26T04:56:20.508612+0000","last_peered":"2024-02-26T04:56:20.508612+0000","last_clean":"2024-02-26T04:56:20.508612+0000","last_became_active":"2024-02-26T04:55:28.207932+0000","last_became_peered":"2024-02-26T04:55:28.207932+0000","last_unstale":"2024-02-26T04:56:20.508612+0000","last_undegraded":"2024-02-26T04:56:20.508612+0000","last_fullsized":"2024-02-26T04:56:20.508612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:15:39.586191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200068+0000","last_change":"2024-02-26T04:55:28.201105+0000","last_active":"2024-02-26T04:56:20.200068+0000","last_peered":"2024-02-26T04:56:20.200068+0000","last_clean":"2024-02-26T04:56:20.200068+0000","last_became_active":"2024-02-26T04:55:28.200796+0000","last_became_peered":"2024-02-26T04:55:28.200796+0000","last_unstale":"2024-02-26T04:56:20.200068+0000","last_undegraded":"2024-02-26T04:56:20.200068+0000","last_fullsized":"2024-02-26T04:56:20.200068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:53:43.713055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.509323+0000","last_change":"2024-02-26T04:55:28.215725+0000","last_active":"2024-02-26T04:56:20.509323+0000","last_peered":"2024-02-26T04:56:20.509323+0000","last_clean":"2024-02-26T04:56:20.509323+0000","last_became_active":"2024-02-26T04:55:28.215341+0000","last_became_peered":"2024-02-26T04:55:28.215341+0000","last_unstale":"2024-02-26T04:56:20.509323+0000","last_undegraded":"2024-02-26T04:56:20.509323+0000","last_fullsized":"2024-02-26T04:56:20.509323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:26:26.960474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506987+0000","last_change":"2024-02-26T04:55:32.233303+0000","last_active":"2024-02-26T04:56:20.506987+0000","last_peered":"2024-02-26T04:56:20.506987+0000","last_clean":"2024-02-26T04:56:20.506987+0000","last_became_active":"2024-02-26T04:55:32.232901+0000","last_became_peered":"2024-02-26T04:55:32.232901+0000","last_unstale":"2024-02-26T04:56:20.506987+0000","last_undegraded":"2024-02-26T04:56:20.506987+0000","last_fullsized":"2024-02-26T04:56:20.506987+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:37:49.659668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201603+0000","last_change":"2024-02-26T04:55:27.199682+0000","last_active":"2024-02-26T04:56:20.201603+0000","last_peered":"2024-02-26T04:56:20.201603+0000","last_clean":"2024-02-26T04:56:20.201603+0000","last_became_active":"2024-02-26T04:55:27.199293+0000","last_became_peered":"2024-02-26T04:55:27.199293+0000","last_unstale":"2024-02-26T04:56:20.201603+0000","last_undegraded":"2024-02-26T04:56:20.201603+0000","last_fullsized":"2024-02-26T04:56:20.201603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:23:21.685615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.511029+0000","last_change":"2024-02-26T04:56:20.511029+0000","last_active":"2024-02-26T04:56:20.511029+0000","last_peered":"2024-02-26T04:56:20.511029+0000","last_clean":"2024-02-26T04:56:20.511029+0000","last_became_active":"2024-02-26T04:56:20.510590+0000","last_became_peered":"2024-02-26T04:56:20.510590+0000","last_unstale":"2024-02-26T04:56:20.511029+0000","last_undegraded":"2024-02-26T04:56:20.511029+0000","last_fullsized":"2024-02-26T04:56:20.511029+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:01:25.641444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197376+0000","last_change":"2024-02-26T04:55:32.233142+0000","last_active":"2024-02-26T04:56:20.197376+0000","last_peered":"2024-02-26T04:56:20.197376+0000","last_clean":"2024-02-26T04:56:20.197376+0000","last_became_active":"2024-02-26T04:55:32.232722+0000","last_became_peered":"2024-02-26T04:55:32.232722+0000","last_unstale":"2024-02-26T04:56:20.197376+0000","last_undegraded":"2024-02-26T04:56:20.197376+0000","last_fullsized":"2024-02-26T04:56:20.197376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:30:20.906104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197798+0000","last_change":"2024-02-26T04:55:27.203420+0000","last_active":"2024-02-26T04:56:20.197798+0000","last_peered":"2024-02-26T04:56:20.197798+0000","last_clean":"2024-02-26T04:56:20.197798+0000","last_became_active":"2024-02-26T04:55:27.202968+0000","last_became_peered":"2024-02-26T04:55:27.202968+0000","last_unstale":"2024-02-26T04:56:20.197798+0000","last_undegraded":"2024-02-26T04:56:20.197798+0000","last_fullsized":"2024-02-26T04:56:20.197798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:44.496528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201808+0000","last_change":"2024-02-26T04:55:28.217422+0000","last_active":"2024-02-26T04:56:20.201808+0000","last_peered":"2024-02-26T04:56:20.201808+0000","last_clean":"2024-02-26T04:56:20.201808+0000","last_became_active":"2024-02-26T04:55:28.217103+0000","last_became_peered":"2024-02-26T04:55:28.217103+0000","last_unstale":"2024-02-26T04:56:20.201808+0000","last_undegraded":"2024-02-26T04:56:20.201808+0000","last_fullsized":"2024-02-26T04:56:20.201808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:08:37.075467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197565+0000","last_change":"2024-02-26T04:55:32.233322+0000","last_active":"2024-02-26T04:56:20.197565+0000","last_peered":"2024-02-26T04:56:20.197565+0000","last_clean":"2024-02-26T04:56:20.197565+0000","last_became_active":"2024-02-26T04:55:32.232679+0000","last_became_peered":"2024-02-26T04:55:32.232679+0000","last_unstale":"2024-02-26T04:56:20.197565+0000","last_undegraded":"2024-02-26T04:56:20.197565+0000","last_fullsized":"2024-02-26T04:56:20.197565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:10.775924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.198176+0000","last_change":"2024-02-26T04:55:27.205960+0000","last_active":"2024-02-26T04:56:20.198176+0000","last_peered":"2024-02-26T04:56:20.198176+0000","last_clean":"2024-02-26T04:56:20.198176+0000","last_became_active":"2024-02-26T04:55:27.205604+0000","last_became_peered":"2024-02-26T04:55:27.205604+0000","last_unstale":"2024-02-26T04:56:20.198176+0000","last_undegraded":"2024-02-26T04:56:20.198176+0000","last_fullsized":"2024-02-26T04:56:20.198176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:45:25.002672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506983+0000","last_change":"2024-02-26T04:55:28.210669+0000","last_active":"2024-02-26T04:56:20.506983+0000","last_peered":"2024-02-26T04:56:20.506983+0000","last_clean":"2024-02-26T04:56:20.506983+0000","last_became_active":"2024-02-26T04:55:28.210322+0000","last_became_peered":"2024-02-26T04:55:28.210322+0000","last_unstale":"2024-02-26T04:56:20.506983+0000","last_undegraded":"2024-02-26T04:56:20.506983+0000","last_fullsized":"2024-02-26T04:56:20.506983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:57:29.121762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197404+0000","last_change":"2024-02-26T04:55:32.229077+0000","last_active":"2024-02-26T04:56:20.197404+0000","last_peered":"2024-02-26T04:56:20.197404+0000","last_clean":"2024-02-26T04:56:20.197404+0000","last_became_active":"2024-02-26T04:55:32.228670+0000","last_became_peered":"2024-02-26T04:55:32.228670+0000","last_unstale":"2024-02-26T04:56:20.197404+0000","last_undegraded":"2024-02-26T04:56:20.197404+0000","last_fullsized":"2024-02-26T04:56:20.197404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:35:20.843351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197770+0000","last_change":"2024-02-26T04:55:27.206088+0000","last_active":"2024-02-26T04:56:20.197770+0000","last_peered":"2024-02-26T04:56:20.197770+0000","last_clean":"2024-02-26T04:56:20.197770+0000","last_became_active":"2024-02-26T04:55:27.205696+0000","last_became_peered":"2024-02-26T04:55:27.205696+0000","last_unstale":"2024-02-26T04:56:20.197770+0000","last_undegraded":"2024-02-26T04:56:20.197770+0000","last_fullsized":"2024-02-26T04:56:20.197770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:14:03.552251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508805+0000","last_change":"2024-02-26T04:56:20.508805+0000","last_active":"2024-02-26T04:56:20.508805+0000","last_peered":"2024-02-26T04:56:20.508805+0000","last_clean":"2024-02-26T04:56:20.508805+0000","last_became_active":"2024-02-26T04:56:20.507796+0000","last_became_peered":"2024-02-26T04:56:20.507796+0000","last_unstale":"2024-02-26T04:56:20.508805+0000","last_undegraded":"2024-02-26T04:56:20.508805+0000","last_fullsized":"2024-02-26T04:56:20.508805+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:25.453111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507484+0000","last_change":"2024-02-26T04:55:32.228629+0000","last_active":"2024-02-26T04:56:20.507484+0000","last_peered":"2024-02-26T04:56:20.507484+0000","last_clean":"2024-02-26T04:56:20.507484+0000","last_became_active":"2024-02-26T04:55:32.227608+0000","last_became_peered":"2024-02-26T04:55:32.227608+0000","last_unstale":"2024-02-26T04:56:20.507484+0000","last_undegraded":"2024-02-26T04:56:20.507484+0000","last_fullsized":"2024-02-26T04:56:20.507484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:10:42.429352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":106,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:57:04.247897+0000","last_change":"2024-02-26T04:55:27.201261+0000","last_active":"2024-02-26T04:57:04.247897+0000","last_peered":"2024-02-26T04:57:04.247897+0000","last_clean":"2024-02-26T04:57:04.247897+0000","last_became_active":"2024-02-26T04:55:27.200892+0000","last_became_peered":"2024-02-26T04:55:27.200892+0000","last_unstale":"2024-02-26T04:57:04.247897+0000","last_undegraded":"2024-02-26T04:57:04.247897+0000","last_fullsized":"2024-02-26T04:57:04.247897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:34:32.774143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197918+0000","last_change":"2024-02-26T04:55:28.200445+0000","last_active":"2024-02-26T04:56:20.197918+0000","last_peered":"2024-02-26T04:56:20.197918+0000","last_clean":"2024-02-26T04:56:20.197918+0000","last_became_active":"2024-02-26T04:55:28.199959+0000","last_became_peered":"2024-02-26T04:55:28.199959+0000","last_unstale":"2024-02-26T04:56:20.197918+0000","last_undegraded":"2024-02-26T04:56:20.197918+0000","last_fullsized":"2024-02-26T04:56:20.197918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:36:27.397644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506917+0000","last_change":"2024-02-26T04:55:32.232619+0000","last_active":"2024-02-26T04:56:20.506917+0000","last_peered":"2024-02-26T04:56:20.506917+0000","last_clean":"2024-02-26T04:56:20.506917+0000","last_became_active":"2024-02-26T04:55:32.232112+0000","last_became_peered":"2024-02-26T04:55:32.232112+0000","last_unstale":"2024-02-26T04:56:20.506917+0000","last_undegraded":"2024-02-26T04:56:20.506917+0000","last_fullsized":"2024-02-26T04:56:20.506917+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:19.741393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.206715+0000","last_change":"2024-02-26T04:55:27.197376+0000","last_active":"2024-02-26T04:56:20.206715+0000","last_peered":"2024-02-26T04:56:20.206715+0000","last_clean":"2024-02-26T04:56:20.206715+0000","last_became_active":"2024-02-26T04:55:27.195258+0000","last_became_peered":"2024-02-26T04:55:27.195258+0000","last_unstale":"2024-02-26T04:56:20.206715+0000","last_undegraded":"2024-02-26T04:56:20.206715+0000","last_fullsized":"2024-02-26T04:56:20.206715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:41:51.284525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200987+0000","last_change":"2024-02-26T04:55:28.214337+0000","last_active":"2024-02-26T04:56:20.200987+0000","last_peered":"2024-02-26T04:56:20.200987+0000","last_clean":"2024-02-26T04:56:20.200987+0000","last_became_active":"2024-02-26T04:55:28.213966+0000","last_became_peered":"2024-02-26T04:55:28.213966+0000","last_unstale":"2024-02-26T04:56:20.200987+0000","last_undegraded":"2024-02-26T04:56:20.200987+0000","last_fullsized":"2024-02-26T04:56:20.200987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:42.974084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507266+0000","last_change":"2024-02-26T04:55:32.235579+0000","last_active":"2024-02-26T04:56:20.507266+0000","last_peered":"2024-02-26T04:56:20.507266+0000","last_clean":"2024-02-26T04:56:20.507266+0000","last_became_active":"2024-02-26T04:55:32.235241+0000","last_became_peered":"2024-02-26T04:55:32.235241+0000","last_unstale":"2024-02-26T04:56:20.507266+0000","last_undegraded":"2024-02-26T04:56:20.507266+0000","last_fullsized":"2024-02-26T04:56:20.507266+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:06.319874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507236+0000","last_change":"2024-02-26T04:55:27.197689+0000","last_active":"2024-02-26T04:56:20.507236+0000","last_peered":"2024-02-26T04:56:20.507236+0000","last_clean":"2024-02-26T04:56:20.507236+0000","last_became_active":"2024-02-26T04:55:27.197285+0000","last_became_peered":"2024-02-26T04:55:27.197285+0000","last_unstale":"2024-02-26T04:56:20.507236+0000","last_undegraded":"2024-02-26T04:56:20.507236+0000","last_fullsized":"2024-02-26T04:56:20.507236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:45:36.688902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.196871+0000","last_change":"2024-02-26T04:55:28.201902+0000","last_active":"2024-02-26T04:56:20.196871+0000","last_peered":"2024-02-26T04:56:20.196871+0000","last_clean":"2024-02-26T04:56:20.196871+0000","last_became_active":"2024-02-26T04:55:28.201604+0000","last_became_peered":"2024-02-26T04:55:28.201604+0000","last_unstale":"2024-02-26T04:56:20.196871+0000","last_undegraded":"2024-02-26T04:56:20.196871+0000","last_fullsized":"2024-02-26T04:56:20.196871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:48:57.711555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201806+0000","last_change":"2024-02-26T04:55:32.230153+0000","last_active":"2024-02-26T04:56:20.201806+0000","last_peered":"2024-02-26T04:56:20.201806+0000","last_clean":"2024-02-26T04:56:20.201806+0000","last_became_active":"2024-02-26T04:55:32.229695+0000","last_became_peered":"2024-02-26T04:55:32.229695+0000","last_unstale":"2024-02-26T04:56:20.201806+0000","last_undegraded":"2024-02-26T04:56:20.201806+0000","last_fullsized":"2024-02-26T04:56:20.201806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:15:39.575727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200217+0000","last_change":"2024-02-26T04:55:27.200943+0000","last_active":"2024-02-26T04:56:20.200217+0000","last_peered":"2024-02-26T04:56:20.200217+0000","last_clean":"2024-02-26T04:56:20.200217+0000","last_became_active":"2024-02-26T04:55:27.200565+0000","last_became_peered":"2024-02-26T04:55:27.200565+0000","last_unstale":"2024-02-26T04:56:20.200217+0000","last_undegraded":"2024-02-26T04:56:20.200217+0000","last_fullsized":"2024-02-26T04:56:20.200217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:54:19.670631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199990+0000","last_change":"2024-02-26T04:55:28.212232+0000","last_active":"2024-02-26T04:56:20.199990+0000","last_peered":"2024-02-26T04:56:20.199990+0000","last_clean":"2024-02-26T04:56:20.199990+0000","last_became_active":"2024-02-26T04:55:28.211875+0000","last_became_peered":"2024-02-26T04:55:28.211875+0000","last_unstale":"2024-02-26T04:56:20.199990+0000","last_undegraded":"2024-02-26T04:56:20.199990+0000","last_fullsized":"2024-02-26T04:56:20.199990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:55.379271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.196973+0000","last_change":"2024-02-26T04:55:32.234961+0000","last_active":"2024-02-26T04:56:20.196973+0000","last_peered":"2024-02-26T04:56:20.196973+0000","last_clean":"2024-02-26T04:56:20.196973+0000","last_became_active":"2024-02-26T04:55:32.234455+0000","last_became_peered":"2024-02-26T04:55:32.234455+0000","last_unstale":"2024-02-26T04:56:20.196973+0000","last_undegraded":"2024-02-26T04:56:20.196973+0000","last_fullsized":"2024-02-26T04:56:20.196973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:00:44.322341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":101,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:57:03.246202+0000","last_change":"2024-02-26T04:55:27.199036+0000","last_active":"2024-02-26T04:57:03.246202+0000","last_peered":"2024-02-26T04:57:03.246202+0000","last_clean":"2024-02-26T04:57:03.246202+0000","last_became_active":"2024-02-26T04:55:27.198694+0000","last_became_peered":"2024-02-26T04:55:27.198694+0000","last_unstale":"2024-02-26T04:57:03.246202+0000","last_undegraded":"2024-02-26T04:57:03.246202+0000","last_fullsized":"2024-02-26T04:57:03.246202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:31:41.935123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.208627+0000","last_change":"2024-02-26T04:56:20.208105+0000","last_active":"2024-02-26T04:56:20.208627+0000","last_peered":"2024-02-26T04:56:20.208627+0000","last_clean":"2024-02-26T04:56:20.208627+0000","last_became_active":"2024-02-26T04:56:20.207691+0000","last_became_peered":"2024-02-26T04:56:20.207691+0000","last_unstale":"2024-02-26T04:56:20.208627+0000","last_undegraded":"2024-02-26T04:56:20.208627+0000","last_fullsized":"2024-02-26T04:56:20.208627+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:49:49.019089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,9],"acting":[8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199839+0000","last_change":"2024-02-26T04:55:32.228033+0000","last_active":"2024-02-26T04:56:20.199839+0000","last_peered":"2024-02-26T04:56:20.199839+0000","last_clean":"2024-02-26T04:56:20.199839+0000","last_became_active":"2024-02-26T04:55:32.224273+0000","last_became_peered":"2024-02-26T04:55:32.224273+0000","last_unstale":"2024-02-26T04:56:20.199839+0000","last_undegraded":"2024-02-26T04:56:20.199839+0000","last_fullsized":"2024-02-26T04:56:20.199839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:25:13.618421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200035+0000","last_change":"2024-02-26T04:55:27.202833+0000","last_active":"2024-02-26T04:56:20.200035+0000","last_peered":"2024-02-26T04:56:20.200035+0000","last_clean":"2024-02-26T04:56:20.200035+0000","last_became_active":"2024-02-26T04:55:27.202167+0000","last_became_peered":"2024-02-26T04:55:27.202167+0000","last_unstale":"2024-02-26T04:56:20.200035+0000","last_undegraded":"2024-02-26T04:56:20.200035+0000","last_fullsized":"2024-02-26T04:56:20.200035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:55:46.605763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.207930+0000","last_change":"2024-02-26T04:56:20.207930+0000","last_active":"2024-02-26T04:56:20.207930+0000","last_peered":"2024-02-26T04:56:20.207930+0000","last_clean":"2024-02-26T04:56:20.207930+0000","last_became_active":"2024-02-26T04:56:20.207586+0000","last_became_peered":"2024-02-26T04:56:20.207586+0000","last_unstale":"2024-02-26T04:56:20.207930+0000","last_undegraded":"2024-02-26T04:56:20.207930+0000","last_fullsized":"2024-02-26T04:56:20.207930+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:37:57.141645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197794+0000","last_change":"2024-02-26T04:55:32.225736+0000","last_active":"2024-02-26T04:56:20.197794+0000","last_peered":"2024-02-26T04:56:20.197794+0000","last_clean":"2024-02-26T04:56:20.197794+0000","last_became_active":"2024-02-26T04:55:32.225186+0000","last_became_peered":"2024-02-26T04:55:32.225186+0000","last_unstale":"2024-02-26T04:56:20.197794+0000","last_undegraded":"2024-02-26T04:56:20.197794+0000","last_fullsized":"2024-02-26T04:56:20.197794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:13:48.999538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201071+0000","last_change":"2024-02-26T04:55:27.200389+0000","last_active":"2024-02-26T04:56:20.201071+0000","last_peered":"2024-02-26T04:56:20.201071+0000","last_clean":"2024-02-26T04:56:20.201071+0000","last_became_active":"2024-02-26T04:55:27.200019+0000","last_became_peered":"2024-02-26T04:55:27.200019+0000","last_unstale":"2024-02-26T04:56:20.201071+0000","last_undegraded":"2024-02-26T04:56:20.201071+0000","last_fullsized":"2024-02-26T04:56:20.201071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:59:11.820988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507226+0000","last_change":"2024-02-26T04:55:28.220309+0000","last_active":"2024-02-26T04:56:20.507226+0000","last_peered":"2024-02-26T04:56:20.507226+0000","last_clean":"2024-02-26T04:56:20.507226+0000","last_became_active":"2024-02-26T04:55:28.219987+0000","last_became_peered":"2024-02-26T04:55:28.219987+0000","last_unstale":"2024-02-26T04:56:20.507226+0000","last_undegraded":"2024-02-26T04:56:20.507226+0000","last_fullsized":"2024-02-26T04:56:20.507226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:39:34.713802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200592+0000","last_change":"2024-02-26T04:55:32.236817+0000","last_active":"2024-02-26T04:56:20.200592+0000","last_peered":"2024-02-26T04:56:20.200592+0000","last_clean":"2024-02-26T04:56:20.200592+0000","last_became_active":"2024-02-26T04:55:32.235818+0000","last_became_peered":"2024-02-26T04:55:32.235818+0000","last_unstale":"2024-02-26T04:56:20.200592+0000","last_undegraded":"2024-02-26T04:56:20.200592+0000","last_fullsized":"2024-02-26T04:56:20.200592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:56:54.121684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'33","reported_seq":210,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:57:04.264482+0000","last_change":"2024-02-26T04:55:27.201582+0000","last_active":"2024-02-26T04:57:04.264482+0000","last_peered":"2024-02-26T04:57:04.264482+0000","last_clean":"2024-02-26T04:57:04.264482+0000","last_became_active":"2024-02-26T04:55:27.201011+0000","last_became_peered":"2024-02-26T04:55:27.201011+0000","last_unstale":"2024-02-26T04:57:04.264482+0000","last_undegraded":"2024-02-26T04:57:04.264482+0000","last_fullsized":"2024-02-26T04:57:04.264482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:07:01.888924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63356,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":150,"num_read_kb":137,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.198024+0000","last_change":"2024-02-26T04:55:28.199313+0000","last_active":"2024-02-26T04:56:20.198024+0000","last_peered":"2024-02-26T04:56:20.198024+0000","last_clean":"2024-02-26T04:56:20.198024+0000","last_became_active":"2024-02-26T04:55:28.198973+0000","last_became_peered":"2024-02-26T04:55:28.198973+0000","last_unstale":"2024-02-26T04:56:20.198024+0000","last_undegraded":"2024-02-26T04:56:20.198024+0000","last_fullsized":"2024-02-26T04:56:20.198024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:33:49.005899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201034+0000","last_change":"2024-02-26T04:55:32.234529+0000","last_active":"2024-02-26T04:56:20.201034+0000","last_peered":"2024-02-26T04:56:20.201034+0000","last_clean":"2024-02-26T04:56:20.201034+0000","last_became_active":"2024-02-26T04:55:32.234105+0000","last_became_peered":"2024-02-26T04:55:32.234105+0000","last_unstale":"2024-02-26T04:56:20.201034+0000","last_undegraded":"2024-02-26T04:56:20.201034+0000","last_fullsized":"2024-02-26T04:56:20.201034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:17:40.909267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508685+0000","last_change":"2024-02-26T04:55:27.209178+0000","last_active":"2024-02-26T04:56:20.508685+0000","last_peered":"2024-02-26T04:56:20.508685+0000","last_clean":"2024-02-26T04:56:20.508685+0000","last_became_active":"2024-02-26T04:55:27.208834+0000","last_became_peered":"2024-02-26T04:55:27.208834+0000","last_unstale":"2024-02-26T04:56:20.508685+0000","last_undegraded":"2024-02-26T04:56:20.508685+0000","last_fullsized":"2024-02-26T04:56:20.508685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:28.187333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507311+0000","last_change":"2024-02-26T04:55:28.215208+0000","last_active":"2024-02-26T04:56:20.507311+0000","last_peered":"2024-02-26T04:56:20.507311+0000","last_clean":"2024-02-26T04:56:20.507311+0000","last_became_active":"2024-02-26T04:55:28.212135+0000","last_became_peered":"2024-02-26T04:55:28.212135+0000","last_unstale":"2024-02-26T04:56:20.507311+0000","last_undegraded":"2024-02-26T04:56:20.507311+0000","last_fullsized":"2024-02-26T04:56:20.507311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:51.269557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506939+0000","last_change":"2024-02-26T04:55:32.235439+0000","last_active":"2024-02-26T04:56:20.506939+0000","last_peered":"2024-02-26T04:56:20.506939+0000","last_clean":"2024-02-26T04:56:20.506939+0000","last_became_active":"2024-02-26T04:55:32.234987+0000","last_became_peered":"2024-02-26T04:55:32.234987+0000","last_unstale":"2024-02-26T04:56:20.506939+0000","last_undegraded":"2024-02-26T04:56:20.506939+0000","last_fullsized":"2024-02-26T04:56:20.506939+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:31:00.637846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200477+0000","last_change":"2024-02-26T04:55:27.205498+0000","last_active":"2024-02-26T04:56:20.200477+0000","last_peered":"2024-02-26T04:56:20.200477+0000","last_clean":"2024-02-26T04:56:20.200477+0000","last_became_active":"2024-02-26T04:55:27.205179+0000","last_became_peered":"2024-02-26T04:55:27.205179+0000","last_unstale":"2024-02-26T04:56:20.200477+0000","last_undegraded":"2024-02-26T04:56:20.200477+0000","last_fullsized":"2024-02-26T04:56:20.200477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:02:55.491896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199981+0000","last_change":"2024-02-26T04:55:28.201110+0000","last_active":"2024-02-26T04:56:20.199981+0000","last_peered":"2024-02-26T04:56:20.199981+0000","last_clean":"2024-02-26T04:56:20.199981+0000","last_became_active":"2024-02-26T04:55:28.200725+0000","last_became_peered":"2024-02-26T04:55:28.200725+0000","last_unstale":"2024-02-26T04:56:20.199981+0000","last_undegraded":"2024-02-26T04:56:20.199981+0000","last_fullsized":"2024-02-26T04:56:20.199981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:50.377925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201494+0000","last_change":"2024-02-26T04:55:32.240328+0000","last_active":"2024-02-26T04:56:20.201494+0000","last_peered":"2024-02-26T04:56:20.201494+0000","last_clean":"2024-02-26T04:56:20.201494+0000","last_became_active":"2024-02-26T04:55:32.238662+0000","last_became_peered":"2024-02-26T04:55:32.238662+0000","last_unstale":"2024-02-26T04:56:20.201494+0000","last_undegraded":"2024-02-26T04:56:20.201494+0000","last_fullsized":"2024-02-26T04:56:20.201494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:20:22.938290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:57:04.248371+0000","last_change":"2024-02-26T04:55:27.197338+0000","last_active":"2024-02-26T04:57:04.248371+0000","last_peered":"2024-02-26T04:57:04.248371+0000","last_clean":"2024-02-26T04:57:04.248371+0000","last_became_active":"2024-02-26T04:55:27.196898+0000","last_became_peered":"2024-02-26T04:55:27.196898+0000","last_unstale":"2024-02-26T04:57:04.248371+0000","last_undegraded":"2024-02-26T04:57:04.248371+0000","last_fullsized":"2024-02-26T04:57:04.248371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:27:09.635379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199402+0000","last_change":"2024-02-26T04:55:28.200665+0000","last_active":"2024-02-26T04:56:20.199402+0000","last_peered":"2024-02-26T04:56:20.199402+0000","last_clean":"2024-02-26T04:56:20.199402+0000","last_became_active":"2024-02-26T04:55:28.200329+0000","last_became_peered":"2024-02-26T04:55:28.200329+0000","last_unstale":"2024-02-26T04:56:20.199402+0000","last_undegraded":"2024-02-26T04:56:20.199402+0000","last_fullsized":"2024-02-26T04:56:20.199402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:27.090259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506258+0000","last_change":"2024-02-26T04:55:32.225861+0000","last_active":"2024-02-26T04:56:20.506258+0000","last_peered":"2024-02-26T04:56:20.506258+0000","last_clean":"2024-02-26T04:56:20.506258+0000","last_became_active":"2024-02-26T04:55:32.225187+0000","last_became_peered":"2024-02-26T04:55:32.225187+0000","last_unstale":"2024-02-26T04:56:20.506258+0000","last_undegraded":"2024-02-26T04:56:20.506258+0000","last_fullsized":"2024-02-26T04:56:20.506258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:14:09.077383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197675+0000","last_change":"2024-02-26T04:55:27.200321+0000","last_active":"2024-02-26T04:56:20.197675+0000","last_peered":"2024-02-26T04:56:20.197675+0000","last_clean":"2024-02-26T04:56:20.197675+0000","last_became_active":"2024-02-26T04:55:27.199978+0000","last_became_peered":"2024-02-26T04:55:27.199978+0000","last_unstale":"2024-02-26T04:56:20.197675+0000","last_undegraded":"2024-02-26T04:56:20.197675+0000","last_fullsized":"2024-02-26T04:56:20.197675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:08:20.381010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200440+0000","last_change":"2024-02-26T04:55:28.213940+0000","last_active":"2024-02-26T04:56:20.200440+0000","last_peered":"2024-02-26T04:56:20.200440+0000","last_clean":"2024-02-26T04:56:20.200440+0000","last_became_active":"2024-02-26T04:55:28.213590+0000","last_became_peered":"2024-02-26T04:55:28.213590+0000","last_unstale":"2024-02-26T04:56:20.200440+0000","last_undegraded":"2024-02-26T04:56:20.200440+0000","last_fullsized":"2024-02-26T04:56:20.200440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:29:29.151391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200211+0000","last_change":"2024-02-26T04:55:32.232669+0000","last_active":"2024-02-26T04:56:20.200211+0000","last_peered":"2024-02-26T04:56:20.200211+0000","last_clean":"2024-02-26T04:56:20.200211+0000","last_became_active":"2024-02-26T04:55:32.232288+0000","last_became_peered":"2024-02-26T04:55:32.232288+0000","last_unstale":"2024-02-26T04:56:20.200211+0000","last_undegraded":"2024-02-26T04:56:20.200211+0000","last_fullsized":"2024-02-26T04:56:20.200211+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:31:42.436516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.198355+0000","last_change":"2024-02-26T04:55:27.205993+0000","last_active":"2024-02-26T04:56:20.198355+0000","last_peered":"2024-02-26T04:56:20.198355+0000","last_clean":"2024-02-26T04:56:20.198355+0000","last_became_active":"2024-02-26T04:55:27.205594+0000","last_became_peered":"2024-02-26T04:55:27.205594+0000","last_unstale":"2024-02-26T04:56:20.198355+0000","last_undegraded":"2024-02-26T04:56:20.198355+0000","last_fullsized":"2024-02-26T04:56:20.198355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:44:19.993925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508598+0000","last_change":"2024-02-26T04:55:28.203429+0000","last_active":"2024-02-26T04:56:20.508598+0000","last_peered":"2024-02-26T04:56:20.508598+0000","last_clean":"2024-02-26T04:56:20.508598+0000","last_became_active":"2024-02-26T04:55:28.203002+0000","last_became_peered":"2024-02-26T04:55:28.203002+0000","last_unstale":"2024-02-26T04:56:20.508598+0000","last_undegraded":"2024-02-26T04:56:20.508598+0000","last_fullsized":"2024-02-26T04:56:20.508598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:08:05.872241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197747+0000","last_change":"2024-02-26T04:55:32.239323+0000","last_active":"2024-02-26T04:56:20.197747+0000","last_peered":"2024-02-26T04:56:20.197747+0000","last_clean":"2024-02-26T04:56:20.197747+0000","last_became_active":"2024-02-26T04:55:32.238843+0000","last_became_peered":"2024-02-26T04:55:32.238843+0000","last_unstale":"2024-02-26T04:56:20.197747+0000","last_undegraded":"2024-02-26T04:56:20.197747+0000","last_fullsized":"2024-02-26T04:56:20.197747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:39:56.645429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200638+0000","last_change":"2024-02-26T04:55:27.203222+0000","last_active":"2024-02-26T04:56:20.200638+0000","last_peered":"2024-02-26T04:56:20.200638+0000","last_clean":"2024-02-26T04:56:20.200638+0000","last_became_active":"2024-02-26T04:55:27.202845+0000","last_became_peered":"2024-02-26T04:55:27.202845+0000","last_unstale":"2024-02-26T04:56:20.200638+0000","last_undegraded":"2024-02-26T04:56:20.200638+0000","last_fullsized":"2024-02-26T04:56:20.200638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:16:07.524260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.207212+0000","last_change":"2024-02-26T04:55:28.209122+0000","last_active":"2024-02-26T04:56:20.207212+0000","last_peered":"2024-02-26T04:56:20.207212+0000","last_clean":"2024-02-26T04:56:20.207212+0000","last_became_active":"2024-02-26T04:55:28.207598+0000","last_became_peered":"2024-02-26T04:55:28.207598+0000","last_unstale":"2024-02-26T04:56:20.207212+0000","last_undegraded":"2024-02-26T04:56:20.207212+0000","last_fullsized":"2024-02-26T04:56:20.207212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:02:10.763515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506773+0000","last_change":"2024-02-26T04:55:32.227333+0000","last_active":"2024-02-26T04:56:20.506773+0000","last_peered":"2024-02-26T04:56:20.506773+0000","last_clean":"2024-02-26T04:56:20.506773+0000","last_became_active":"2024-02-26T04:55:32.226956+0000","last_became_peered":"2024-02-26T04:55:32.226956+0000","last_unstale":"2024-02-26T04:56:20.506773+0000","last_undegraded":"2024-02-26T04:56:20.506773+0000","last_fullsized":"2024-02-26T04:56:20.506773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:51:40.710242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199829+0000","last_change":"2024-02-26T04:55:27.198781+0000","last_active":"2024-02-26T04:56:20.199829+0000","last_peered":"2024-02-26T04:56:20.199829+0000","last_clean":"2024-02-26T04:56:20.199829+0000","last_became_active":"2024-02-26T04:55:27.198288+0000","last_became_peered":"2024-02-26T04:55:27.198288+0000","last_unstale":"2024-02-26T04:56:20.199829+0000","last_undegraded":"2024-02-26T04:56:20.199829+0000","last_fullsized":"2024-02-26T04:56:20.199829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:52:18.665816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.206954+0000","last_change":"2024-02-26T04:55:28.213044+0000","last_active":"2024-02-26T04:56:20.206954+0000","last_peered":"2024-02-26T04:56:20.206954+0000","last_clean":"2024-02-26T04:56:20.206954+0000","last_became_active":"2024-02-26T04:55:28.212712+0000","last_became_peered":"2024-02-26T04:55:28.212712+0000","last_unstale":"2024-02-26T04:56:20.206954+0000","last_undegraded":"2024-02-26T04:56:20.206954+0000","last_fullsized":"2024-02-26T04:56:20.206954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:02:09.155551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506311+0000","last_change":"2024-02-26T04:55:32.231867+0000","last_active":"2024-02-26T04:56:20.506311+0000","last_peered":"2024-02-26T04:56:20.506311+0000","last_clean":"2024-02-26T04:56:20.506311+0000","last_became_active":"2024-02-26T04:55:32.231450+0000","last_became_peered":"2024-02-26T04:55:32.231450+0000","last_unstale":"2024-02-26T04:56:20.506311+0000","last_undegraded":"2024-02-26T04:56:20.506311+0000","last_fullsized":"2024-02-26T04:56:20.506311+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:12:14.367427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200434+0000","last_change":"2024-02-26T04:55:28.207746+0000","last_active":"2024-02-26T04:56:20.200434+0000","last_peered":"2024-02-26T04:56:20.200434+0000","last_clean":"2024-02-26T04:56:20.200434+0000","last_became_active":"2024-02-26T04:55:28.207438+0000","last_became_peered":"2024-02-26T04:55:28.207438+0000","last_unstale":"2024-02-26T04:56:20.200434+0000","last_undegraded":"2024-02-26T04:56:20.200434+0000","last_fullsized":"2024-02-26T04:56:20.200434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:09.409635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:57:06.250749+0000","last_change":"2024-02-26T04:55:27.192233+0000","last_active":"2024-02-26T04:57:06.250749+0000","last_peered":"2024-02-26T04:57:06.250749+0000","last_clean":"2024-02-26T04:57:06.250749+0000","last_became_active":"2024-02-26T04:55:27.191868+0000","last_became_peered":"2024-02-26T04:55:27.191868+0000","last_unstale":"2024-02-26T04:57:06.250749+0000","last_undegraded":"2024-02-26T04:57:06.250749+0000","last_fullsized":"2024-02-26T04:57:06.250749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:37:58.430504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200198+0000","last_change":"2024-02-26T04:55:32.226577+0000","last_active":"2024-02-26T04:56:20.200198+0000","last_peered":"2024-02-26T04:56:20.200198+0000","last_clean":"2024-02-26T04:56:20.200198+0000","last_became_active":"2024-02-26T04:55:32.226148+0000","last_became_peered":"2024-02-26T04:55:32.226148+0000","last_unstale":"2024-02-26T04:56:20.200198+0000","last_undegraded":"2024-02-26T04:56:20.200198+0000","last_fullsized":"2024-02-26T04:56:20.200198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:34:53.099211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197342+0000","last_change":"2024-02-26T04:55:28.217865+0000","last_active":"2024-02-26T04:56:20.197342+0000","last_peered":"2024-02-26T04:56:20.197342+0000","last_clean":"2024-02-26T04:56:20.197342+0000","last_became_active":"2024-02-26T04:55:28.217075+0000","last_became_peered":"2024-02-26T04:55:28.217075+0000","last_unstale":"2024-02-26T04:56:20.197342+0000","last_undegraded":"2024-02-26T04:56:20.197342+0000","last_fullsized":"2024-02-26T04:56:20.197342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:30:38.720694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197666+0000","last_change":"2024-02-26T04:55:27.204300+0000","last_active":"2024-02-26T04:56:20.197666+0000","last_peered":"2024-02-26T04:56:20.197666+0000","last_clean":"2024-02-26T04:56:20.197666+0000","last_became_active":"2024-02-26T04:55:27.203933+0000","last_became_peered":"2024-02-26T04:55:27.203933+0000","last_unstale":"2024-02-26T04:56:20.197666+0000","last_undegraded":"2024-02-26T04:56:20.197666+0000","last_fullsized":"2024-02-26T04:56:20.197666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:19:36.247209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199280+0000","last_change":"2024-02-26T04:54:13.145088+0000","last_active":"2024-02-26T04:56:20.199280+0000","last_peered":"2024-02-26T04:56:20.199280+0000","last_clean":"2024-02-26T04:56:20.199280+0000","last_became_active":"2024-02-26T04:54:13.144657+0000","last_became_peered":"2024-02-26T04:54:13.144657+0000","last_unstale":"2024-02-26T04:56:20.199280+0000","last_undegraded":"2024-02-26T04:56:20.199280+0000","last_fullsized":"2024-02-26T04:56:20.199280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:50:45.246330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:50:45.246330+0000","last_clean_scrub_stamp":"2024-02-26T04:50:45.246330+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:00:50.842020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200524+0000","last_change":"2024-02-26T04:55:32.240124+0000","last_active":"2024-02-26T04:56:20.200524+0000","last_peered":"2024-02-26T04:56:20.200524+0000","last_clean":"2024-02-26T04:56:20.200524+0000","last_became_active":"2024-02-26T04:55:32.239624+0000","last_became_peered":"2024-02-26T04:55:32.239624+0000","last_unstale":"2024-02-26T04:56:20.200524+0000","last_undegraded":"2024-02-26T04:56:20.200524+0000","last_fullsized":"2024-02-26T04:56:20.200524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:40:33.770917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507083+0000","last_change":"2024-02-26T04:55:28.204631+0000","last_active":"2024-02-26T04:56:20.507083+0000","last_peered":"2024-02-26T04:56:20.507083+0000","last_clean":"2024-02-26T04:56:20.507083+0000","last_became_active":"2024-02-26T04:55:28.204274+0000","last_became_peered":"2024-02-26T04:55:28.204274+0000","last_unstale":"2024-02-26T04:56:20.507083+0000","last_undegraded":"2024-02-26T04:56:20.507083+0000","last_fullsized":"2024-02-26T04:56:20.507083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:18:18.336015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199214+0000","last_change":"2024-02-26T04:55:27.201719+0000","last_active":"2024-02-26T04:56:20.199214+0000","last_peered":"2024-02-26T04:56:20.199214+0000","last_clean":"2024-02-26T04:56:20.199214+0000","last_became_active":"2024-02-26T04:55:27.199060+0000","last_became_peered":"2024-02-26T04:55:27.199060+0000","last_unstale":"2024-02-26T04:56:20.199214+0000","last_undegraded":"2024-02-26T04:56:20.199214+0000","last_fullsized":"2024-02-26T04:56:20.199214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:37:37.620970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506701+0000","last_change":"2024-02-26T04:55:32.228900+0000","last_active":"2024-02-26T04:56:20.506701+0000","last_peered":"2024-02-26T04:56:20.506701+0000","last_clean":"2024-02-26T04:56:20.506701+0000","last_became_active":"2024-02-26T04:55:32.228262+0000","last_became_peered":"2024-02-26T04:55:32.228262+0000","last_unstale":"2024-02-26T04:56:20.506701+0000","last_undegraded":"2024-02-26T04:56:20.506701+0000","last_fullsized":"2024-02-26T04:56:20.506701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:33.069756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.198038+0000","last_change":"2024-02-26T04:55:27.205264+0000","last_active":"2024-02-26T04:56:20.198038+0000","last_peered":"2024-02-26T04:56:20.198038+0000","last_clean":"2024-02-26T04:56:20.198038+0000","last_became_active":"2024-02-26T04:55:27.204882+0000","last_became_peered":"2024-02-26T04:55:27.204882+0000","last_unstale":"2024-02-26T04:56:20.198038+0000","last_undegraded":"2024-02-26T04:56:20.198038+0000","last_fullsized":"2024-02-26T04:56:20.198038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:14:19.497725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507326+0000","last_change":"2024-02-26T04:55:28.200264+0000","last_active":"2024-02-26T04:56:20.507326+0000","last_peered":"2024-02-26T04:56:20.507326+0000","last_clean":"2024-02-26T04:56:20.507326+0000","last_became_active":"2024-02-26T04:55:28.199913+0000","last_became_peered":"2024-02-26T04:55:28.199913+0000","last_unstale":"2024-02-26T04:56:20.507326+0000","last_undegraded":"2024-02-26T04:56:20.507326+0000","last_fullsized":"2024-02-26T04:56:20.507326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:54:51.529149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.196867+0000","last_change":"2024-02-26T04:55:32.233362+0000","last_active":"2024-02-26T04:56:20.196867+0000","last_peered":"2024-02-26T04:56:20.196867+0000","last_clean":"2024-02-26T04:56:20.196867+0000","last_became_active":"2024-02-26T04:55:32.232907+0000","last_became_peered":"2024-02-26T04:55:32.232907+0000","last_unstale":"2024-02-26T04:56:20.196867+0000","last_undegraded":"2024-02-26T04:56:20.196867+0000","last_fullsized":"2024-02-26T04:56:20.196867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:49:11.889863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200207+0000","last_change":"2024-02-26T04:55:27.207371+0000","last_active":"2024-02-26T04:56:20.200207+0000","last_peered":"2024-02-26T04:56:20.200207+0000","last_clean":"2024-02-26T04:56:20.200207+0000","last_became_active":"2024-02-26T04:55:27.207047+0000","last_became_peered":"2024-02-26T04:55:27.207047+0000","last_unstale":"2024-02-26T04:56:20.200207+0000","last_undegraded":"2024-02-26T04:56:20.200207+0000","last_fullsized":"2024-02-26T04:56:20.200207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:29.575633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197403+0000","last_change":"2024-02-26T04:55:28.207132+0000","last_active":"2024-02-26T04:56:20.197403+0000","last_peered":"2024-02-26T04:56:20.197403+0000","last_clean":"2024-02-26T04:56:20.197403+0000","last_became_active":"2024-02-26T04:55:28.206376+0000","last_became_peered":"2024-02-26T04:55:28.206376+0000","last_unstale":"2024-02-26T04:56:20.197403+0000","last_undegraded":"2024-02-26T04:56:20.197403+0000","last_fullsized":"2024-02-26T04:56:20.197403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:17:53.802386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.509195+0000","last_change":"2024-02-26T04:55:32.226381+0000","last_active":"2024-02-26T04:56:20.509195+0000","last_peered":"2024-02-26T04:56:20.509195+0000","last_clean":"2024-02-26T04:56:20.509195+0000","last_became_active":"2024-02-26T04:55:32.225940+0000","last_became_peered":"2024-02-26T04:55:32.225940+0000","last_unstale":"2024-02-26T04:56:20.509195+0000","last_undegraded":"2024-02-26T04:56:20.509195+0000","last_fullsized":"2024-02-26T04:56:20.509195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:51:20.190514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.198213+0000","last_change":"2024-02-26T04:55:27.200246+0000","last_active":"2024-02-26T04:56:20.198213+0000","last_peered":"2024-02-26T04:56:20.198213+0000","last_clean":"2024-02-26T04:56:20.198213+0000","last_became_active":"2024-02-26T04:55:27.199906+0000","last_became_peered":"2024-02-26T04:55:27.199906+0000","last_unstale":"2024-02-26T04:56:20.198213+0000","last_undegraded":"2024-02-26T04:56:20.198213+0000","last_fullsized":"2024-02-26T04:56:20.198213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:29.037230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.204222+0000","last_change":"2024-02-26T04:56:20.204222+0000","last_active":"2024-02-26T04:56:20.204222+0000","last_peered":"2024-02-26T04:56:20.204222+0000","last_clean":"2024-02-26T04:56:20.204222+0000","last_became_active":"2024-02-26T04:56:20.203726+0000","last_became_peered":"2024-02-26T04:56:20.203726+0000","last_unstale":"2024-02-26T04:56:20.204222+0000","last_undegraded":"2024-02-26T04:56:20.204222+0000","last_fullsized":"2024-02-26T04:56:20.204222+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:37:59.371734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197790+0000","last_change":"2024-02-26T04:55:32.227765+0000","last_active":"2024-02-26T04:56:20.197790+0000","last_peered":"2024-02-26T04:56:20.197790+0000","last_clean":"2024-02-26T04:56:20.197790+0000","last_became_active":"2024-02-26T04:55:32.227346+0000","last_became_peered":"2024-02-26T04:55:32.227346+0000","last_unstale":"2024-02-26T04:56:20.197790+0000","last_undegraded":"2024-02-26T04:56:20.197790+0000","last_fullsized":"2024-02-26T04:56:20.197790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:05:12.634958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506443+0000","last_change":"2024-02-26T04:55:27.199979+0000","last_active":"2024-02-26T04:56:20.506443+0000","last_peered":"2024-02-26T04:56:20.506443+0000","last_clean":"2024-02-26T04:56:20.506443+0000","last_became_active":"2024-02-26T04:55:27.199618+0000","last_became_peered":"2024-02-26T04:55:27.199618+0000","last_unstale":"2024-02-26T04:56:20.506443+0000","last_undegraded":"2024-02-26T04:56:20.506443+0000","last_fullsized":"2024-02-26T04:56:20.506443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:20:59.649473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197522+0000","last_change":"2024-02-26T04:55:28.202253+0000","last_active":"2024-02-26T04:56:20.197522+0000","last_peered":"2024-02-26T04:56:20.197522+0000","last_clean":"2024-02-26T04:56:20.197522+0000","last_became_active":"2024-02-26T04:55:28.199825+0000","last_became_peered":"2024-02-26T04:55:28.199825+0000","last_unstale":"2024-02-26T04:56:20.197522+0000","last_undegraded":"2024-02-26T04:56:20.197522+0000","last_fullsized":"2024-02-26T04:56:20.197522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:21:59.148995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201305+0000","last_change":"2024-02-26T04:55:32.239327+0000","last_active":"2024-02-26T04:56:20.201305+0000","last_peered":"2024-02-26T04:56:20.201305+0000","last_clean":"2024-02-26T04:56:20.201305+0000","last_became_active":"2024-02-26T04:55:32.238742+0000","last_became_peered":"2024-02-26T04:55:32.238742+0000","last_unstale":"2024-02-26T04:56:20.201305+0000","last_undegraded":"2024-02-26T04:56:20.201305+0000","last_fullsized":"2024-02-26T04:56:20.201305+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:38:08.821549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197054+0000","last_change":"2024-02-26T04:55:27.206529+0000","last_active":"2024-02-26T04:56:20.197054+0000","last_peered":"2024-02-26T04:56:20.197054+0000","last_clean":"2024-02-26T04:56:20.197054+0000","last_became_active":"2024-02-26T04:55:27.206203+0000","last_became_peered":"2024-02-26T04:55:27.206203+0000","last_unstale":"2024-02-26T04:56:20.197054+0000","last_undegraded":"2024-02-26T04:56:20.197054+0000","last_fullsized":"2024-02-26T04:56:20.197054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:24.491832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507816+0000","last_change":"2024-02-26T04:55:28.205915+0000","last_active":"2024-02-26T04:56:20.507816+0000","last_peered":"2024-02-26T04:56:20.507816+0000","last_clean":"2024-02-26T04:56:20.507816+0000","last_became_active":"2024-02-26T04:55:28.205576+0000","last_became_peered":"2024-02-26T04:55:28.205576+0000","last_unstale":"2024-02-26T04:56:20.507816+0000","last_undegraded":"2024-02-26T04:56:20.507816+0000","last_fullsized":"2024-02-26T04:56:20.507816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:18:54.639778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197601+0000","last_change":"2024-02-26T04:55:32.226253+0000","last_active":"2024-02-26T04:56:20.197601+0000","last_peered":"2024-02-26T04:56:20.197601+0000","last_clean":"2024-02-26T04:56:20.197601+0000","last_became_active":"2024-02-26T04:55:32.225815+0000","last_became_peered":"2024-02-26T04:55:32.225815+0000","last_unstale":"2024-02-26T04:56:20.197601+0000","last_undegraded":"2024-02-26T04:56:20.197601+0000","last_fullsized":"2024-02-26T04:56:20.197601+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:22.764756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507987+0000","last_change":"2024-02-26T04:55:27.209066+0000","last_active":"2024-02-26T04:56:20.507987+0000","last_peered":"2024-02-26T04:56:20.507987+0000","last_clean":"2024-02-26T04:56:20.507987+0000","last_became_active":"2024-02-26T04:55:27.208556+0000","last_became_peered":"2024-02-26T04:55:27.208556+0000","last_unstale":"2024-02-26T04:56:20.507987+0000","last_undegraded":"2024-02-26T04:56:20.507987+0000","last_fullsized":"2024-02-26T04:56:20.507987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:14:34.950262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197222+0000","last_change":"2024-02-26T04:55:28.213065+0000","last_active":"2024-02-26T04:56:20.197222+0000","last_peered":"2024-02-26T04:56:20.197222+0000","last_clean":"2024-02-26T04:56:20.197222+0000","last_became_active":"2024-02-26T04:55:28.211112+0000","last_became_peered":"2024-02-26T04:55:28.211112+0000","last_unstale":"2024-02-26T04:56:20.197222+0000","last_undegraded":"2024-02-26T04:56:20.197222+0000","last_fullsized":"2024-02-26T04:56:20.197222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:41:10.070183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197607+0000","last_change":"2024-02-26T04:55:32.229838+0000","last_active":"2024-02-26T04:56:20.197607+0000","last_peered":"2024-02-26T04:56:20.197607+0000","last_clean":"2024-02-26T04:56:20.197607+0000","last_became_active":"2024-02-26T04:55:32.228696+0000","last_became_peered":"2024-02-26T04:55:32.228696+0000","last_unstale":"2024-02-26T04:56:20.197607+0000","last_undegraded":"2024-02-26T04:56:20.197607+0000","last_fullsized":"2024-02-26T04:56:20.197607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:59:42.359301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197678+0000","last_change":"2024-02-26T04:55:27.205688+0000","last_active":"2024-02-26T04:56:20.197678+0000","last_peered":"2024-02-26T04:56:20.197678+0000","last_clean":"2024-02-26T04:56:20.197678+0000","last_became_active":"2024-02-26T04:55:27.205306+0000","last_became_peered":"2024-02-26T04:55:27.205306+0000","last_unstale":"2024-02-26T04:56:20.197678+0000","last_undegraded":"2024-02-26T04:56:20.197678+0000","last_fullsized":"2024-02-26T04:56:20.197678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:23:50.203565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199851+0000","last_change":"2024-02-26T04:55:28.210073+0000","last_active":"2024-02-26T04:56:20.199851+0000","last_peered":"2024-02-26T04:56:20.199851+0000","last_clean":"2024-02-26T04:56:20.199851+0000","last_became_active":"2024-02-26T04:55:28.208475+0000","last_became_peered":"2024-02-26T04:55:28.208475+0000","last_unstale":"2024-02-26T04:56:20.199851+0000","last_undegraded":"2024-02-26T04:56:20.199851+0000","last_fullsized":"2024-02-26T04:56:20.199851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:06:49.513916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506115+0000","last_change":"2024-02-26T04:55:32.228708+0000","last_active":"2024-02-26T04:56:20.506115+0000","last_peered":"2024-02-26T04:56:20.506115+0000","last_clean":"2024-02-26T04:56:20.506115+0000","last_became_active":"2024-02-26T04:55:32.228304+0000","last_became_peered":"2024-02-26T04:55:32.228304+0000","last_unstale":"2024-02-26T04:56:20.506115+0000","last_undegraded":"2024-02-26T04:56:20.506115+0000","last_fullsized":"2024-02-26T04:56:20.506115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:53:02.390712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.509071+0000","last_change":"2024-02-26T04:55:27.199761+0000","last_active":"2024-02-26T04:56:20.509071+0000","last_peered":"2024-02-26T04:56:20.509071+0000","last_clean":"2024-02-26T04:56:20.509071+0000","last_became_active":"2024-02-26T04:55:27.199113+0000","last_became_peered":"2024-02-26T04:55:27.199113+0000","last_unstale":"2024-02-26T04:56:20.509071+0000","last_undegraded":"2024-02-26T04:56:20.509071+0000","last_fullsized":"2024-02-26T04:56:20.509071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:14:10.631886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199755+0000","last_change":"2024-02-26T04:55:28.211484+0000","last_active":"2024-02-26T04:56:20.199755+0000","last_peered":"2024-02-26T04:56:20.199755+0000","last_clean":"2024-02-26T04:56:20.199755+0000","last_became_active":"2024-02-26T04:55:28.211131+0000","last_became_peered":"2024-02-26T04:55:28.211131+0000","last_unstale":"2024-02-26T04:56:20.199755+0000","last_undegraded":"2024-02-26T04:56:20.199755+0000","last_fullsized":"2024-02-26T04:56:20.199755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:54:09.393847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200334+0000","last_change":"2024-02-26T04:55:32.227493+0000","last_active":"2024-02-26T04:56:20.200334+0000","last_peered":"2024-02-26T04:56:20.200334+0000","last_clean":"2024-02-26T04:56:20.200334+0000","last_became_active":"2024-02-26T04:55:32.227106+0000","last_became_peered":"2024-02-26T04:55:32.227106+0000","last_unstale":"2024-02-26T04:56:20.200334+0000","last_undegraded":"2024-02-26T04:56:20.200334+0000","last_fullsized":"2024-02-26T04:56:20.200334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:33.040966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200737+0000","last_change":"2024-02-26T04:55:27.205331+0000","last_active":"2024-02-26T04:56:20.200737+0000","last_peered":"2024-02-26T04:56:20.200737+0000","last_clean":"2024-02-26T04:56:20.200737+0000","last_became_active":"2024-02-26T04:55:27.204998+0000","last_became_peered":"2024-02-26T04:55:27.204998+0000","last_unstale":"2024-02-26T04:56:20.200737+0000","last_undegraded":"2024-02-26T04:56:20.200737+0000","last_fullsized":"2024-02-26T04:56:20.200737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:10:03.671016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508854+0000","last_change":"2024-02-26T04:56:20.508854+0000","last_active":"2024-02-26T04:56:20.508854+0000","last_peered":"2024-02-26T04:56:20.508854+0000","last_clean":"2024-02-26T04:56:20.508854+0000","last_became_active":"2024-02-26T04:56:20.508396+0000","last_became_peered":"2024-02-26T04:56:20.508396+0000","last_unstale":"2024-02-26T04:56:20.508854+0000","last_undegraded":"2024-02-26T04:56:20.508854+0000","last_fullsized":"2024-02-26T04:56:20.508854+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:47:33.249035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200022+0000","last_change":"2024-02-26T04:55:32.232222+0000","last_active":"2024-02-26T04:56:20.200022+0000","last_peered":"2024-02-26T04:56:20.200022+0000","last_clean":"2024-02-26T04:56:20.200022+0000","last_became_active":"2024-02-26T04:55:32.231836+0000","last_became_peered":"2024-02-26T04:55:32.231836+0000","last_unstale":"2024-02-26T04:56:20.200022+0000","last_undegraded":"2024-02-26T04:56:20.200022+0000","last_fullsized":"2024-02-26T04:56:20.200022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:18:00.766266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507710+0000","last_change":"2024-02-26T04:55:27.200385+0000","last_active":"2024-02-26T04:56:20.507710+0000","last_peered":"2024-02-26T04:56:20.507710+0000","last_clean":"2024-02-26T04:56:20.507710+0000","last_became_active":"2024-02-26T04:55:27.200046+0000","last_became_peered":"2024-02-26T04:55:27.200046+0000","last_unstale":"2024-02-26T04:56:20.507710+0000","last_undegraded":"2024-02-26T04:56:20.507710+0000","last_fullsized":"2024-02-26T04:56:20.507710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:01:58.112457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200220+0000","last_change":"2024-02-26T04:55:28.220702+0000","last_active":"2024-02-26T04:56:20.200220+0000","last_peered":"2024-02-26T04:56:20.200220+0000","last_clean":"2024-02-26T04:56:20.200220+0000","last_became_active":"2024-02-26T04:55:28.220386+0000","last_became_peered":"2024-02-26T04:55:28.220386+0000","last_unstale":"2024-02-26T04:56:20.200220+0000","last_undegraded":"2024-02-26T04:56:20.200220+0000","last_fullsized":"2024-02-26T04:56:20.200220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:40:37.774831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507883+0000","last_change":"2024-02-26T04:55:28.206409+0000","last_active":"2024-02-26T04:56:20.507883+0000","last_peered":"2024-02-26T04:56:20.507883+0000","last_clean":"2024-02-26T04:56:20.507883+0000","last_became_active":"2024-02-26T04:55:28.206046+0000","last_became_peered":"2024-02-26T04:55:28.206046+0000","last_unstale":"2024-02-26T04:56:20.507883+0000","last_undegraded":"2024-02-26T04:56:20.507883+0000","last_fullsized":"2024-02-26T04:56:20.507883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:35:43.277376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.509537+0000","last_change":"2024-02-26T04:55:28.216146+0000","last_active":"2024-02-26T04:56:20.509537+0000","last_peered":"2024-02-26T04:56:20.509537+0000","last_clean":"2024-02-26T04:56:20.509537+0000","last_became_active":"2024-02-26T04:55:28.215800+0000","last_became_peered":"2024-02-26T04:55:28.215800+0000","last_unstale":"2024-02-26T04:56:20.509537+0000","last_undegraded":"2024-02-26T04:56:20.509537+0000","last_fullsized":"2024-02-26T04:56:20.509537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:18:47.902694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507974+0000","last_change":"2024-02-26T04:55:28.221587+0000","last_active":"2024-02-26T04:56:20.507974+0000","last_peered":"2024-02-26T04:56:20.507974+0000","last_clean":"2024-02-26T04:56:20.507974+0000","last_became_active":"2024-02-26T04:55:28.220990+0000","last_became_peered":"2024-02-26T04:55:28.220990+0000","last_unstale":"2024-02-26T04:56:20.507974+0000","last_undegraded":"2024-02-26T04:56:20.507974+0000","last_fullsized":"2024-02-26T04:56:20.507974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:24:13.302081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200761+0000","last_change":"2024-02-26T04:55:28.220586+0000","last_active":"2024-02-26T04:56:20.200761+0000","last_peered":"2024-02-26T04:56:20.200761+0000","last_clean":"2024-02-26T04:56:20.200761+0000","last_became_active":"2024-02-26T04:55:28.219585+0000","last_became_peered":"2024-02-26T04:55:28.219585+0000","last_unstale":"2024-02-26T04:56:20.200761+0000","last_undegraded":"2024-02-26T04:56:20.200761+0000","last_fullsized":"2024-02-26T04:56:20.200761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:41.786142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507368+0000","last_change":"2024-02-26T04:55:28.204599+0000","last_active":"2024-02-26T04:56:20.507368+0000","last_peered":"2024-02-26T04:56:20.507368+0000","last_clean":"2024-02-26T04:56:20.507368+0000","last_became_active":"2024-02-26T04:55:28.204252+0000","last_became_peered":"2024-02-26T04:55:28.204252+0000","last_unstale":"2024-02-26T04:56:20.507368+0000","last_undegraded":"2024-02-26T04:56:20.507368+0000","last_fullsized":"2024-02-26T04:56:20.507368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:30:19.807897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508555+0000","last_change":"2024-02-26T04:55:28.214233+0000","last_active":"2024-02-26T04:56:20.508555+0000","last_peered":"2024-02-26T04:56:20.508555+0000","last_clean":"2024-02-26T04:56:20.508555+0000","last_became_active":"2024-02-26T04:55:28.213927+0000","last_became_peered":"2024-02-26T04:55:28.213927+0000","last_unstale":"2024-02-26T04:56:20.508555+0000","last_undegraded":"2024-02-26T04:56:20.508555+0000","last_fullsized":"2024-02-26T04:56:20.508555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:31:07.967134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.509966+0000","last_change":"2024-02-26T04:55:28.217610+0000","last_active":"2024-02-26T04:56:20.509966+0000","last_peered":"2024-02-26T04:56:20.509966+0000","last_clean":"2024-02-26T04:56:20.509966+0000","last_became_active":"2024-02-26T04:55:28.217263+0000","last_became_peered":"2024-02-26T04:55:28.217263+0000","last_unstale":"2024-02-26T04:56:20.509966+0000","last_undegraded":"2024-02-26T04:56:20.509966+0000","last_fullsized":"2024-02-26T04:56:20.509966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:28:18.831887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.510859+0000","last_change":"2024-02-26T04:56:20.510859+0000","last_active":"2024-02-26T04:56:20.510859+0000","last_peered":"2024-02-26T04:56:20.510859+0000","last_clean":"2024-02-26T04:56:20.510859+0000","last_became_active":"2024-02-26T04:56:20.510420+0000","last_became_peered":"2024-02-26T04:56:20.510420+0000","last_unstale":"2024-02-26T04:56:20.510859+0000","last_undegraded":"2024-02-26T04:56:20.510859+0000","last_fullsized":"2024-02-26T04:56:20.510859+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:02.961996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200654+0000","last_change":"2024-02-26T04:55:28.217247+0000","last_active":"2024-02-26T04:56:20.200654+0000","last_peered":"2024-02-26T04:56:20.200654+0000","last_clean":"2024-02-26T04:56:20.200654+0000","last_became_active":"2024-02-26T04:55:28.214691+0000","last_became_peered":"2024-02-26T04:55:28.214691+0000","last_unstale":"2024-02-26T04:56:20.200654+0000","last_undegraded":"2024-02-26T04:56:20.200654+0000","last_fullsized":"2024-02-26T04:56:20.200654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:27:45.564520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197229+0000","last_change":"2024-02-26T04:55:28.210491+0000","last_active":"2024-02-26T04:56:20.197229+0000","last_peered":"2024-02-26T04:56:20.197229+0000","last_clean":"2024-02-26T04:56:20.197229+0000","last_became_active":"2024-02-26T04:55:28.210144+0000","last_became_peered":"2024-02-26T04:55:28.210144+0000","last_unstale":"2024-02-26T04:56:20.197229+0000","last_undegraded":"2024-02-26T04:56:20.197229+0000","last_fullsized":"2024-02-26T04:56:20.197229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:11.214716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506822+0000","last_change":"2024-02-26T04:55:28.216037+0000","last_active":"2024-02-26T04:56:20.506822+0000","last_peered":"2024-02-26T04:56:20.506822+0000","last_clean":"2024-02-26T04:56:20.506822+0000","last_became_active":"2024-02-26T04:55:28.215712+0000","last_became_peered":"2024-02-26T04:55:28.215712+0000","last_unstale":"2024-02-26T04:56:20.506822+0000","last_undegraded":"2024-02-26T04:56:20.506822+0000","last_fullsized":"2024-02-26T04:56:20.506822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:38:55.437642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507768+0000","last_change":"2024-02-26T04:55:28.205469+0000","last_active":"2024-02-26T04:56:20.507768+0000","last_peered":"2024-02-26T04:56:20.507768+0000","last_clean":"2024-02-26T04:56:20.507768+0000","last_became_active":"2024-02-26T04:55:28.205137+0000","last_became_peered":"2024-02-26T04:55:28.205137+0000","last_unstale":"2024-02-26T04:56:20.507768+0000","last_undegraded":"2024-02-26T04:56:20.507768+0000","last_fullsized":"2024-02-26T04:56:20.507768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:50.425010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508100+0000","last_change":"2024-02-26T04:55:28.213066+0000","last_active":"2024-02-26T04:56:20.508100+0000","last_peered":"2024-02-26T04:56:20.508100+0000","last_clean":"2024-02-26T04:56:20.508100+0000","last_became_active":"2024-02-26T04:55:28.212706+0000","last_became_peered":"2024-02-26T04:55:28.212706+0000","last_unstale":"2024-02-26T04:56:20.508100+0000","last_undegraded":"2024-02-26T04:56:20.508100+0000","last_fullsized":"2024-02-26T04:56:20.508100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:21:24.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281347,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66966,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":457,"num_read_kb":300,"num_write":160,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":200898,"data_compressed":4869,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":500807,"data_compressed":20559,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54900000000000004,"15min":0.55400000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68500000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67100000000000004}]},{"osd":1,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79300000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.46700000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85299999999999998,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.502}]},{"osd":2,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.51400000000000001,"15min":0.52400000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67800000000000005,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.498,"15min":0.498},"min":{"1min":0.36299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":4,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.83499999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.46700000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.82599999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59899999999999998}]},{"osd":5,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61399999999999999,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.42599999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86699999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.68799999999999994}]},{"osd":6,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.73199999999999998,"15min":0.74299999999999999},"min":{"1min":0.55200000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90000000000000002,"5min":1.175,"15min":1.175},"last":0.58299999999999996}]},{"osd":7,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94199999999999995,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.57699999999999996,"5min":0.435,"15min":0.435},"max":{"1min":0.92800000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.64500000000000002}]},{"osd":8,"last update":"Mon Feb 26 04:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.44},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.47699999999999998}]},{"osd":9,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78700000000000003,"15min":0.78900000000000003},"min":{"1min":0.61099999999999999,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.97399999999999998,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.59999999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.022,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.70899999999999996}]},{"osd":10,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.81000000000000005},"min":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.97299999999999998,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82099999999999995,"15min":0.82699999999999996},"min":{"1min":0.59699999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.92900000000000005,"5min":1.246,"15min":1.246},"last":0.63200000000000001}]}]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1138688,"data_stored":1151431,"data_compressed":57965,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:29 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":9.6609999999999996,"5min":9.6609999999999996,"15min":9.6609999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":9.6229999999999993,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":0.56100000000000005}]},{"osd":1,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.497,"15min":0.501},"min":{"1min":0.28599999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.497,"15min":0.499},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.53500000000000003}]},{"osd":2,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.75800000000000001}]},{"osd":3,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82899999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.432,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69199999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.65000000000000002}]},{"osd":4,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78300000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.443},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56799999999999995,"15min":0.57499999999999996},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.751,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.61899999999999999}]},{"osd":5,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68500000000000005,"15min":0.68799999999999994},"min":{"1min":0.443,"5min":0.435,"15min":0.435},"max":{"1min":0.91300000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.45900000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.93100000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90400000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.49199999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.94499999999999995,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.70399999999999996}]},{"osd":7,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.72699999999999998}]},{"osd":8,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.75900000000000001,"15min":0.76700000000000002},"min":{"1min":0.441,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0549999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.745},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.80700000000000005,"15min":0.81899999999999995},"min":{"1min":0.51800000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97799999999999998,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.76900000000000002}]},{"osd":9,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0169999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.82499999999999996,"15min":0.83799999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86599999999999999,"5min":1.274,"15min":1.274},"last":0.82999999999999996}]},{"osd":11,"last update":"Mon Feb 26 04:56:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.034,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.56100000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.997,"5min":1.21,"15min":1.21},"last":0.85199999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1015808,"data_stored":505139,"data_compressed":20559,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.44400000000000001,"15min":0.47399999999999998},"min":{"1min":0.30299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.54100000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.46300000000000002,"15min":0.49299999999999999},"min":{"1min":0.28100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64900000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.32600000000000001}]},{"osd":1,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60399999999999998,"15min":0.627},"min":{"1min":0.40999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77000000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.64600000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79700000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42299999999999999}]},{"osd":2,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.64600000000000002,"15min":0.66900000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73499999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.502},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.72999999999999998,"15min":0.79300000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77000000000000002,"5min":3.173,"15min":3.173},"last":0.47599999999999998}]},{"osd":3,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.49299999999999999,"15min":0.52100000000000002},"min":{"1min":0.27400000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.60099999999999998,"5min":0.753,"15min":0.753},"last":0.436},{"interface":"front","average":{"1min":0.44,"5min":0.498,"15min":0.51800000000000002},"min":{"1min":0.30599999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64000000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.34599999999999997}]},{"osd":4,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.64500000000000002,"15min":0.68400000000000005},"min":{"1min":0.44400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77500000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63700000000000001,"15min":0.66500000000000004},"min":{"1min":0.36899999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.52600000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.67600000000000005,"15min":0.70199999999999996},"min":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.68799999999999994,"15min":0.71699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.76600000000000001,"5min":1.133,"15min":1.133},"last":0.57599999999999996}]},{"osd":6,"last update":"Mon Feb 26 04:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.71199999999999997,"15min":0.754},"min":{"1min":0.46300000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83299999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.73499999999999999,"15min":0.77200000000000002},"min":{"1min":0.46100000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81899999999999995,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55100000000000005}]},{"osd":8,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.84799999999999998,"15min":0.91900000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85699999999999998,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76400000000000001,"15min":0.80200000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.627}]},{"osd":10,"last update":"Mon Feb 26 04:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.751,"15min":0.76100000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58699999999999997}]},{"osd":11,"last update":"Mon Feb 26 04:56:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61099999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":500963,"data_compressed":20559,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.38100000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.196,"5min":0.65600000000000003,"15min":0.56999999999999995},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":12.359999999999999,"5min":12.359999999999999,"15min":12.359999999999999},"last":0.51600000000000001}]},{"osd":1,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.47399999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78400000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.63300000000000001,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.87}]},{"osd":2,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.44800000000000001,"15min":0.433},"min":{"1min":0.33100000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46300000000000002,"15min":0.44},"min":{"1min":0.33300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.47299999999999998}]},{"osd":3,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.40999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.622},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.60199999999999998}]},{"osd":4,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.52400000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.88},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.629,"15min":0.61599999999999999},"min":{"1min":0.54800000000000004,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.80700000000000005}]},{"osd":5,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.55700000000000005,"15min":0.53600000000000003},"min":{"1min":0.374,"5min":0.309,"15min":0.309},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.56100000000000005,"15min":0.54100000000000004},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.72399999999999998}]},{"osd":6,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":1.411,"5min":0.81799999999999995,"15min":0.72299999999999998},"min":{"1min":0.57699999999999996,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":12.384,"5min":12.384,"15min":12.384},"last":0.74299999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.71899999999999997,"15min":0.70299999999999996},"min":{"1min":0.63700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.70699999999999996,"15min":0.67700000000000005},"min":{"1min":0.61899999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.82899999999999996}]},{"osd":9,"last update":"Mon Feb 26 04:56:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.73699999999999999,"15min":0.70899999999999996},"min":{"1min":0.626,"5min":0.435,"15min":0.435},"max":{"1min":1.0700000000000001,"5min":1.073,"15min":1.073},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.71599999999999997,"15min":0.68300000000000005},"min":{"1min":0.59499999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.998,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.78300000000000003}]},{"osd":10,"last update":"Mon Feb 26 04:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.75600000000000001,"15min":0.73999999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.78200000000000003,"15min":0.76700000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.81899999999999995}]},{"osd":11,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.76000000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":1096481,"data_compressed":56342,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67000000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.40799999999999997}]},{"osd":1,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42399999999999999,"15min":0.41799999999999998},"min":{"1min":0.27900000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45700000000000002,"15min":0.443},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.58999999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.50600000000000001}]},{"osd":2,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82299999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84599999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.77400000000000002}]},{"osd":3,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.49299999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79600000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81599999999999995,"5min":0.877,"15min":0.877},"last":0.67500000000000004}]},{"osd":4,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.30499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.753,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.755}]},{"osd":5,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.46500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86499999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66500000000000004,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.81899999999999995}]},{"osd":6,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.46200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.56200000000000006,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85499999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.71199999999999997}]},{"osd":8,"last update":"Mon Feb 26 04:56:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72699999999999998,"15min":0.73299999999999998},"min":{"1min":0.57699999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.71199999999999997,"15min":0.71099999999999997},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.442},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.89900000000000002}]},{"osd":9,"last update":"Mon Feb 26 04:56:50 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0189999999999999,"5min":1.093,"15min":1.093},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.72899999999999998,"15min":0.71499999999999997},"min":{"1min":0.55500000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.97299999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.871}]},{"osd":10,"last update":"Mon Feb 26 04:56:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85499999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.84499999999999997}]},{"osd":11,"last update":"Mon Feb 26 04:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.90600000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":496219,"data_compressed":20559,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.41599999999999998,"15min":0.38800000000000001},"min":{"1min":0.30399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67100000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.45800000000000002,"15min":0.42899999999999999},"min":{"1min":0.32300000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.72499999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.47799999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.63,"15min":0.61599999999999999},"min":{"1min":0.48599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.60499999999999998,"15min":0.57399999999999995},"min":{"1min":0.47499999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86199999999999999,"5min":1.032,"15min":1.032},"last":0.40699999999999997}]},{"osd":2,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.60499999999999998,"15min":0.57299999999999995},"min":{"1min":0.45300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.85199999999999998,"15min":1.054},"min":{"1min":0.496,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92600000000000005,"5min":14.884,"15min":14.884},"last":0.59299999999999997}]},{"osd":3,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48299999999999998,"15min":0.45700000000000002},"min":{"1min":0.35499999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70099999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53700000000000003,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.38300000000000001}]},{"osd":4,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.43099999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.99399999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.441},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.90000000000000002,"15min":1.1080000000000001},"min":{"1min":0.45400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96699999999999997,"5min":14.898,"15min":14.898},"last":0.45600000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.64200000000000002},"min":{"1min":0.47099999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.69499999999999995,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.67400000000000004}]},{"osd":7,"last update":"Mon Feb 26 04:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76500000000000001,"15min":0.74199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71399999999999997,"15min":0.69199999999999995},"min":{"1min":0.501,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.044,"5min":1.075,"15min":1.075},"last":0.53700000000000003}]},{"osd":8,"last update":"Mon Feb 26 04:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.56100000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.60899999999999999}]},{"osd":9,"last update":"Mon Feb 26 04:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77300000000000002,"15min":0.80100000000000005},"min":{"1min":0.47399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.76500000000000001,"15min":0.79600000000000004},"min":{"1min":0.48299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.497}]},{"osd":10,"last update":"Mon Feb 26 04:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.871,"15min":0.877},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.622},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.65400000000000003}]},{"osd":11,"last update":"Mon Feb 26 04:56:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.70599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":559485,"data_compressed":22182,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.54600000000000004,"15min":0.499},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.92400000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.55300000000000005,"15min":0.504},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.38400000000000001}]},{"osd":2,"last update":"Mon Feb 26 04:56:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.60599999999999998,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59299999999999997,"15min":0.54100000000000004},"min":{"1min":0.48199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83899999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66700000000000004}]},{"osd":3,"last update":"Mon Feb 26 04:56:23 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56899999999999995,"15min":0.55300000000000005},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90300000000000002,"5min":0.997,"15min":0.997},"last":0.433},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.626,"15min":0.57499999999999996},"min":{"1min":0.439,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.60599999999999998}]},{"osd":4,"last update":"Mon Feb 26 04:56:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.45600000000000002,"15min":0.42199999999999999},"min":{"1min":0.28299999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.46300000000000002,"15min":0.42699999999999999},"min":{"1min":0.30099999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.51700000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61599999999999999,"15min":0.56299999999999994},"min":{"1min":0.47899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.59999999999999998,"15min":0.55100000000000005},"min":{"1min":0.50900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58499999999999996}]},{"osd":6,"last update":"Mon Feb 26 04:56:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.48899999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82599999999999996,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.505},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.70799999999999996,"15min":0.66300000000000003},"min":{"1min":0.55300000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97799999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.64400000000000002}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.55200000000000005},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88300000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.625},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75600000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.56599999999999995}]},{"osd":8,"last update":"Mon Feb 26 04:56:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73999999999999999,"15min":0.71599999999999997},"min":{"1min":0.52400000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.03,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.73399999999999999,"15min":0.71499999999999997},"min":{"1min":0.59399999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.028,"5min":1.04,"15min":1.04},"last":0.74399999999999999}]},{"osd":9,"last update":"Mon Feb 26 04:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1080000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.81100000000000005}]},{"osd":10,"last update":"Mon Feb 26 04:56:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.504,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0720000000000001,"5min":1.095,"15min":1.095},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95999999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70099999999999996}]},{"osd":11,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.78100000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":497607,"data_compressed":20559,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 04:56:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53400000000000003,"15min":0.50700000000000001},"min":{"1min":0.42499999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79200000000000004,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55200000000000005,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.52100000000000002}]},{"osd":2,"last update":"Mon Feb 26 04:56:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56899999999999995,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.33500000000000002,"15min":0.28999999999999998},"max":{"1min":0.81899999999999995,"5min":1.006,"15min":1.006},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60899999999999999,"15min":0.55700000000000005},"min":{"1min":0.45000000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Mon Feb 26 04:56:28 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.32000000000000001}]},{"osd":4,"last update":"Mon Feb 26 04:56:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.625,"15min":0.56100000000000005},"min":{"1min":0.45700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.55400000000000005},"min":{"1min":0.44700000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.63900000000000001}]},{"osd":5,"last update":"Mon Feb 26 04:56:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58499999999999996,"15min":0.55300000000000005},"min":{"1min":0.39200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79100000000000004,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63500000000000001,"15min":0.58899999999999997},"min":{"1min":0.41799999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0740000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.45700000000000002}]},{"osd":6,"last update":"Mon Feb 26 04:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.374,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.35399999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.38}]},{"osd":7,"last update":"Mon Feb 26 04:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.65600000000000003}]},{"osd":8,"last update":"Mon Feb 26 04:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.58999999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.745,"15min":0.754},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93400000000000005,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.58299999999999996}]},{"osd":9,"last update":"Mon Feb 26 04:56:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.46500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.621},{"interface":"front","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.80100000000000005},"min":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.56299999999999994}]},{"osd":10,"last update":"Mon Feb 26 04:56:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.83599999999999997,"15min":0.84199999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.80500000000000005,"15min":0.81899999999999995},"min":{"1min":0.55600000000000005,"5min":0.5,"15min":0.5},"max":{"1min":0.85699999999999998,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.74299999999999999}]},{"osd":11,"last update":"Mon Feb 26 04:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72799999999999998,"15min":0.72599999999999998},"min":{"1min":0.53200000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.89500000000000002,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78100000000000003,"15min":0.78200000000000003},"min":{"1min":0.629,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.004,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.71399999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":560835,"data_compressed":22182,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.53900000000000003,"15min":0.47299999999999998},"min":{"1min":0.41599999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.96399999999999997,"5min":1.002,"15min":1.002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53000000000000003,"15min":0.47399999999999998},"min":{"1min":0.378,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69399999999999995}]},{"osd":1,"last update":"Mon Feb 26 04:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.50600000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.432},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.57899999999999996,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.48099999999999998}]},{"osd":3,"last update":"Mon Feb 26 04:56:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.59499999999999997,"15min":0.55300000000000005},"min":{"1min":0.45800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.61099999999999999,"15min":0.57499999999999996},"min":{"1min":0.51700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83399999999999996}]},{"osd":4,"last update":"Mon Feb 26 04:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61499999999999999}]},{"osd":5,"last update":"Mon Feb 26 04:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.29599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74399999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.46800000000000003,"15min":0.44600000000000001},"min":{"1min":0.32800000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77800000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.378}]},{"osd":6,"last update":"Mon Feb 26 04:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.60699999999999998,"15min":0.55900000000000005},"min":{"1min":0.54100000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.60799999999999998,"15min":0.56599999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80400000000000005}]},{"osd":7,"last update":"Mon Feb 26 04:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.59599999999999997}]},{"osd":8,"last update":"Mon Feb 26 04:56:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.97799999999999998,"5min":1.115,"15min":1.115},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.44900000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.85399999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.50800000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.68899999999999995,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.73699999999999999,"15min":0.71899999999999997},"min":{"1min":0.48699999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.042,"5min":1.056,"15min":1.056},"last":0.77300000000000002}]},{"osd":10,"last update":"Mon Feb 26 04:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.72099999999999997,"15min":0.69399999999999995},"min":{"1min":0.57099999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.59199999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.78500000000000003}]},{"osd":11,"last update":"Mon Feb 26 04:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.81899999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1003520,"data_stored":496413,"data_compressed":20559,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:20 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34699999999999998,"15min":0.33500000000000002},"min":{"1min":0.28199999999999997,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.35199999999999998,"15min":0.33400000000000002},"min":{"1min":0.28199999999999997,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.38100000000000001}]},{"osd":1,"last update":"Mon Feb 26 04:56:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55600000000000005,"15min":0.52200000000000002},"min":{"1min":0.46300000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.42599999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75800000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.72099999999999997}]},{"osd":2,"last update":"Mon Feb 26 04:56:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.55100000000000005},"min":{"1min":0.36699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80900000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.42099999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88400000000000001,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Mon Feb 26 04:56:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.42199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89200000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63800000000000001,"15min":0.61899999999999999},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85899999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76100000000000001}]},{"osd":5,"last update":"Mon Feb 26 04:56:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.45700000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.751},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.436,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.75700000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59299999999999997}]},{"osd":6,"last update":"Mon Feb 26 04:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.504,"15min":0.47799999999999998},"min":{"1min":0.34000000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71199999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.441},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53500000000000003,"15min":0.51200000000000001},"min":{"1min":0.33700000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66900000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.626}]},{"osd":7,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66600000000000004,"15min":0.629},"min":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95999999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.61599999999999999}]},{"osd":8,"last update":"Mon Feb 26 04:55:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.874,"5min":0.995,"15min":0.995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.878,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63100000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90500000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56000000000000005}]},{"osd":10,"last update":"Mon Feb 26 04:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.89600000000000002,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.98399999999999999,"5min":1.232,"15min":1.232},"last":0.70199999999999996}]},{"osd":11,"last update":"Mon Feb 26 04:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73499999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":500869,"data_compressed":20559,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:54:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.51100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79500000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42899999999999999}]},{"osd":1,"last update":"Mon Feb 26 04:56:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.41899999999999998,"15min":0.373},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.53800000000000003,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.433},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.42199999999999999,"15min":0.38600000000000001},"min":{"1min":0.29999999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.72499999999999998,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.36399999999999999}]},{"osd":2,"last update":"Mon Feb 26 04:56:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.52700000000000002},"min":{"1min":0.39500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90300000000000002,"5min":1.657,"15min":1.657},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.57699999999999996,"15min":0.54300000000000004},"min":{"1min":0.44900000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.749,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.66600000000000004}]},{"osd":3,"last update":"Mon Feb 26 04:56:41 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60999999999999999,"15min":0.55900000000000005},"min":{"1min":0.42199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.75800000000000001,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":0.498},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.53900000000000003},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.745,"5min":2.133,"15min":2.133},"last":0.46999999999999997}]},{"osd":5,"last update":"Mon Feb 26 04:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.57599999999999996,"15min":0.54800000000000004},"min":{"1min":0.46300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.60799999999999998,"15min":0.55600000000000005},"min":{"1min":0.52300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.59899999999999998}]},{"osd":6,"last update":"Mon Feb 26 04:56:36 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.627,"15min":0.58399999999999996},"min":{"1min":0.44900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80300000000000005,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.68100000000000005,"15min":0.65000000000000002},"min":{"1min":0.46800000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85099999999999998,"5min":4.1639999999999997,"15min":4.1639999999999997},"last":0.57299999999999995}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.55300000000000005},"min":{"1min":0.34200000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.55500000000000005},"min":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":4.2300000000000004,"15min":4.2300000000000004},"last":0.71099999999999997}]},{"osd":8,"last update":"Mon Feb 26 04:56:26 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69899999999999995,"15min":0.68100000000000005},"min":{"1min":0.57299999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.66400000000000003,"15min":0.63800000000000001},"min":{"1min":0.505,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.63500000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:56:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.82499999999999996,"15min":0.747},"min":{"1min":0.54600000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94399999999999995,"5min":10.051,"15min":10.051},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81799999999999995,"15min":0.76100000000000001},"min":{"1min":0.63,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94099999999999995,"5min":6.3360000000000003,"15min":6.3360000000000003},"last":0.72799999999999998}]},{"osd":10,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.78600000000000003,"15min":0.77900000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.754},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71599999999999997,"15min":0.70899999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.68300000000000005}]},{"osd":11,"last update":"Mon Feb 26 04:56:26 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":0.745}]}]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":1092069,"data_compressed":56342,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.495,"15min":0.48499999999999999},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48499999999999999,"15min":0.48599999999999999},"min":{"1min":0.32400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67400000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.499}]},{"osd":1,"last update":"Mon Feb 26 04:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.42999999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75600000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.39500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76700000000000002,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Mon Feb 26 04:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.36199999999999999},"min":{"1min":0.27600000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.40100000000000002,"15min":0.38100000000000001},"min":{"1min":0.30099999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.42199999999999999}]},{"osd":3,"last update":"Mon Feb 26 04:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52600000000000002,"15min":0.498},"min":{"1min":0.45500000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58099999999999996}]},{"osd":4,"last update":"Mon Feb 26 04:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.61599999999999999,"15min":0.58599999999999997},"min":{"1min":0.47699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.57199999999999995,"15min":0.55400000000000005},"min":{"1min":0.41599999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.70999999999999996}]},{"osd":6,"last update":"Mon Feb 26 04:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61199999999999999,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68200000000000005}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67400000000000004,"15min":0.65700000000000003},"min":{"1min":0.46500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89600000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.73899999999999999}]},{"osd":8,"last update":"Mon Feb 26 04:56:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.434,"5min":0.316,"15min":0.316},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.47199999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76700000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.75600000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:56:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.67600000000000005,"15min":0.67000000000000004},"min":{"1min":0.49199999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.70699999999999996,"15min":0.68500000000000005},"min":{"1min":0.50700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80900000000000005}]},{"osd":10,"last update":"Mon Feb 26 04:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.52800000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95899999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.82299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":63356,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64706,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64934,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T04:57:10.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph pg dump --format=json 2024-02-26T04:57:10.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:10 smithi156 ceph-mon[35805]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:10.615 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:57:11.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:11 smithi191 ceph-mon[36352]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:11.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:11 smithi156 ceph-mon[35805]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:11.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:11 smithi060 ceph-mon[32382]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:12.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:12 smithi060 ceph-mon[32382]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:57:12.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:12 smithi191 ceph-mon[36352]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:57:12.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:12 smithi156 ceph-mon[35805]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:57:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.513+0000 7f8a8704c700 1 -- 172.21.15.60:0/3454713486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a80106ca0 msgr2=0x7f8a80107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.513+0000 7f8a8704c700 1 --2- 172.21.15.60:0/3454713486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a80106ca0 0x7f8a80107080 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009990 tx=0x7f8a74009ca0 comp rx=0 tx=0).stop 2024-02-26T04:57:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.513+0000 7f8a8704c700 1 -- 172.21.15.60:0/3454713486 shutdown_connections 2024-02-26T04:57:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.513+0000 7f8a8704c700 1 --2- 172.21.15.60:0/3454713486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a80101810 0x7f8a8010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.513+0000 7f8a8704c700 1 --2- 172.21.15.60:0/3454713486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a80106ca0 0x7f8a80107080 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.513+0000 7f8a8704c700 1 --2- 172.21.15.60:0/3454713486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a80100e50 0x7f8a801012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.513+0000 7f8a8704c700 1 -- 172.21.15.60:0/3454713486 >> 172.21.15.60:0/3454713486 conn(0x7f8a80077a30 msgr2=0x7f8a80077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.513+0000 7f8a8704c700 1 -- 172.21.15.60:0/3454713486 shutdown_connections 2024-02-26T04:57:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.513+0000 7f8a8704c700 1 -- 172.21.15.60:0/3454713486 wait complete. 2024-02-26T04:57:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.513+0000 7f8a8704c700 1 Processor -- start 2024-02-26T04:57:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.513+0000 7f8a8704c700 1 -- start start 2024-02-26T04:57:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a8704c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a80100e50 0x7f8a80198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a8704c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a80101810 0x7f8a80198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a8704c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a80106ca0 0x7f8a8019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a8704c700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8019cfb0 con 0x7f8a80100e50 2024-02-26T04:57:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a8704c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8019d120 con 0x7f8a80101810 2024-02-26T04:57:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a8704c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8019d2e0 con 0x7f8a80106ca0 2024-02-26T04:57:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a84de8700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a80100e50 0x7f8a80198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a84de8700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a80100e50 0x7f8a80198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41540/0 (socket says 172.21.15.60:41540) 2024-02-26T04:57:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a84de8700 1 -- 172.21.15.60:0/803610383 learned_addr learned my addr 172.21.15.60:0/803610383 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a855e9700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a80106ca0 0x7f8a8019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a7ffff700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a80101810 0x7f8a80198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a84de8700 1 -- 172.21.15.60:0/803610383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a80106ca0 msgr2=0x7f8a8019c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a84de8700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a80106ca0 0x7f8a8019c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a84de8700 1 -- 172.21.15.60:0/803610383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a80101810 msgr2=0x7f8a80198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a84de8700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a80101810 0x7f8a80198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a84de8700 1 -- 172.21.15.60:0/803610383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a740095f0 con 0x7f8a80100e50 2024-02-26T04:57:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a855e9700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a80106ca0 0x7f8a8019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a7ffff700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a80101810 0x7f8a80198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a84de8700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a80100e50 0x7f8a80198120 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8a740098e0 tx=0x7f8a7400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a7dffb700 1 -- 172.21.15.60:0/803610383 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7401d070 con 0x7f8a80100e50 2024-02-26T04:57:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a8704c700 1 -- 172.21.15.60:0/803610383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8019d560 con 0x7f8a80100e50 2024-02-26T04:57:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a7dffb700 1 -- 172.21.15.60:0/803610383 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a7400fc70 con 0x7f8a80100e50 2024-02-26T04:57:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.514+0000 7f8a7dffb700 1 -- 172.21.15.60:0/803610383 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a740176e0 con 0x7f8a80100e50 2024-02-26T04:57:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.515+0000 7f8a8704c700 1 -- 172.21.15.60:0/803610383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a801a4e20 con 0x7f8a80100e50 2024-02-26T04:57:12.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.516+0000 7f8a8704c700 1 -- 172.21.15.60:0/803610383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a801090f0 con 0x7f8a80100e50 2024-02-26T04:57:12.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.517+0000 7f8a7dffb700 1 -- 172.21.15.60:0/803610383 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f8a740229e0 con 0x7f8a80100e50 2024-02-26T04:57:12.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.517+0000 7f8a7dffb700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a700a21e0 0x7f8a700a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:12.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.518+0000 7f8a7ffff700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a700a21e0 0x7f8a700a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:12.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.518+0000 7f8a7dffb700 1 -- 172.21.15.60:0/803610383 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f8a740cf710 con 0x7f8a80100e50 2024-02-26T04:57:12.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.518+0000 7f8a7ffff700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a700a21e0 0x7f8a700a46a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c005fd0 tx=0x7f8a6c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:12.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.521+0000 7f8a7dffb700 1 -- 172.21.15.60:0/803610383 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8a740982b0 con 0x7f8a80100e50 2024-02-26T04:57:12.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.718+0000 7f8a8704c700 1 -- 172.21.15.60:0/803610383 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a80199430 con 0x7f8a700a21e0 2024-02-26T04:57:12.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.732+0000 7f8a7dffb700 1 -- 172.21.15.60:0/803610383 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427953 (secure 0 0 0) 0x7f8a80199430 con 0x7f8a700a21e0 2024-02-26T04:57:12.740 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.736+0000 7f8a8704c700 1 -- 172.21.15.60:0/803610383 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a700a21e0 msgr2=0x7f8a700a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.736+0000 7f8a8704c700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a700a21e0 0x7f8a700a46a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c005fd0 tx=0x7f8a6c005e20 comp rx=0 tx=0).stop 2024-02-26T04:57:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.736+0000 7f8a8704c700 1 -- 172.21.15.60:0/803610383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a80100e50 msgr2=0x7f8a80198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.736+0000 7f8a8704c700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a80100e50 0x7f8a80198120 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8a740098e0 tx=0x7f8a7400f740 comp rx=0 tx=0).stop 2024-02-26T04:57:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.736+0000 7f8a8704c700 1 -- 172.21.15.60:0/803610383 shutdown_connections 2024-02-26T04:57:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.736+0000 7f8a8704c700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a80106ca0 0x7f8a8019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.736+0000 7f8a8704c700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a80100e50 0x7f8a80198120 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.736+0000 7f8a8704c700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a700a21e0 0x7f8a700a46a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.736+0000 7f8a8704c700 1 --2- 172.21.15.60:0/803610383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a80101810 0x7f8a80198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.736+0000 7f8a8704c700 1 -- 172.21.15.60:0/803610383 >> 172.21.15.60:0/803610383 conn(0x7f8a80077a30 msgr2=0x7f8a800ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:12.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.736+0000 7f8a8704c700 1 -- 172.21.15.60:0/803610383 shutdown_connections 2024-02-26T04:57:12.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:12.736+0000 7f8a8704c700 1 -- 172.21.15.60:0/803610383 wait complete. 2024-02-26T04:57:12.744 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-02-26T04:57:13.621 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-02-26T04:57:12.333117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657568,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":608,"num_read_kb":419,"num_write":343,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386324,"kb_used_data":12152,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555500,"statfs":{"total":1151940427776,"available":1151544832000,"internally_reserved":0,"allocated":12443648,"data_stored":7958318,"data_compressed":358926,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003406"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508738+0000","last_change":"2024-02-26T04:55:28.208832+0000","last_active":"2024-02-26T04:56:20.508738+0000","last_peered":"2024-02-26T04:56:20.508738+0000","last_clean":"2024-02-26T04:56:20.508738+0000","last_became_active":"2024-02-26T04:55:28.208454+0000","last_became_peered":"2024-02-26T04:55:28.208454+0000","last_unstale":"2024-02-26T04:56:20.508738+0000","last_undegraded":"2024-02-26T04:56:20.508738+0000","last_fullsized":"2024-02-26T04:56:20.508738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:55:54.132384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201989+0000","last_change":"2024-02-26T04:55:28.203876+0000","last_active":"2024-02-26T04:56:20.201989+0000","last_peered":"2024-02-26T04:56:20.201989+0000","last_clean":"2024-02-26T04:56:20.201989+0000","last_became_active":"2024-02-26T04:55:28.203399+0000","last_became_peered":"2024-02-26T04:55:28.203399+0000","last_unstale":"2024-02-26T04:56:20.201989+0000","last_undegraded":"2024-02-26T04:56:20.201989+0000","last_fullsized":"2024-02-26T04:56:20.201989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:31:40.927929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200636+0000","last_change":"2024-02-26T04:55:28.211388+0000","last_active":"2024-02-26T04:56:20.200636+0000","last_peered":"2024-02-26T04:56:20.200636+0000","last_clean":"2024-02-26T04:56:20.200636+0000","last_became_active":"2024-02-26T04:55:28.209007+0000","last_became_peered":"2024-02-26T04:55:28.209007+0000","last_unstale":"2024-02-26T04:56:20.200636+0000","last_undegraded":"2024-02-26T04:56:20.200636+0000","last_fullsized":"2024-02-26T04:56:20.200636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:37:01.197589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.198030+0000","last_change":"2024-02-26T04:55:28.216378+0000","last_active":"2024-02-26T04:56:20.198030+0000","last_peered":"2024-02-26T04:56:20.198030+0000","last_clean":"2024-02-26T04:56:20.198030+0000","last_became_active":"2024-02-26T04:55:28.216074+0000","last_became_peered":"2024-02-26T04:55:28.216074+0000","last_unstale":"2024-02-26T04:56:20.198030+0000","last_undegraded":"2024-02-26T04:56:20.198030+0000","last_fullsized":"2024-02-26T04:56:20.198030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:15:20.460699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197626+0000","last_change":"2024-02-26T04:55:28.213026+0000","last_active":"2024-02-26T04:56:20.197626+0000","last_peered":"2024-02-26T04:56:20.197626+0000","last_clean":"2024-02-26T04:56:20.197626+0000","last_became_active":"2024-02-26T04:55:28.212706+0000","last_became_peered":"2024-02-26T04:55:28.212706+0000","last_unstale":"2024-02-26T04:56:20.197626+0000","last_undegraded":"2024-02-26T04:56:20.197626+0000","last_fullsized":"2024-02-26T04:56:20.197626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:57:35.961956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.206795+0000","last_change":"2024-02-26T04:55:28.202123+0000","last_active":"2024-02-26T04:56:20.206795+0000","last_peered":"2024-02-26T04:56:20.206795+0000","last_clean":"2024-02-26T04:56:20.206795+0000","last_became_active":"2024-02-26T04:55:28.201733+0000","last_became_peered":"2024-02-26T04:55:28.201733+0000","last_unstale":"2024-02-26T04:56:20.206795+0000","last_undegraded":"2024-02-26T04:56:20.206795+0000","last_fullsized":"2024-02-26T04:56:20.206795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:43:11.300006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508346+0000","last_change":"2024-02-26T04:56:20.508346+0000","last_active":"2024-02-26T04:56:20.508346+0000","last_peered":"2024-02-26T04:56:20.508346+0000","last_clean":"2024-02-26T04:56:20.508346+0000","last_became_active":"2024-02-26T04:56:20.507941+0000","last_became_peered":"2024-02-26T04:56:20.507941+0000","last_unstale":"2024-02-26T04:56:20.508346+0000","last_undegraded":"2024-02-26T04:56:20.508346+0000","last_fullsized":"2024-02-26T04:56:20.508346+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:35:27.425477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197071+0000","last_change":"2024-02-26T04:55:28.214631+0000","last_active":"2024-02-26T04:56:20.197071+0000","last_peered":"2024-02-26T04:56:20.197071+0000","last_clean":"2024-02-26T04:56:20.197071+0000","last_became_active":"2024-02-26T04:55:28.214290+0000","last_became_peered":"2024-02-26T04:55:28.214290+0000","last_unstale":"2024-02-26T04:56:20.197071+0000","last_undegraded":"2024-02-26T04:56:20.197071+0000","last_fullsized":"2024-02-26T04:56:20.197071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:30:11.494833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508783+0000","last_change":"2024-02-26T04:56:20.508783+0000","last_active":"2024-02-26T04:56:20.508783+0000","last_peered":"2024-02-26T04:56:20.508783+0000","last_clean":"2024-02-26T04:56:20.508783+0000","last_became_active":"2024-02-26T04:56:20.508372+0000","last_became_peered":"2024-02-26T04:56:20.508372+0000","last_unstale":"2024-02-26T04:56:20.508783+0000","last_undegraded":"2024-02-26T04:56:20.508783+0000","last_fullsized":"2024-02-26T04:56:20.508783+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:32.415022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.207364+0000","last_change":"2024-02-26T04:55:28.204176+0000","last_active":"2024-02-26T04:56:20.207364+0000","last_peered":"2024-02-26T04:56:20.207364+0000","last_clean":"2024-02-26T04:56:20.207364+0000","last_became_active":"2024-02-26T04:55:28.203821+0000","last_became_peered":"2024-02-26T04:55:28.203821+0000","last_unstale":"2024-02-26T04:56:20.207364+0000","last_undegraded":"2024-02-26T04:56:20.207364+0000","last_fullsized":"2024-02-26T04:56:20.207364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:24:03.132650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199686+0000","last_change":"2024-02-26T04:55:28.215958+0000","last_active":"2024-02-26T04:56:20.199686+0000","last_peered":"2024-02-26T04:56:20.199686+0000","last_clean":"2024-02-26T04:56:20.199686+0000","last_became_active":"2024-02-26T04:55:28.215594+0000","last_became_peered":"2024-02-26T04:55:28.215594+0000","last_unstale":"2024-02-26T04:56:20.199686+0000","last_undegraded":"2024-02-26T04:56:20.199686+0000","last_fullsized":"2024-02-26T04:56:20.199686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:45:47.915379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197890+0000","last_change":"2024-02-26T04:55:28.218392+0000","last_active":"2024-02-26T04:56:20.197890+0000","last_peered":"2024-02-26T04:56:20.197890+0000","last_clean":"2024-02-26T04:56:20.197890+0000","last_became_active":"2024-02-26T04:55:28.217933+0000","last_became_peered":"2024-02-26T04:55:28.217933+0000","last_unstale":"2024-02-26T04:56:20.197890+0000","last_undegraded":"2024-02-26T04:56:20.197890+0000","last_fullsized":"2024-02-26T04:56:20.197890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:30:38.880649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507627+0000","last_change":"2024-02-26T04:55:28.197099+0000","last_active":"2024-02-26T04:56:20.507627+0000","last_peered":"2024-02-26T04:56:20.507627+0000","last_clean":"2024-02-26T04:56:20.507627+0000","last_became_active":"2024-02-26T04:55:28.196755+0000","last_became_peered":"2024-02-26T04:55:28.196755+0000","last_unstale":"2024-02-26T04:56:20.507627+0000","last_undegraded":"2024-02-26T04:56:20.507627+0000","last_fullsized":"2024-02-26T04:56:20.507627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:37:31.685845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199949+0000","last_change":"2024-02-26T04:55:28.212064+0000","last_active":"2024-02-26T04:56:20.199949+0000","last_peered":"2024-02-26T04:56:20.199949+0000","last_clean":"2024-02-26T04:56:20.199949+0000","last_became_active":"2024-02-26T04:55:28.208742+0000","last_became_peered":"2024-02-26T04:55:28.208742+0000","last_unstale":"2024-02-26T04:56:20.199949+0000","last_undegraded":"2024-02-26T04:56:20.199949+0000","last_fullsized":"2024-02-26T04:56:20.199949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:38:19.742468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.511014+0000","last_change":"2024-02-26T04:56:20.511014+0000","last_active":"2024-02-26T04:56:20.511014+0000","last_peered":"2024-02-26T04:56:20.511014+0000","last_clean":"2024-02-26T04:56:20.511014+0000","last_became_active":"2024-02-26T04:56:20.510463+0000","last_became_peered":"2024-02-26T04:56:20.510463+0000","last_unstale":"2024-02-26T04:56:20.511014+0000","last_undegraded":"2024-02-26T04:56:20.511014+0000","last_fullsized":"2024-02-26T04:56:20.511014+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:44:59.574588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506436+0000","last_change":"2024-02-26T04:55:28.222810+0000","last_active":"2024-02-26T04:56:20.506436+0000","last_peered":"2024-02-26T04:56:20.506436+0000","last_clean":"2024-02-26T04:56:20.506436+0000","last_became_active":"2024-02-26T04:55:28.222341+0000","last_became_peered":"2024-02-26T04:55:28.222341+0000","last_unstale":"2024-02-26T04:56:20.506436+0000","last_undegraded":"2024-02-26T04:56:20.506436+0000","last_fullsized":"2024-02-26T04:56:20.506436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:59:21.627517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508612+0000","last_change":"2024-02-26T04:55:28.208377+0000","last_active":"2024-02-26T04:56:20.508612+0000","last_peered":"2024-02-26T04:56:20.508612+0000","last_clean":"2024-02-26T04:56:20.508612+0000","last_became_active":"2024-02-26T04:55:28.207932+0000","last_became_peered":"2024-02-26T04:55:28.207932+0000","last_unstale":"2024-02-26T04:56:20.508612+0000","last_undegraded":"2024-02-26T04:56:20.508612+0000","last_fullsized":"2024-02-26T04:56:20.508612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:15:39.586191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200068+0000","last_change":"2024-02-26T04:55:28.201105+0000","last_active":"2024-02-26T04:56:20.200068+0000","last_peered":"2024-02-26T04:56:20.200068+0000","last_clean":"2024-02-26T04:56:20.200068+0000","last_became_active":"2024-02-26T04:55:28.200796+0000","last_became_peered":"2024-02-26T04:55:28.200796+0000","last_unstale":"2024-02-26T04:56:20.200068+0000","last_undegraded":"2024-02-26T04:56:20.200068+0000","last_fullsized":"2024-02-26T04:56:20.200068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:53:43.713055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.509323+0000","last_change":"2024-02-26T04:55:28.215725+0000","last_active":"2024-02-26T04:56:20.509323+0000","last_peered":"2024-02-26T04:56:20.509323+0000","last_clean":"2024-02-26T04:56:20.509323+0000","last_became_active":"2024-02-26T04:55:28.215341+0000","last_became_peered":"2024-02-26T04:55:28.215341+0000","last_unstale":"2024-02-26T04:56:20.509323+0000","last_undegraded":"2024-02-26T04:56:20.509323+0000","last_fullsized":"2024-02-26T04:56:20.509323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:26:26.960474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506987+0000","last_change":"2024-02-26T04:55:32.233303+0000","last_active":"2024-02-26T04:56:20.506987+0000","last_peered":"2024-02-26T04:56:20.506987+0000","last_clean":"2024-02-26T04:56:20.506987+0000","last_became_active":"2024-02-26T04:55:32.232901+0000","last_became_peered":"2024-02-26T04:55:32.232901+0000","last_unstale":"2024-02-26T04:56:20.506987+0000","last_undegraded":"2024-02-26T04:56:20.506987+0000","last_fullsized":"2024-02-26T04:56:20.506987+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:37:49.659668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201603+0000","last_change":"2024-02-26T04:55:27.199682+0000","last_active":"2024-02-26T04:56:20.201603+0000","last_peered":"2024-02-26T04:56:20.201603+0000","last_clean":"2024-02-26T04:56:20.201603+0000","last_became_active":"2024-02-26T04:55:27.199293+0000","last_became_peered":"2024-02-26T04:55:27.199293+0000","last_unstale":"2024-02-26T04:56:20.201603+0000","last_undegraded":"2024-02-26T04:56:20.201603+0000","last_fullsized":"2024-02-26T04:56:20.201603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:23:21.685615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.511029+0000","last_change":"2024-02-26T04:56:20.511029+0000","last_active":"2024-02-26T04:56:20.511029+0000","last_peered":"2024-02-26T04:56:20.511029+0000","last_clean":"2024-02-26T04:56:20.511029+0000","last_became_active":"2024-02-26T04:56:20.510590+0000","last_became_peered":"2024-02-26T04:56:20.510590+0000","last_unstale":"2024-02-26T04:56:20.511029+0000","last_undegraded":"2024-02-26T04:56:20.511029+0000","last_fullsized":"2024-02-26T04:56:20.511029+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:01:25.641444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197376+0000","last_change":"2024-02-26T04:55:32.233142+0000","last_active":"2024-02-26T04:56:20.197376+0000","last_peered":"2024-02-26T04:56:20.197376+0000","last_clean":"2024-02-26T04:56:20.197376+0000","last_became_active":"2024-02-26T04:55:32.232722+0000","last_became_peered":"2024-02-26T04:55:32.232722+0000","last_unstale":"2024-02-26T04:56:20.197376+0000","last_undegraded":"2024-02-26T04:56:20.197376+0000","last_fullsized":"2024-02-26T04:56:20.197376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:30:20.906104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197798+0000","last_change":"2024-02-26T04:55:27.203420+0000","last_active":"2024-02-26T04:56:20.197798+0000","last_peered":"2024-02-26T04:56:20.197798+0000","last_clean":"2024-02-26T04:56:20.197798+0000","last_became_active":"2024-02-26T04:55:27.202968+0000","last_became_peered":"2024-02-26T04:55:27.202968+0000","last_unstale":"2024-02-26T04:56:20.197798+0000","last_undegraded":"2024-02-26T04:56:20.197798+0000","last_fullsized":"2024-02-26T04:56:20.197798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:44.496528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201808+0000","last_change":"2024-02-26T04:55:28.217422+0000","last_active":"2024-02-26T04:56:20.201808+0000","last_peered":"2024-02-26T04:56:20.201808+0000","last_clean":"2024-02-26T04:56:20.201808+0000","last_became_active":"2024-02-26T04:55:28.217103+0000","last_became_peered":"2024-02-26T04:55:28.217103+0000","last_unstale":"2024-02-26T04:56:20.201808+0000","last_undegraded":"2024-02-26T04:56:20.201808+0000","last_fullsized":"2024-02-26T04:56:20.201808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:08:37.075467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197565+0000","last_change":"2024-02-26T04:55:32.233322+0000","last_active":"2024-02-26T04:56:20.197565+0000","last_peered":"2024-02-26T04:56:20.197565+0000","last_clean":"2024-02-26T04:56:20.197565+0000","last_became_active":"2024-02-26T04:55:32.232679+0000","last_became_peered":"2024-02-26T04:55:32.232679+0000","last_unstale":"2024-02-26T04:56:20.197565+0000","last_undegraded":"2024-02-26T04:56:20.197565+0000","last_fullsized":"2024-02-26T04:56:20.197565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:10.775924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.198176+0000","last_change":"2024-02-26T04:55:27.205960+0000","last_active":"2024-02-26T04:56:20.198176+0000","last_peered":"2024-02-26T04:56:20.198176+0000","last_clean":"2024-02-26T04:56:20.198176+0000","last_became_active":"2024-02-26T04:55:27.205604+0000","last_became_peered":"2024-02-26T04:55:27.205604+0000","last_unstale":"2024-02-26T04:56:20.198176+0000","last_undegraded":"2024-02-26T04:56:20.198176+0000","last_fullsized":"2024-02-26T04:56:20.198176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:45:25.002672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506983+0000","last_change":"2024-02-26T04:55:28.210669+0000","last_active":"2024-02-26T04:56:20.506983+0000","last_peered":"2024-02-26T04:56:20.506983+0000","last_clean":"2024-02-26T04:56:20.506983+0000","last_became_active":"2024-02-26T04:55:28.210322+0000","last_became_peered":"2024-02-26T04:55:28.210322+0000","last_unstale":"2024-02-26T04:56:20.506983+0000","last_undegraded":"2024-02-26T04:56:20.506983+0000","last_fullsized":"2024-02-26T04:56:20.506983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:57:29.121762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197404+0000","last_change":"2024-02-26T04:55:32.229077+0000","last_active":"2024-02-26T04:56:20.197404+0000","last_peered":"2024-02-26T04:56:20.197404+0000","last_clean":"2024-02-26T04:56:20.197404+0000","last_became_active":"2024-02-26T04:55:32.228670+0000","last_became_peered":"2024-02-26T04:55:32.228670+0000","last_unstale":"2024-02-26T04:56:20.197404+0000","last_undegraded":"2024-02-26T04:56:20.197404+0000","last_fullsized":"2024-02-26T04:56:20.197404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:35:20.843351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197770+0000","last_change":"2024-02-26T04:55:27.206088+0000","last_active":"2024-02-26T04:56:20.197770+0000","last_peered":"2024-02-26T04:56:20.197770+0000","last_clean":"2024-02-26T04:56:20.197770+0000","last_became_active":"2024-02-26T04:55:27.205696+0000","last_became_peered":"2024-02-26T04:55:27.205696+0000","last_unstale":"2024-02-26T04:56:20.197770+0000","last_undegraded":"2024-02-26T04:56:20.197770+0000","last_fullsized":"2024-02-26T04:56:20.197770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:14:03.552251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508805+0000","last_change":"2024-02-26T04:56:20.508805+0000","last_active":"2024-02-26T04:56:20.508805+0000","last_peered":"2024-02-26T04:56:20.508805+0000","last_clean":"2024-02-26T04:56:20.508805+0000","last_became_active":"2024-02-26T04:56:20.507796+0000","last_became_peered":"2024-02-26T04:56:20.507796+0000","last_unstale":"2024-02-26T04:56:20.508805+0000","last_undegraded":"2024-02-26T04:56:20.508805+0000","last_fullsized":"2024-02-26T04:56:20.508805+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:25.453111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507484+0000","last_change":"2024-02-26T04:55:32.228629+0000","last_active":"2024-02-26T04:56:20.507484+0000","last_peered":"2024-02-26T04:56:20.507484+0000","last_clean":"2024-02-26T04:56:20.507484+0000","last_became_active":"2024-02-26T04:55:32.227608+0000","last_became_peered":"2024-02-26T04:55:32.227608+0000","last_unstale":"2024-02-26T04:56:20.507484+0000","last_undegraded":"2024-02-26T04:56:20.507484+0000","last_fullsized":"2024-02-26T04:56:20.507484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:10:42.429352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":111,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:57:09.255436+0000","last_change":"2024-02-26T04:55:27.201261+0000","last_active":"2024-02-26T04:57:09.255436+0000","last_peered":"2024-02-26T04:57:09.255436+0000","last_clean":"2024-02-26T04:57:09.255436+0000","last_became_active":"2024-02-26T04:55:27.200892+0000","last_became_peered":"2024-02-26T04:55:27.200892+0000","last_unstale":"2024-02-26T04:57:09.255436+0000","last_undegraded":"2024-02-26T04:57:09.255436+0000","last_fullsized":"2024-02-26T04:57:09.255436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:34:32.774143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197918+0000","last_change":"2024-02-26T04:55:28.200445+0000","last_active":"2024-02-26T04:56:20.197918+0000","last_peered":"2024-02-26T04:56:20.197918+0000","last_clean":"2024-02-26T04:56:20.197918+0000","last_became_active":"2024-02-26T04:55:28.199959+0000","last_became_peered":"2024-02-26T04:55:28.199959+0000","last_unstale":"2024-02-26T04:56:20.197918+0000","last_undegraded":"2024-02-26T04:56:20.197918+0000","last_fullsized":"2024-02-26T04:56:20.197918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:36:27.397644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506917+0000","last_change":"2024-02-26T04:55:32.232619+0000","last_active":"2024-02-26T04:56:20.506917+0000","last_peered":"2024-02-26T04:56:20.506917+0000","last_clean":"2024-02-26T04:56:20.506917+0000","last_became_active":"2024-02-26T04:55:32.232112+0000","last_became_peered":"2024-02-26T04:55:32.232112+0000","last_unstale":"2024-02-26T04:56:20.506917+0000","last_undegraded":"2024-02-26T04:56:20.506917+0000","last_fullsized":"2024-02-26T04:56:20.506917+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:19.741393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.206715+0000","last_change":"2024-02-26T04:55:27.197376+0000","last_active":"2024-02-26T04:56:20.206715+0000","last_peered":"2024-02-26T04:56:20.206715+0000","last_clean":"2024-02-26T04:56:20.206715+0000","last_became_active":"2024-02-26T04:55:27.195258+0000","last_became_peered":"2024-02-26T04:55:27.195258+0000","last_unstale":"2024-02-26T04:56:20.206715+0000","last_undegraded":"2024-02-26T04:56:20.206715+0000","last_fullsized":"2024-02-26T04:56:20.206715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:41:51.284525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200987+0000","last_change":"2024-02-26T04:55:28.214337+0000","last_active":"2024-02-26T04:56:20.200987+0000","last_peered":"2024-02-26T04:56:20.200987+0000","last_clean":"2024-02-26T04:56:20.200987+0000","last_became_active":"2024-02-26T04:55:28.213966+0000","last_became_peered":"2024-02-26T04:55:28.213966+0000","last_unstale":"2024-02-26T04:56:20.200987+0000","last_undegraded":"2024-02-26T04:56:20.200987+0000","last_fullsized":"2024-02-26T04:56:20.200987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:42.974084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507266+0000","last_change":"2024-02-26T04:55:32.235579+0000","last_active":"2024-02-26T04:56:20.507266+0000","last_peered":"2024-02-26T04:56:20.507266+0000","last_clean":"2024-02-26T04:56:20.507266+0000","last_became_active":"2024-02-26T04:55:32.235241+0000","last_became_peered":"2024-02-26T04:55:32.235241+0000","last_unstale":"2024-02-26T04:56:20.507266+0000","last_undegraded":"2024-02-26T04:56:20.507266+0000","last_fullsized":"2024-02-26T04:56:20.507266+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:06.319874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507236+0000","last_change":"2024-02-26T04:55:27.197689+0000","last_active":"2024-02-26T04:56:20.507236+0000","last_peered":"2024-02-26T04:56:20.507236+0000","last_clean":"2024-02-26T04:56:20.507236+0000","last_became_active":"2024-02-26T04:55:27.197285+0000","last_became_peered":"2024-02-26T04:55:27.197285+0000","last_unstale":"2024-02-26T04:56:20.507236+0000","last_undegraded":"2024-02-26T04:56:20.507236+0000","last_fullsized":"2024-02-26T04:56:20.507236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:45:36.688902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.196871+0000","last_change":"2024-02-26T04:55:28.201902+0000","last_active":"2024-02-26T04:56:20.196871+0000","last_peered":"2024-02-26T04:56:20.196871+0000","last_clean":"2024-02-26T04:56:20.196871+0000","last_became_active":"2024-02-26T04:55:28.201604+0000","last_became_peered":"2024-02-26T04:55:28.201604+0000","last_unstale":"2024-02-26T04:56:20.196871+0000","last_undegraded":"2024-02-26T04:56:20.196871+0000","last_fullsized":"2024-02-26T04:56:20.196871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:48:57.711555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201806+0000","last_change":"2024-02-26T04:55:32.230153+0000","last_active":"2024-02-26T04:56:20.201806+0000","last_peered":"2024-02-26T04:56:20.201806+0000","last_clean":"2024-02-26T04:56:20.201806+0000","last_became_active":"2024-02-26T04:55:32.229695+0000","last_became_peered":"2024-02-26T04:55:32.229695+0000","last_unstale":"2024-02-26T04:56:20.201806+0000","last_undegraded":"2024-02-26T04:56:20.201806+0000","last_fullsized":"2024-02-26T04:56:20.201806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:15:39.575727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200217+0000","last_change":"2024-02-26T04:55:27.200943+0000","last_active":"2024-02-26T04:56:20.200217+0000","last_peered":"2024-02-26T04:56:20.200217+0000","last_clean":"2024-02-26T04:56:20.200217+0000","last_became_active":"2024-02-26T04:55:27.200565+0000","last_became_peered":"2024-02-26T04:55:27.200565+0000","last_unstale":"2024-02-26T04:56:20.200217+0000","last_undegraded":"2024-02-26T04:56:20.200217+0000","last_fullsized":"2024-02-26T04:56:20.200217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:54:19.670631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199990+0000","last_change":"2024-02-26T04:55:28.212232+0000","last_active":"2024-02-26T04:56:20.199990+0000","last_peered":"2024-02-26T04:56:20.199990+0000","last_clean":"2024-02-26T04:56:20.199990+0000","last_became_active":"2024-02-26T04:55:28.211875+0000","last_became_peered":"2024-02-26T04:55:28.211875+0000","last_unstale":"2024-02-26T04:56:20.199990+0000","last_undegraded":"2024-02-26T04:56:20.199990+0000","last_fullsized":"2024-02-26T04:56:20.199990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:55.379271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.196973+0000","last_change":"2024-02-26T04:55:32.234961+0000","last_active":"2024-02-26T04:56:20.196973+0000","last_peered":"2024-02-26T04:56:20.196973+0000","last_clean":"2024-02-26T04:56:20.196973+0000","last_became_active":"2024-02-26T04:55:32.234455+0000","last_became_peered":"2024-02-26T04:55:32.234455+0000","last_unstale":"2024-02-26T04:56:20.196973+0000","last_undegraded":"2024-02-26T04:56:20.196973+0000","last_fullsized":"2024-02-26T04:56:20.196973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:00:44.322341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":106,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:57:08.253054+0000","last_change":"2024-02-26T04:55:27.199036+0000","last_active":"2024-02-26T04:57:08.253054+0000","last_peered":"2024-02-26T04:57:08.253054+0000","last_clean":"2024-02-26T04:57:08.253054+0000","last_became_active":"2024-02-26T04:55:27.198694+0000","last_became_peered":"2024-02-26T04:55:27.198694+0000","last_unstale":"2024-02-26T04:57:08.253054+0000","last_undegraded":"2024-02-26T04:57:08.253054+0000","last_fullsized":"2024-02-26T04:57:08.253054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:31:41.935123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.208627+0000","last_change":"2024-02-26T04:56:20.208105+0000","last_active":"2024-02-26T04:56:20.208627+0000","last_peered":"2024-02-26T04:56:20.208627+0000","last_clean":"2024-02-26T04:56:20.208627+0000","last_became_active":"2024-02-26T04:56:20.207691+0000","last_became_peered":"2024-02-26T04:56:20.207691+0000","last_unstale":"2024-02-26T04:56:20.208627+0000","last_undegraded":"2024-02-26T04:56:20.208627+0000","last_fullsized":"2024-02-26T04:56:20.208627+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:49:49.019089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,9],"acting":[8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199839+0000","last_change":"2024-02-26T04:55:32.228033+0000","last_active":"2024-02-26T04:56:20.199839+0000","last_peered":"2024-02-26T04:56:20.199839+0000","last_clean":"2024-02-26T04:56:20.199839+0000","last_became_active":"2024-02-26T04:55:32.224273+0000","last_became_peered":"2024-02-26T04:55:32.224273+0000","last_unstale":"2024-02-26T04:56:20.199839+0000","last_undegraded":"2024-02-26T04:56:20.199839+0000","last_fullsized":"2024-02-26T04:56:20.199839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:25:13.618421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200035+0000","last_change":"2024-02-26T04:55:27.202833+0000","last_active":"2024-02-26T04:56:20.200035+0000","last_peered":"2024-02-26T04:56:20.200035+0000","last_clean":"2024-02-26T04:56:20.200035+0000","last_became_active":"2024-02-26T04:55:27.202167+0000","last_became_peered":"2024-02-26T04:55:27.202167+0000","last_unstale":"2024-02-26T04:56:20.200035+0000","last_undegraded":"2024-02-26T04:56:20.200035+0000","last_fullsized":"2024-02-26T04:56:20.200035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:55:46.605763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.207930+0000","last_change":"2024-02-26T04:56:20.207930+0000","last_active":"2024-02-26T04:56:20.207930+0000","last_peered":"2024-02-26T04:56:20.207930+0000","last_clean":"2024-02-26T04:56:20.207930+0000","last_became_active":"2024-02-26T04:56:20.207586+0000","last_became_peered":"2024-02-26T04:56:20.207586+0000","last_unstale":"2024-02-26T04:56:20.207930+0000","last_undegraded":"2024-02-26T04:56:20.207930+0000","last_fullsized":"2024-02-26T04:56:20.207930+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:37:57.141645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197794+0000","last_change":"2024-02-26T04:55:32.225736+0000","last_active":"2024-02-26T04:56:20.197794+0000","last_peered":"2024-02-26T04:56:20.197794+0000","last_clean":"2024-02-26T04:56:20.197794+0000","last_became_active":"2024-02-26T04:55:32.225186+0000","last_became_peered":"2024-02-26T04:55:32.225186+0000","last_unstale":"2024-02-26T04:56:20.197794+0000","last_undegraded":"2024-02-26T04:56:20.197794+0000","last_fullsized":"2024-02-26T04:56:20.197794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:13:48.999538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201071+0000","last_change":"2024-02-26T04:55:27.200389+0000","last_active":"2024-02-26T04:56:20.201071+0000","last_peered":"2024-02-26T04:56:20.201071+0000","last_clean":"2024-02-26T04:56:20.201071+0000","last_became_active":"2024-02-26T04:55:27.200019+0000","last_became_peered":"2024-02-26T04:55:27.200019+0000","last_unstale":"2024-02-26T04:56:20.201071+0000","last_undegraded":"2024-02-26T04:56:20.201071+0000","last_fullsized":"2024-02-26T04:56:20.201071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:59:11.820988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507226+0000","last_change":"2024-02-26T04:55:28.220309+0000","last_active":"2024-02-26T04:56:20.507226+0000","last_peered":"2024-02-26T04:56:20.507226+0000","last_clean":"2024-02-26T04:56:20.507226+0000","last_became_active":"2024-02-26T04:55:28.219987+0000","last_became_peered":"2024-02-26T04:55:28.219987+0000","last_unstale":"2024-02-26T04:56:20.507226+0000","last_undegraded":"2024-02-26T04:56:20.507226+0000","last_fullsized":"2024-02-26T04:56:20.507226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:39:34.713802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200592+0000","last_change":"2024-02-26T04:55:32.236817+0000","last_active":"2024-02-26T04:56:20.200592+0000","last_peered":"2024-02-26T04:56:20.200592+0000","last_clean":"2024-02-26T04:56:20.200592+0000","last_became_active":"2024-02-26T04:55:32.235818+0000","last_became_peered":"2024-02-26T04:55:32.235818+0000","last_unstale":"2024-02-26T04:56:20.200592+0000","last_undegraded":"2024-02-26T04:56:20.200592+0000","last_fullsized":"2024-02-26T04:56:20.200592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:56:54.121684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'33","reported_seq":220,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:57:09.271571+0000","last_change":"2024-02-26T04:55:27.201582+0000","last_active":"2024-02-26T04:57:09.271571+0000","last_peered":"2024-02-26T04:57:09.271571+0000","last_clean":"2024-02-26T04:57:09.271571+0000","last_became_active":"2024-02-26T04:55:27.201011+0000","last_became_peered":"2024-02-26T04:55:27.201011+0000","last_unstale":"2024-02-26T04:57:09.271571+0000","last_undegraded":"2024-02-26T04:57:09.271571+0000","last_fullsized":"2024-02-26T04:57:09.271571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:07:01.888924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63356,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":160,"num_read_kb":142,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.198024+0000","last_change":"2024-02-26T04:55:28.199313+0000","last_active":"2024-02-26T04:56:20.198024+0000","last_peered":"2024-02-26T04:56:20.198024+0000","last_clean":"2024-02-26T04:56:20.198024+0000","last_became_active":"2024-02-26T04:55:28.198973+0000","last_became_peered":"2024-02-26T04:55:28.198973+0000","last_unstale":"2024-02-26T04:56:20.198024+0000","last_undegraded":"2024-02-26T04:56:20.198024+0000","last_fullsized":"2024-02-26T04:56:20.198024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:33:49.005899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201034+0000","last_change":"2024-02-26T04:55:32.234529+0000","last_active":"2024-02-26T04:56:20.201034+0000","last_peered":"2024-02-26T04:56:20.201034+0000","last_clean":"2024-02-26T04:56:20.201034+0000","last_became_active":"2024-02-26T04:55:32.234105+0000","last_became_peered":"2024-02-26T04:55:32.234105+0000","last_unstale":"2024-02-26T04:56:20.201034+0000","last_undegraded":"2024-02-26T04:56:20.201034+0000","last_fullsized":"2024-02-26T04:56:20.201034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:17:40.909267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508685+0000","last_change":"2024-02-26T04:55:27.209178+0000","last_active":"2024-02-26T04:56:20.508685+0000","last_peered":"2024-02-26T04:56:20.508685+0000","last_clean":"2024-02-26T04:56:20.508685+0000","last_became_active":"2024-02-26T04:55:27.208834+0000","last_became_peered":"2024-02-26T04:55:27.208834+0000","last_unstale":"2024-02-26T04:56:20.508685+0000","last_undegraded":"2024-02-26T04:56:20.508685+0000","last_fullsized":"2024-02-26T04:56:20.508685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:28.187333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507311+0000","last_change":"2024-02-26T04:55:28.215208+0000","last_active":"2024-02-26T04:56:20.507311+0000","last_peered":"2024-02-26T04:56:20.507311+0000","last_clean":"2024-02-26T04:56:20.507311+0000","last_became_active":"2024-02-26T04:55:28.212135+0000","last_became_peered":"2024-02-26T04:55:28.212135+0000","last_unstale":"2024-02-26T04:56:20.507311+0000","last_undegraded":"2024-02-26T04:56:20.507311+0000","last_fullsized":"2024-02-26T04:56:20.507311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:51.269557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506939+0000","last_change":"2024-02-26T04:55:32.235439+0000","last_active":"2024-02-26T04:56:20.506939+0000","last_peered":"2024-02-26T04:56:20.506939+0000","last_clean":"2024-02-26T04:56:20.506939+0000","last_became_active":"2024-02-26T04:55:32.234987+0000","last_became_peered":"2024-02-26T04:55:32.234987+0000","last_unstale":"2024-02-26T04:56:20.506939+0000","last_undegraded":"2024-02-26T04:56:20.506939+0000","last_fullsized":"2024-02-26T04:56:20.506939+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:31:00.637846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200477+0000","last_change":"2024-02-26T04:55:27.205498+0000","last_active":"2024-02-26T04:56:20.200477+0000","last_peered":"2024-02-26T04:56:20.200477+0000","last_clean":"2024-02-26T04:56:20.200477+0000","last_became_active":"2024-02-26T04:55:27.205179+0000","last_became_peered":"2024-02-26T04:55:27.205179+0000","last_unstale":"2024-02-26T04:56:20.200477+0000","last_undegraded":"2024-02-26T04:56:20.200477+0000","last_fullsized":"2024-02-26T04:56:20.200477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:02:55.491896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199981+0000","last_change":"2024-02-26T04:55:28.201110+0000","last_active":"2024-02-26T04:56:20.199981+0000","last_peered":"2024-02-26T04:56:20.199981+0000","last_clean":"2024-02-26T04:56:20.199981+0000","last_became_active":"2024-02-26T04:55:28.200725+0000","last_became_peered":"2024-02-26T04:55:28.200725+0000","last_unstale":"2024-02-26T04:56:20.199981+0000","last_undegraded":"2024-02-26T04:56:20.199981+0000","last_fullsized":"2024-02-26T04:56:20.199981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:50.377925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201494+0000","last_change":"2024-02-26T04:55:32.240328+0000","last_active":"2024-02-26T04:56:20.201494+0000","last_peered":"2024-02-26T04:56:20.201494+0000","last_clean":"2024-02-26T04:56:20.201494+0000","last_became_active":"2024-02-26T04:55:32.238662+0000","last_became_peered":"2024-02-26T04:55:32.238662+0000","last_unstale":"2024-02-26T04:56:20.201494+0000","last_undegraded":"2024-02-26T04:56:20.201494+0000","last_fullsized":"2024-02-26T04:56:20.201494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:20:22.938290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:57:09.255556+0000","last_change":"2024-02-26T04:55:27.197338+0000","last_active":"2024-02-26T04:57:09.255556+0000","last_peered":"2024-02-26T04:57:09.255556+0000","last_clean":"2024-02-26T04:57:09.255556+0000","last_became_active":"2024-02-26T04:55:27.196898+0000","last_became_peered":"2024-02-26T04:55:27.196898+0000","last_unstale":"2024-02-26T04:57:09.255556+0000","last_undegraded":"2024-02-26T04:57:09.255556+0000","last_fullsized":"2024-02-26T04:57:09.255556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:27:09.635379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199402+0000","last_change":"2024-02-26T04:55:28.200665+0000","last_active":"2024-02-26T04:56:20.199402+0000","last_peered":"2024-02-26T04:56:20.199402+0000","last_clean":"2024-02-26T04:56:20.199402+0000","last_became_active":"2024-02-26T04:55:28.200329+0000","last_became_peered":"2024-02-26T04:55:28.200329+0000","last_unstale":"2024-02-26T04:56:20.199402+0000","last_undegraded":"2024-02-26T04:56:20.199402+0000","last_fullsized":"2024-02-26T04:56:20.199402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:27.090259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506258+0000","last_change":"2024-02-26T04:55:32.225861+0000","last_active":"2024-02-26T04:56:20.506258+0000","last_peered":"2024-02-26T04:56:20.506258+0000","last_clean":"2024-02-26T04:56:20.506258+0000","last_became_active":"2024-02-26T04:55:32.225187+0000","last_became_peered":"2024-02-26T04:55:32.225187+0000","last_unstale":"2024-02-26T04:56:20.506258+0000","last_undegraded":"2024-02-26T04:56:20.506258+0000","last_fullsized":"2024-02-26T04:56:20.506258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:14:09.077383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197675+0000","last_change":"2024-02-26T04:55:27.200321+0000","last_active":"2024-02-26T04:56:20.197675+0000","last_peered":"2024-02-26T04:56:20.197675+0000","last_clean":"2024-02-26T04:56:20.197675+0000","last_became_active":"2024-02-26T04:55:27.199978+0000","last_became_peered":"2024-02-26T04:55:27.199978+0000","last_unstale":"2024-02-26T04:56:20.197675+0000","last_undegraded":"2024-02-26T04:56:20.197675+0000","last_fullsized":"2024-02-26T04:56:20.197675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:08:20.381010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200440+0000","last_change":"2024-02-26T04:55:28.213940+0000","last_active":"2024-02-26T04:56:20.200440+0000","last_peered":"2024-02-26T04:56:20.200440+0000","last_clean":"2024-02-26T04:56:20.200440+0000","last_became_active":"2024-02-26T04:55:28.213590+0000","last_became_peered":"2024-02-26T04:55:28.213590+0000","last_unstale":"2024-02-26T04:56:20.200440+0000","last_undegraded":"2024-02-26T04:56:20.200440+0000","last_fullsized":"2024-02-26T04:56:20.200440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:29:29.151391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200211+0000","last_change":"2024-02-26T04:55:32.232669+0000","last_active":"2024-02-26T04:56:20.200211+0000","last_peered":"2024-02-26T04:56:20.200211+0000","last_clean":"2024-02-26T04:56:20.200211+0000","last_became_active":"2024-02-26T04:55:32.232288+0000","last_became_peered":"2024-02-26T04:55:32.232288+0000","last_unstale":"2024-02-26T04:56:20.200211+0000","last_undegraded":"2024-02-26T04:56:20.200211+0000","last_fullsized":"2024-02-26T04:56:20.200211+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:31:42.436516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.198355+0000","last_change":"2024-02-26T04:55:27.205993+0000","last_active":"2024-02-26T04:56:20.198355+0000","last_peered":"2024-02-26T04:56:20.198355+0000","last_clean":"2024-02-26T04:56:20.198355+0000","last_became_active":"2024-02-26T04:55:27.205594+0000","last_became_peered":"2024-02-26T04:55:27.205594+0000","last_unstale":"2024-02-26T04:56:20.198355+0000","last_undegraded":"2024-02-26T04:56:20.198355+0000","last_fullsized":"2024-02-26T04:56:20.198355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:44:19.993925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508598+0000","last_change":"2024-02-26T04:55:28.203429+0000","last_active":"2024-02-26T04:56:20.508598+0000","last_peered":"2024-02-26T04:56:20.508598+0000","last_clean":"2024-02-26T04:56:20.508598+0000","last_became_active":"2024-02-26T04:55:28.203002+0000","last_became_peered":"2024-02-26T04:55:28.203002+0000","last_unstale":"2024-02-26T04:56:20.508598+0000","last_undegraded":"2024-02-26T04:56:20.508598+0000","last_fullsized":"2024-02-26T04:56:20.508598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:08:05.872241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197747+0000","last_change":"2024-02-26T04:55:32.239323+0000","last_active":"2024-02-26T04:56:20.197747+0000","last_peered":"2024-02-26T04:56:20.197747+0000","last_clean":"2024-02-26T04:56:20.197747+0000","last_became_active":"2024-02-26T04:55:32.238843+0000","last_became_peered":"2024-02-26T04:55:32.238843+0000","last_unstale":"2024-02-26T04:56:20.197747+0000","last_undegraded":"2024-02-26T04:56:20.197747+0000","last_fullsized":"2024-02-26T04:56:20.197747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:39:56.645429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200638+0000","last_change":"2024-02-26T04:55:27.203222+0000","last_active":"2024-02-26T04:56:20.200638+0000","last_peered":"2024-02-26T04:56:20.200638+0000","last_clean":"2024-02-26T04:56:20.200638+0000","last_became_active":"2024-02-26T04:55:27.202845+0000","last_became_peered":"2024-02-26T04:55:27.202845+0000","last_unstale":"2024-02-26T04:56:20.200638+0000","last_undegraded":"2024-02-26T04:56:20.200638+0000","last_fullsized":"2024-02-26T04:56:20.200638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:16:07.524260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.207212+0000","last_change":"2024-02-26T04:55:28.209122+0000","last_active":"2024-02-26T04:56:20.207212+0000","last_peered":"2024-02-26T04:56:20.207212+0000","last_clean":"2024-02-26T04:56:20.207212+0000","last_became_active":"2024-02-26T04:55:28.207598+0000","last_became_peered":"2024-02-26T04:55:28.207598+0000","last_unstale":"2024-02-26T04:56:20.207212+0000","last_undegraded":"2024-02-26T04:56:20.207212+0000","last_fullsized":"2024-02-26T04:56:20.207212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:02:10.763515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506773+0000","last_change":"2024-02-26T04:55:32.227333+0000","last_active":"2024-02-26T04:56:20.506773+0000","last_peered":"2024-02-26T04:56:20.506773+0000","last_clean":"2024-02-26T04:56:20.506773+0000","last_became_active":"2024-02-26T04:55:32.226956+0000","last_became_peered":"2024-02-26T04:55:32.226956+0000","last_unstale":"2024-02-26T04:56:20.506773+0000","last_undegraded":"2024-02-26T04:56:20.506773+0000","last_fullsized":"2024-02-26T04:56:20.506773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:51:40.710242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199829+0000","last_change":"2024-02-26T04:55:27.198781+0000","last_active":"2024-02-26T04:56:20.199829+0000","last_peered":"2024-02-26T04:56:20.199829+0000","last_clean":"2024-02-26T04:56:20.199829+0000","last_became_active":"2024-02-26T04:55:27.198288+0000","last_became_peered":"2024-02-26T04:55:27.198288+0000","last_unstale":"2024-02-26T04:56:20.199829+0000","last_undegraded":"2024-02-26T04:56:20.199829+0000","last_fullsized":"2024-02-26T04:56:20.199829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:52:18.665816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.206954+0000","last_change":"2024-02-26T04:55:28.213044+0000","last_active":"2024-02-26T04:56:20.206954+0000","last_peered":"2024-02-26T04:56:20.206954+0000","last_clean":"2024-02-26T04:56:20.206954+0000","last_became_active":"2024-02-26T04:55:28.212712+0000","last_became_peered":"2024-02-26T04:55:28.212712+0000","last_unstale":"2024-02-26T04:56:20.206954+0000","last_undegraded":"2024-02-26T04:56:20.206954+0000","last_fullsized":"2024-02-26T04:56:20.206954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:02:09.155551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506311+0000","last_change":"2024-02-26T04:55:32.231867+0000","last_active":"2024-02-26T04:56:20.506311+0000","last_peered":"2024-02-26T04:56:20.506311+0000","last_clean":"2024-02-26T04:56:20.506311+0000","last_became_active":"2024-02-26T04:55:32.231450+0000","last_became_peered":"2024-02-26T04:55:32.231450+0000","last_unstale":"2024-02-26T04:56:20.506311+0000","last_undegraded":"2024-02-26T04:56:20.506311+0000","last_fullsized":"2024-02-26T04:56:20.506311+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:12:14.367427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200434+0000","last_change":"2024-02-26T04:55:28.207746+0000","last_active":"2024-02-26T04:56:20.200434+0000","last_peered":"2024-02-26T04:56:20.200434+0000","last_clean":"2024-02-26T04:56:20.200434+0000","last_became_active":"2024-02-26T04:55:28.207438+0000","last_became_peered":"2024-02-26T04:55:28.207438+0000","last_unstale":"2024-02-26T04:56:20.200434+0000","last_undegraded":"2024-02-26T04:56:20.200434+0000","last_fullsized":"2024-02-26T04:56:20.200434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:09.409635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:57:06.250749+0000","last_change":"2024-02-26T04:55:27.192233+0000","last_active":"2024-02-26T04:57:06.250749+0000","last_peered":"2024-02-26T04:57:06.250749+0000","last_clean":"2024-02-26T04:57:06.250749+0000","last_became_active":"2024-02-26T04:55:27.191868+0000","last_became_peered":"2024-02-26T04:55:27.191868+0000","last_unstale":"2024-02-26T04:57:06.250749+0000","last_undegraded":"2024-02-26T04:57:06.250749+0000","last_fullsized":"2024-02-26T04:57:06.250749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:37:58.430504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200198+0000","last_change":"2024-02-26T04:55:32.226577+0000","last_active":"2024-02-26T04:56:20.200198+0000","last_peered":"2024-02-26T04:56:20.200198+0000","last_clean":"2024-02-26T04:56:20.200198+0000","last_became_active":"2024-02-26T04:55:32.226148+0000","last_became_peered":"2024-02-26T04:55:32.226148+0000","last_unstale":"2024-02-26T04:56:20.200198+0000","last_undegraded":"2024-02-26T04:56:20.200198+0000","last_fullsized":"2024-02-26T04:56:20.200198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:34:53.099211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197342+0000","last_change":"2024-02-26T04:55:28.217865+0000","last_active":"2024-02-26T04:56:20.197342+0000","last_peered":"2024-02-26T04:56:20.197342+0000","last_clean":"2024-02-26T04:56:20.197342+0000","last_became_active":"2024-02-26T04:55:28.217075+0000","last_became_peered":"2024-02-26T04:55:28.217075+0000","last_unstale":"2024-02-26T04:56:20.197342+0000","last_undegraded":"2024-02-26T04:56:20.197342+0000","last_fullsized":"2024-02-26T04:56:20.197342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:30:38.720694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197666+0000","last_change":"2024-02-26T04:55:27.204300+0000","last_active":"2024-02-26T04:56:20.197666+0000","last_peered":"2024-02-26T04:56:20.197666+0000","last_clean":"2024-02-26T04:56:20.197666+0000","last_became_active":"2024-02-26T04:55:27.203933+0000","last_became_peered":"2024-02-26T04:55:27.203933+0000","last_unstale":"2024-02-26T04:56:20.197666+0000","last_undegraded":"2024-02-26T04:56:20.197666+0000","last_fullsized":"2024-02-26T04:56:20.197666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:19:36.247209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199280+0000","last_change":"2024-02-26T04:54:13.145088+0000","last_active":"2024-02-26T04:56:20.199280+0000","last_peered":"2024-02-26T04:56:20.199280+0000","last_clean":"2024-02-26T04:56:20.199280+0000","last_became_active":"2024-02-26T04:54:13.144657+0000","last_became_peered":"2024-02-26T04:54:13.144657+0000","last_unstale":"2024-02-26T04:56:20.199280+0000","last_undegraded":"2024-02-26T04:56:20.199280+0000","last_fullsized":"2024-02-26T04:56:20.199280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:50:45.246330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:50:45.246330+0000","last_clean_scrub_stamp":"2024-02-26T04:50:45.246330+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:00:50.842020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200524+0000","last_change":"2024-02-26T04:55:32.240124+0000","last_active":"2024-02-26T04:56:20.200524+0000","last_peered":"2024-02-26T04:56:20.200524+0000","last_clean":"2024-02-26T04:56:20.200524+0000","last_became_active":"2024-02-26T04:55:32.239624+0000","last_became_peered":"2024-02-26T04:55:32.239624+0000","last_unstale":"2024-02-26T04:56:20.200524+0000","last_undegraded":"2024-02-26T04:56:20.200524+0000","last_fullsized":"2024-02-26T04:56:20.200524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:40:33.770917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507083+0000","last_change":"2024-02-26T04:55:28.204631+0000","last_active":"2024-02-26T04:56:20.507083+0000","last_peered":"2024-02-26T04:56:20.507083+0000","last_clean":"2024-02-26T04:56:20.507083+0000","last_became_active":"2024-02-26T04:55:28.204274+0000","last_became_peered":"2024-02-26T04:55:28.204274+0000","last_unstale":"2024-02-26T04:56:20.507083+0000","last_undegraded":"2024-02-26T04:56:20.507083+0000","last_fullsized":"2024-02-26T04:56:20.507083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:18:18.336015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199214+0000","last_change":"2024-02-26T04:55:27.201719+0000","last_active":"2024-02-26T04:56:20.199214+0000","last_peered":"2024-02-26T04:56:20.199214+0000","last_clean":"2024-02-26T04:56:20.199214+0000","last_became_active":"2024-02-26T04:55:27.199060+0000","last_became_peered":"2024-02-26T04:55:27.199060+0000","last_unstale":"2024-02-26T04:56:20.199214+0000","last_undegraded":"2024-02-26T04:56:20.199214+0000","last_fullsized":"2024-02-26T04:56:20.199214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:37:37.620970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506701+0000","last_change":"2024-02-26T04:55:32.228900+0000","last_active":"2024-02-26T04:56:20.506701+0000","last_peered":"2024-02-26T04:56:20.506701+0000","last_clean":"2024-02-26T04:56:20.506701+0000","last_became_active":"2024-02-26T04:55:32.228262+0000","last_became_peered":"2024-02-26T04:55:32.228262+0000","last_unstale":"2024-02-26T04:56:20.506701+0000","last_undegraded":"2024-02-26T04:56:20.506701+0000","last_fullsized":"2024-02-26T04:56:20.506701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:33.069756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.198038+0000","last_change":"2024-02-26T04:55:27.205264+0000","last_active":"2024-02-26T04:56:20.198038+0000","last_peered":"2024-02-26T04:56:20.198038+0000","last_clean":"2024-02-26T04:56:20.198038+0000","last_became_active":"2024-02-26T04:55:27.204882+0000","last_became_peered":"2024-02-26T04:55:27.204882+0000","last_unstale":"2024-02-26T04:56:20.198038+0000","last_undegraded":"2024-02-26T04:56:20.198038+0000","last_fullsized":"2024-02-26T04:56:20.198038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:14:19.497725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507326+0000","last_change":"2024-02-26T04:55:28.200264+0000","last_active":"2024-02-26T04:56:20.507326+0000","last_peered":"2024-02-26T04:56:20.507326+0000","last_clean":"2024-02-26T04:56:20.507326+0000","last_became_active":"2024-02-26T04:55:28.199913+0000","last_became_peered":"2024-02-26T04:55:28.199913+0000","last_unstale":"2024-02-26T04:56:20.507326+0000","last_undegraded":"2024-02-26T04:56:20.507326+0000","last_fullsized":"2024-02-26T04:56:20.507326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:54:51.529149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.196867+0000","last_change":"2024-02-26T04:55:32.233362+0000","last_active":"2024-02-26T04:56:20.196867+0000","last_peered":"2024-02-26T04:56:20.196867+0000","last_clean":"2024-02-26T04:56:20.196867+0000","last_became_active":"2024-02-26T04:55:32.232907+0000","last_became_peered":"2024-02-26T04:55:32.232907+0000","last_unstale":"2024-02-26T04:56:20.196867+0000","last_undegraded":"2024-02-26T04:56:20.196867+0000","last_fullsized":"2024-02-26T04:56:20.196867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:49:11.889863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200207+0000","last_change":"2024-02-26T04:55:27.207371+0000","last_active":"2024-02-26T04:56:20.200207+0000","last_peered":"2024-02-26T04:56:20.200207+0000","last_clean":"2024-02-26T04:56:20.200207+0000","last_became_active":"2024-02-26T04:55:27.207047+0000","last_became_peered":"2024-02-26T04:55:27.207047+0000","last_unstale":"2024-02-26T04:56:20.200207+0000","last_undegraded":"2024-02-26T04:56:20.200207+0000","last_fullsized":"2024-02-26T04:56:20.200207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:29.575633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197403+0000","last_change":"2024-02-26T04:55:28.207132+0000","last_active":"2024-02-26T04:56:20.197403+0000","last_peered":"2024-02-26T04:56:20.197403+0000","last_clean":"2024-02-26T04:56:20.197403+0000","last_became_active":"2024-02-26T04:55:28.206376+0000","last_became_peered":"2024-02-26T04:55:28.206376+0000","last_unstale":"2024-02-26T04:56:20.197403+0000","last_undegraded":"2024-02-26T04:56:20.197403+0000","last_fullsized":"2024-02-26T04:56:20.197403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:17:53.802386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.509195+0000","last_change":"2024-02-26T04:55:32.226381+0000","last_active":"2024-02-26T04:56:20.509195+0000","last_peered":"2024-02-26T04:56:20.509195+0000","last_clean":"2024-02-26T04:56:20.509195+0000","last_became_active":"2024-02-26T04:55:32.225940+0000","last_became_peered":"2024-02-26T04:55:32.225940+0000","last_unstale":"2024-02-26T04:56:20.509195+0000","last_undegraded":"2024-02-26T04:56:20.509195+0000","last_fullsized":"2024-02-26T04:56:20.509195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:51:20.190514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.198213+0000","last_change":"2024-02-26T04:55:27.200246+0000","last_active":"2024-02-26T04:56:20.198213+0000","last_peered":"2024-02-26T04:56:20.198213+0000","last_clean":"2024-02-26T04:56:20.198213+0000","last_became_active":"2024-02-26T04:55:27.199906+0000","last_became_peered":"2024-02-26T04:55:27.199906+0000","last_unstale":"2024-02-26T04:56:20.198213+0000","last_undegraded":"2024-02-26T04:56:20.198213+0000","last_fullsized":"2024-02-26T04:56:20.198213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:29.037230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.204222+0000","last_change":"2024-02-26T04:56:20.204222+0000","last_active":"2024-02-26T04:56:20.204222+0000","last_peered":"2024-02-26T04:56:20.204222+0000","last_clean":"2024-02-26T04:56:20.204222+0000","last_became_active":"2024-02-26T04:56:20.203726+0000","last_became_peered":"2024-02-26T04:56:20.203726+0000","last_unstale":"2024-02-26T04:56:20.204222+0000","last_undegraded":"2024-02-26T04:56:20.204222+0000","last_fullsized":"2024-02-26T04:56:20.204222+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:37:59.371734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197790+0000","last_change":"2024-02-26T04:55:32.227765+0000","last_active":"2024-02-26T04:56:20.197790+0000","last_peered":"2024-02-26T04:56:20.197790+0000","last_clean":"2024-02-26T04:56:20.197790+0000","last_became_active":"2024-02-26T04:55:32.227346+0000","last_became_peered":"2024-02-26T04:55:32.227346+0000","last_unstale":"2024-02-26T04:56:20.197790+0000","last_undegraded":"2024-02-26T04:56:20.197790+0000","last_fullsized":"2024-02-26T04:56:20.197790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:05:12.634958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506443+0000","last_change":"2024-02-26T04:55:27.199979+0000","last_active":"2024-02-26T04:56:20.506443+0000","last_peered":"2024-02-26T04:56:20.506443+0000","last_clean":"2024-02-26T04:56:20.506443+0000","last_became_active":"2024-02-26T04:55:27.199618+0000","last_became_peered":"2024-02-26T04:55:27.199618+0000","last_unstale":"2024-02-26T04:56:20.506443+0000","last_undegraded":"2024-02-26T04:56:20.506443+0000","last_fullsized":"2024-02-26T04:56:20.506443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:20:59.649473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197522+0000","last_change":"2024-02-26T04:55:28.202253+0000","last_active":"2024-02-26T04:56:20.197522+0000","last_peered":"2024-02-26T04:56:20.197522+0000","last_clean":"2024-02-26T04:56:20.197522+0000","last_became_active":"2024-02-26T04:55:28.199825+0000","last_became_peered":"2024-02-26T04:55:28.199825+0000","last_unstale":"2024-02-26T04:56:20.197522+0000","last_undegraded":"2024-02-26T04:56:20.197522+0000","last_fullsized":"2024-02-26T04:56:20.197522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:21:59.148995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.201305+0000","last_change":"2024-02-26T04:55:32.239327+0000","last_active":"2024-02-26T04:56:20.201305+0000","last_peered":"2024-02-26T04:56:20.201305+0000","last_clean":"2024-02-26T04:56:20.201305+0000","last_became_active":"2024-02-26T04:55:32.238742+0000","last_became_peered":"2024-02-26T04:55:32.238742+0000","last_unstale":"2024-02-26T04:56:20.201305+0000","last_undegraded":"2024-02-26T04:56:20.201305+0000","last_fullsized":"2024-02-26T04:56:20.201305+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:38:08.821549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197054+0000","last_change":"2024-02-26T04:55:27.206529+0000","last_active":"2024-02-26T04:56:20.197054+0000","last_peered":"2024-02-26T04:56:20.197054+0000","last_clean":"2024-02-26T04:56:20.197054+0000","last_became_active":"2024-02-26T04:55:27.206203+0000","last_became_peered":"2024-02-26T04:55:27.206203+0000","last_unstale":"2024-02-26T04:56:20.197054+0000","last_undegraded":"2024-02-26T04:56:20.197054+0000","last_fullsized":"2024-02-26T04:56:20.197054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:24.491832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507816+0000","last_change":"2024-02-26T04:55:28.205915+0000","last_active":"2024-02-26T04:56:20.507816+0000","last_peered":"2024-02-26T04:56:20.507816+0000","last_clean":"2024-02-26T04:56:20.507816+0000","last_became_active":"2024-02-26T04:55:28.205576+0000","last_became_peered":"2024-02-26T04:55:28.205576+0000","last_unstale":"2024-02-26T04:56:20.507816+0000","last_undegraded":"2024-02-26T04:56:20.507816+0000","last_fullsized":"2024-02-26T04:56:20.507816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:18:54.639778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197601+0000","last_change":"2024-02-26T04:55:32.226253+0000","last_active":"2024-02-26T04:56:20.197601+0000","last_peered":"2024-02-26T04:56:20.197601+0000","last_clean":"2024-02-26T04:56:20.197601+0000","last_became_active":"2024-02-26T04:55:32.225815+0000","last_became_peered":"2024-02-26T04:55:32.225815+0000","last_unstale":"2024-02-26T04:56:20.197601+0000","last_undegraded":"2024-02-26T04:56:20.197601+0000","last_fullsized":"2024-02-26T04:56:20.197601+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:22.764756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507987+0000","last_change":"2024-02-26T04:55:27.209066+0000","last_active":"2024-02-26T04:56:20.507987+0000","last_peered":"2024-02-26T04:56:20.507987+0000","last_clean":"2024-02-26T04:56:20.507987+0000","last_became_active":"2024-02-26T04:55:27.208556+0000","last_became_peered":"2024-02-26T04:55:27.208556+0000","last_unstale":"2024-02-26T04:56:20.507987+0000","last_undegraded":"2024-02-26T04:56:20.507987+0000","last_fullsized":"2024-02-26T04:56:20.507987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:14:34.950262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197222+0000","last_change":"2024-02-26T04:55:28.213065+0000","last_active":"2024-02-26T04:56:20.197222+0000","last_peered":"2024-02-26T04:56:20.197222+0000","last_clean":"2024-02-26T04:56:20.197222+0000","last_became_active":"2024-02-26T04:55:28.211112+0000","last_became_peered":"2024-02-26T04:55:28.211112+0000","last_unstale":"2024-02-26T04:56:20.197222+0000","last_undegraded":"2024-02-26T04:56:20.197222+0000","last_fullsized":"2024-02-26T04:56:20.197222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:41:10.070183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197607+0000","last_change":"2024-02-26T04:55:32.229838+0000","last_active":"2024-02-26T04:56:20.197607+0000","last_peered":"2024-02-26T04:56:20.197607+0000","last_clean":"2024-02-26T04:56:20.197607+0000","last_became_active":"2024-02-26T04:55:32.228696+0000","last_became_peered":"2024-02-26T04:55:32.228696+0000","last_unstale":"2024-02-26T04:56:20.197607+0000","last_undegraded":"2024-02-26T04:56:20.197607+0000","last_fullsized":"2024-02-26T04:56:20.197607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:59:42.359301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197678+0000","last_change":"2024-02-26T04:55:27.205688+0000","last_active":"2024-02-26T04:56:20.197678+0000","last_peered":"2024-02-26T04:56:20.197678+0000","last_clean":"2024-02-26T04:56:20.197678+0000","last_became_active":"2024-02-26T04:55:27.205306+0000","last_became_peered":"2024-02-26T04:55:27.205306+0000","last_unstale":"2024-02-26T04:56:20.197678+0000","last_undegraded":"2024-02-26T04:56:20.197678+0000","last_fullsized":"2024-02-26T04:56:20.197678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:23:50.203565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199851+0000","last_change":"2024-02-26T04:55:28.210073+0000","last_active":"2024-02-26T04:56:20.199851+0000","last_peered":"2024-02-26T04:56:20.199851+0000","last_clean":"2024-02-26T04:56:20.199851+0000","last_became_active":"2024-02-26T04:55:28.208475+0000","last_became_peered":"2024-02-26T04:55:28.208475+0000","last_unstale":"2024-02-26T04:56:20.199851+0000","last_undegraded":"2024-02-26T04:56:20.199851+0000","last_fullsized":"2024-02-26T04:56:20.199851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:06:49.513916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506115+0000","last_change":"2024-02-26T04:55:32.228708+0000","last_active":"2024-02-26T04:56:20.506115+0000","last_peered":"2024-02-26T04:56:20.506115+0000","last_clean":"2024-02-26T04:56:20.506115+0000","last_became_active":"2024-02-26T04:55:32.228304+0000","last_became_peered":"2024-02-26T04:55:32.228304+0000","last_unstale":"2024-02-26T04:56:20.506115+0000","last_undegraded":"2024-02-26T04:56:20.506115+0000","last_fullsized":"2024-02-26T04:56:20.506115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:53:02.390712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.509071+0000","last_change":"2024-02-26T04:55:27.199761+0000","last_active":"2024-02-26T04:56:20.509071+0000","last_peered":"2024-02-26T04:56:20.509071+0000","last_clean":"2024-02-26T04:56:20.509071+0000","last_became_active":"2024-02-26T04:55:27.199113+0000","last_became_peered":"2024-02-26T04:55:27.199113+0000","last_unstale":"2024-02-26T04:56:20.509071+0000","last_undegraded":"2024-02-26T04:56:20.509071+0000","last_fullsized":"2024-02-26T04:56:20.509071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:14:10.631886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.199755+0000","last_change":"2024-02-26T04:55:28.211484+0000","last_active":"2024-02-26T04:56:20.199755+0000","last_peered":"2024-02-26T04:56:20.199755+0000","last_clean":"2024-02-26T04:56:20.199755+0000","last_became_active":"2024-02-26T04:55:28.211131+0000","last_became_peered":"2024-02-26T04:55:28.211131+0000","last_unstale":"2024-02-26T04:56:20.199755+0000","last_undegraded":"2024-02-26T04:56:20.199755+0000","last_fullsized":"2024-02-26T04:56:20.199755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:54:09.393847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200334+0000","last_change":"2024-02-26T04:55:32.227493+0000","last_active":"2024-02-26T04:56:20.200334+0000","last_peered":"2024-02-26T04:56:20.200334+0000","last_clean":"2024-02-26T04:56:20.200334+0000","last_became_active":"2024-02-26T04:55:32.227106+0000","last_became_peered":"2024-02-26T04:55:32.227106+0000","last_unstale":"2024-02-26T04:56:20.200334+0000","last_undegraded":"2024-02-26T04:56:20.200334+0000","last_fullsized":"2024-02-26T04:56:20.200334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:33.040966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200737+0000","last_change":"2024-02-26T04:55:27.205331+0000","last_active":"2024-02-26T04:56:20.200737+0000","last_peered":"2024-02-26T04:56:20.200737+0000","last_clean":"2024-02-26T04:56:20.200737+0000","last_became_active":"2024-02-26T04:55:27.204998+0000","last_became_peered":"2024-02-26T04:55:27.204998+0000","last_unstale":"2024-02-26T04:56:20.200737+0000","last_undegraded":"2024-02-26T04:56:20.200737+0000","last_fullsized":"2024-02-26T04:56:20.200737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:10:03.671016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508854+0000","last_change":"2024-02-26T04:56:20.508854+0000","last_active":"2024-02-26T04:56:20.508854+0000","last_peered":"2024-02-26T04:56:20.508854+0000","last_clean":"2024-02-26T04:56:20.508854+0000","last_became_active":"2024-02-26T04:56:20.508396+0000","last_became_peered":"2024-02-26T04:56:20.508396+0000","last_unstale":"2024-02-26T04:56:20.508854+0000","last_undegraded":"2024-02-26T04:56:20.508854+0000","last_fullsized":"2024-02-26T04:56:20.508854+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:47:33.249035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200022+0000","last_change":"2024-02-26T04:55:32.232222+0000","last_active":"2024-02-26T04:56:20.200022+0000","last_peered":"2024-02-26T04:56:20.200022+0000","last_clean":"2024-02-26T04:56:20.200022+0000","last_became_active":"2024-02-26T04:55:32.231836+0000","last_became_peered":"2024-02-26T04:55:32.231836+0000","last_unstale":"2024-02-26T04:56:20.200022+0000","last_undegraded":"2024-02-26T04:56:20.200022+0000","last_fullsized":"2024-02-26T04:56:20.200022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:18:00.766266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507710+0000","last_change":"2024-02-26T04:55:27.200385+0000","last_active":"2024-02-26T04:56:20.507710+0000","last_peered":"2024-02-26T04:56:20.507710+0000","last_clean":"2024-02-26T04:56:20.507710+0000","last_became_active":"2024-02-26T04:55:27.200046+0000","last_became_peered":"2024-02-26T04:55:27.200046+0000","last_unstale":"2024-02-26T04:56:20.507710+0000","last_undegraded":"2024-02-26T04:56:20.507710+0000","last_fullsized":"2024-02-26T04:56:20.507710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:01:58.112457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200220+0000","last_change":"2024-02-26T04:55:28.220702+0000","last_active":"2024-02-26T04:56:20.200220+0000","last_peered":"2024-02-26T04:56:20.200220+0000","last_clean":"2024-02-26T04:56:20.200220+0000","last_became_active":"2024-02-26T04:55:28.220386+0000","last_became_peered":"2024-02-26T04:55:28.220386+0000","last_unstale":"2024-02-26T04:56:20.200220+0000","last_undegraded":"2024-02-26T04:56:20.200220+0000","last_fullsized":"2024-02-26T04:56:20.200220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:40:37.774831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507883+0000","last_change":"2024-02-26T04:55:28.206409+0000","last_active":"2024-02-26T04:56:20.507883+0000","last_peered":"2024-02-26T04:56:20.507883+0000","last_clean":"2024-02-26T04:56:20.507883+0000","last_became_active":"2024-02-26T04:55:28.206046+0000","last_became_peered":"2024-02-26T04:55:28.206046+0000","last_unstale":"2024-02-26T04:56:20.507883+0000","last_undegraded":"2024-02-26T04:56:20.507883+0000","last_fullsized":"2024-02-26T04:56:20.507883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:35:43.277376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.509537+0000","last_change":"2024-02-26T04:55:28.216146+0000","last_active":"2024-02-26T04:56:20.509537+0000","last_peered":"2024-02-26T04:56:20.509537+0000","last_clean":"2024-02-26T04:56:20.509537+0000","last_became_active":"2024-02-26T04:55:28.215800+0000","last_became_peered":"2024-02-26T04:55:28.215800+0000","last_unstale":"2024-02-26T04:56:20.509537+0000","last_undegraded":"2024-02-26T04:56:20.509537+0000","last_fullsized":"2024-02-26T04:56:20.509537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:18:47.902694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507974+0000","last_change":"2024-02-26T04:55:28.221587+0000","last_active":"2024-02-26T04:56:20.507974+0000","last_peered":"2024-02-26T04:56:20.507974+0000","last_clean":"2024-02-26T04:56:20.507974+0000","last_became_active":"2024-02-26T04:55:28.220990+0000","last_became_peered":"2024-02-26T04:55:28.220990+0000","last_unstale":"2024-02-26T04:56:20.507974+0000","last_undegraded":"2024-02-26T04:56:20.507974+0000","last_fullsized":"2024-02-26T04:56:20.507974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:24:13.302081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200761+0000","last_change":"2024-02-26T04:55:28.220586+0000","last_active":"2024-02-26T04:56:20.200761+0000","last_peered":"2024-02-26T04:56:20.200761+0000","last_clean":"2024-02-26T04:56:20.200761+0000","last_became_active":"2024-02-26T04:55:28.219585+0000","last_became_peered":"2024-02-26T04:55:28.219585+0000","last_unstale":"2024-02-26T04:56:20.200761+0000","last_undegraded":"2024-02-26T04:56:20.200761+0000","last_fullsized":"2024-02-26T04:56:20.200761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:41.786142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507368+0000","last_change":"2024-02-26T04:55:28.204599+0000","last_active":"2024-02-26T04:56:20.507368+0000","last_peered":"2024-02-26T04:56:20.507368+0000","last_clean":"2024-02-26T04:56:20.507368+0000","last_became_active":"2024-02-26T04:55:28.204252+0000","last_became_peered":"2024-02-26T04:55:28.204252+0000","last_unstale":"2024-02-26T04:56:20.507368+0000","last_undegraded":"2024-02-26T04:56:20.507368+0000","last_fullsized":"2024-02-26T04:56:20.507368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:30:19.807897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508555+0000","last_change":"2024-02-26T04:55:28.214233+0000","last_active":"2024-02-26T04:56:20.508555+0000","last_peered":"2024-02-26T04:56:20.508555+0000","last_clean":"2024-02-26T04:56:20.508555+0000","last_became_active":"2024-02-26T04:55:28.213927+0000","last_became_peered":"2024-02-26T04:55:28.213927+0000","last_unstale":"2024-02-26T04:56:20.508555+0000","last_undegraded":"2024-02-26T04:56:20.508555+0000","last_fullsized":"2024-02-26T04:56:20.508555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:31:07.967134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.509966+0000","last_change":"2024-02-26T04:55:28.217610+0000","last_active":"2024-02-26T04:56:20.509966+0000","last_peered":"2024-02-26T04:56:20.509966+0000","last_clean":"2024-02-26T04:56:20.509966+0000","last_became_active":"2024-02-26T04:55:28.217263+0000","last_became_peered":"2024-02-26T04:55:28.217263+0000","last_unstale":"2024-02-26T04:56:20.509966+0000","last_undegraded":"2024-02-26T04:56:20.509966+0000","last_fullsized":"2024-02-26T04:56:20.509966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:28:18.831887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.510859+0000","last_change":"2024-02-26T04:56:20.510859+0000","last_active":"2024-02-26T04:56:20.510859+0000","last_peered":"2024-02-26T04:56:20.510859+0000","last_clean":"2024-02-26T04:56:20.510859+0000","last_became_active":"2024-02-26T04:56:20.510420+0000","last_became_peered":"2024-02-26T04:56:20.510420+0000","last_unstale":"2024-02-26T04:56:20.510859+0000","last_undegraded":"2024-02-26T04:56:20.510859+0000","last_fullsized":"2024-02-26T04:56:20.510859+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:02.961996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.200654+0000","last_change":"2024-02-26T04:55:28.217247+0000","last_active":"2024-02-26T04:56:20.200654+0000","last_peered":"2024-02-26T04:56:20.200654+0000","last_clean":"2024-02-26T04:56:20.200654+0000","last_became_active":"2024-02-26T04:55:28.214691+0000","last_became_peered":"2024-02-26T04:55:28.214691+0000","last_unstale":"2024-02-26T04:56:20.200654+0000","last_undegraded":"2024-02-26T04:56:20.200654+0000","last_fullsized":"2024-02-26T04:56:20.200654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:27:45.564520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.197229+0000","last_change":"2024-02-26T04:55:28.210491+0000","last_active":"2024-02-26T04:56:20.197229+0000","last_peered":"2024-02-26T04:56:20.197229+0000","last_clean":"2024-02-26T04:56:20.197229+0000","last_became_active":"2024-02-26T04:55:28.210144+0000","last_became_peered":"2024-02-26T04:55:28.210144+0000","last_unstale":"2024-02-26T04:56:20.197229+0000","last_undegraded":"2024-02-26T04:56:20.197229+0000","last_fullsized":"2024-02-26T04:56:20.197229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:11.214716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.506822+0000","last_change":"2024-02-26T04:55:28.216037+0000","last_active":"2024-02-26T04:56:20.506822+0000","last_peered":"2024-02-26T04:56:20.506822+0000","last_clean":"2024-02-26T04:56:20.506822+0000","last_became_active":"2024-02-26T04:55:28.215712+0000","last_became_peered":"2024-02-26T04:55:28.215712+0000","last_unstale":"2024-02-26T04:56:20.506822+0000","last_undegraded":"2024-02-26T04:56:20.506822+0000","last_fullsized":"2024-02-26T04:56:20.506822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:38:55.437642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.507768+0000","last_change":"2024-02-26T04:55:28.205469+0000","last_active":"2024-02-26T04:56:20.507768+0000","last_peered":"2024-02-26T04:56:20.507768+0000","last_clean":"2024-02-26T04:56:20.507768+0000","last_became_active":"2024-02-26T04:55:28.205137+0000","last_became_peered":"2024-02-26T04:55:28.205137+0000","last_unstale":"2024-02-26T04:56:20.507768+0000","last_undegraded":"2024-02-26T04:56:20.507768+0000","last_fullsized":"2024-02-26T04:56:20.507768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:50.425010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-02-26T04:56:20.508100+0000","last_change":"2024-02-26T04:55:28.213066+0000","last_active":"2024-02-26T04:56:20.508100+0000","last_peered":"2024-02-26T04:56:20.508100+0000","last_clean":"2024-02-26T04:56:20.508100+0000","last_became_active":"2024-02-26T04:55:28.212706+0000","last_became_peered":"2024-02-26T04:55:28.212706+0000","last_unstale":"2024-02-26T04:56:20.508100+0000","last_undegraded":"2024-02-26T04:56:20.508100+0000","last_fullsized":"2024-02-26T04:56:20.508100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:21:24.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281347,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66966,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":482,"num_read_kb":310,"num_write":160,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":200898,"data_compressed":4869,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":500807,"data_compressed":20559,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54900000000000004,"15min":0.55400000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68500000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67100000000000004}]},{"osd":1,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79300000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.46700000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85299999999999998,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.502}]},{"osd":2,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.51400000000000001,"15min":0.52400000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67800000000000005,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.498,"15min":0.498},"min":{"1min":0.36299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":4,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.83499999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.46700000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.82599999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59899999999999998}]},{"osd":5,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61399999999999999,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74099999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.42599999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86699999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.68799999999999994}]},{"osd":6,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.73199999999999998,"15min":0.74299999999999999},"min":{"1min":0.55200000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90000000000000002,"5min":1.175,"15min":1.175},"last":0.58299999999999996}]},{"osd":7,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94199999999999995,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.57699999999999996,"5min":0.435,"15min":0.435},"max":{"1min":0.92800000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.64500000000000002}]},{"osd":8,"last update":"Mon Feb 26 04:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.44},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.47699999999999998}]},{"osd":9,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78700000000000003,"15min":0.78900000000000003},"min":{"1min":0.61099999999999999,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.97399999999999998,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.59999999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.022,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.70899999999999996}]},{"osd":10,"last update":"Mon Feb 26 04:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.81000000000000005},"min":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.97299999999999998,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82099999999999995,"15min":0.82699999999999996},"min":{"1min":0.59699999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.92900000000000005,"5min":1.246,"15min":1.246},"last":0.63200000000000001}]}]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1138688,"data_stored":1151431,"data_compressed":57965,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:29 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":9.6609999999999996,"5min":9.6609999999999996,"15min":9.6609999999999996},"last":0.38900000000000001},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":9.6229999999999993,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":0.36599999999999999}]},{"osd":1,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.497,"15min":0.501},"min":{"1min":0.28599999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.503},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.497,"15min":0.499},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.432}]},{"osd":2,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58599999999999997}]},{"osd":3,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82899999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.432,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69199999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.56399999999999995}]},{"osd":4,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.78300000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56799999999999995,"15min":0.57499999999999996},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.751,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52000000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68500000000000005,"15min":0.68799999999999994},"min":{"1min":0.443,"5min":0.435,"15min":0.435},"max":{"1min":0.91300000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.45900000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.93100000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90400000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.49199999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.94499999999999995,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.65400000000000003}]},{"osd":7,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.72699999999999998}]},{"osd":8,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.75900000000000001,"15min":0.76700000000000002},"min":{"1min":0.441,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0549999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.80700000000000005,"15min":0.81899999999999995},"min":{"1min":0.51800000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97799999999999998,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.75700000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0169999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.82499999999999996,"15min":0.83799999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86599999999999999,"5min":1.274,"15min":1.274},"last":0.72299999999999998}]},{"osd":11,"last update":"Mon Feb 26 04:56:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.034,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.56100000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.997,"5min":1.21,"15min":1.21},"last":0.69699999999999995}]}]},{"osd":9,"up_from":67,"seq":287762808877,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1015808,"data_stored":505139,"data_compressed":20559,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.44400000000000001,"15min":0.47399999999999998},"min":{"1min":0.30299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.54100000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.46300000000000002,"15min":0.49299999999999999},"min":{"1min":0.28100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64900000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.39400000000000002}]},{"osd":1,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60399999999999998,"15min":0.627},"min":{"1min":0.40999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77000000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.64600000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79700000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.46300000000000002}]},{"osd":2,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.64600000000000002,"15min":0.66900000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73499999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.72999999999999998,"15min":0.79300000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77000000000000002,"5min":3.173,"15min":3.173},"last":0.41399999999999998}]},{"osd":3,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.49299999999999999,"15min":0.52100000000000002},"min":{"1min":0.27400000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.60099999999999998,"5min":0.753,"15min":0.753},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.498,"15min":0.51800000000000002},"min":{"1min":0.30599999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64000000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.36599999999999999}]},{"osd":4,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.64500000000000002,"15min":0.68400000000000005},"min":{"1min":0.44400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77500000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63700000000000001,"15min":0.66500000000000004},"min":{"1min":0.36899999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64100000000000001}]},{"osd":5,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.67600000000000005,"15min":0.70199999999999996},"min":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.68799999999999994,"15min":0.71699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.76600000000000001,"5min":1.133,"15min":1.133},"last":0.49399999999999999}]},{"osd":6,"last update":"Mon Feb 26 04:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.71199999999999997,"15min":0.754},"min":{"1min":0.46300000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83299999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.73499999999999999,"15min":0.77200000000000002},"min":{"1min":0.46100000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81899999999999995,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.51900000000000002}]},{"osd":8,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.84799999999999998,"15min":0.91900000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85699999999999998,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76400000000000001,"15min":0.80200000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.73399999999999999}]},{"osd":10,"last update":"Mon Feb 26 04:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.751,"15min":0.76100000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.754}]},{"osd":11,"last update":"Mon Feb 26 04:56:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61599999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":500963,"data_compressed":20559,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.38100000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":1.196,"5min":0.65600000000000003,"15min":0.56999999999999995},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":12.359999999999999,"5min":12.359999999999999,"15min":12.359999999999999},"last":0.54300000000000004}]},{"osd":1,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.47399999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78400000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.63300000000000001,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.81999999999999995}]},{"osd":2,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.44800000000000001,"15min":0.433},"min":{"1min":0.33100000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46300000000000002,"15min":0.44},"min":{"1min":0.33300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.70799999999999996}]},{"osd":3,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.40999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.63200000000000001}]},{"osd":4,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.52400000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.629,"15min":0.61599999999999999},"min":{"1min":0.54800000000000004,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.77000000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.55700000000000005,"15min":0.53600000000000003},"min":{"1min":0.374,"5min":0.309,"15min":0.309},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.879},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.56100000000000005,"15min":0.54100000000000004},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.80400000000000005}]},{"osd":6,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.411,"5min":0.81799999999999995,"15min":0.72299999999999998},"min":{"1min":0.57699999999999996,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":12.384,"5min":12.384,"15min":12.384},"last":0.92500000000000004}]},{"osd":7,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.71899999999999997,"15min":0.70299999999999996},"min":{"1min":0.63700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.70699999999999996,"15min":0.67700000000000005},"min":{"1min":0.61899999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.89900000000000002}]},{"osd":9,"last update":"Mon Feb 26 04:56:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.73699999999999999,"15min":0.70899999999999996},"min":{"1min":0.626,"5min":0.435,"15min":0.435},"max":{"1min":1.0700000000000001,"5min":1.073,"15min":1.073},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.71599999999999997,"15min":0.68300000000000005},"min":{"1min":0.59499999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.998,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.94099999999999995}]},{"osd":10,"last update":"Mon Feb 26 04:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.75600000000000001,"15min":0.73999999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.78200000000000003,"15min":0.76700000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.0629999999999999}]},{"osd":11,"last update":"Mon Feb 26 04:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.87}]}]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":1096481,"data_compressed":56342,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67000000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.40799999999999997}]},{"osd":1,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42399999999999999,"15min":0.41799999999999998},"min":{"1min":0.27900000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45700000000000002,"15min":0.443},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.58999999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.38400000000000001}]},{"osd":2,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82299999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84599999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.621}]},{"osd":3,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.49299999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79600000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81599999999999995,"5min":0.877,"15min":0.877},"last":0.63800000000000001}]},{"osd":4,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.30499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.753,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.54000000000000004}]},{"osd":5,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.46500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86499999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66500000000000004,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67100000000000004}]},{"osd":6,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.46200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.56200000000000006,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85499999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.74399999999999999}]},{"osd":8,"last update":"Mon Feb 26 04:56:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72699999999999998,"15min":0.73299999999999998},"min":{"1min":0.57699999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.754},{"interface":"front","average":{"1min":0.747,"5min":0.71199999999999997,"15min":0.71099999999999997},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.442},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.68899999999999995}]},{"osd":9,"last update":"Mon Feb 26 04:56:50 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0189999999999999,"5min":1.093,"15min":1.093},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.72899999999999998,"15min":0.71499999999999997},"min":{"1min":0.55500000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.97299999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.77200000000000002}]},{"osd":10,"last update":"Mon Feb 26 04:56:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85499999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.80100000000000005}]},{"osd":11,"last update":"Mon Feb 26 04:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.82099999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":496219,"data_compressed":20559,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.41599999999999998,"15min":0.38800000000000001},"min":{"1min":0.30399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67100000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.45800000000000002,"15min":0.42899999999999999},"min":{"1min":0.32300000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.72499999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.47799999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.63,"15min":0.61599999999999999},"min":{"1min":0.48599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.60499999999999998,"15min":0.57399999999999995},"min":{"1min":0.47499999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86199999999999999,"5min":1.032,"15min":1.032},"last":0.40699999999999997}]},{"osd":2,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.60499999999999998,"15min":0.57299999999999995},"min":{"1min":0.45300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.85199999999999998,"15min":1.054},"min":{"1min":0.496,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92600000000000005,"5min":14.884,"15min":14.884},"last":0.59299999999999997}]},{"osd":3,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48299999999999998,"15min":0.45700000000000002},"min":{"1min":0.35499999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70099999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53700000000000003,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.38300000000000001}]},{"osd":4,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.43099999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.99399999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.441},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.90000000000000002,"15min":1.1080000000000001},"min":{"1min":0.45400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96699999999999997,"5min":14.898,"15min":14.898},"last":0.45600000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.64200000000000002},"min":{"1min":0.47099999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.69499999999999995,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.67400000000000004}]},{"osd":7,"last update":"Mon Feb 26 04:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76500000000000001,"15min":0.74199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71399999999999997,"15min":0.69199999999999995},"min":{"1min":0.501,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.044,"5min":1.075,"15min":1.075},"last":0.53700000000000003}]},{"osd":8,"last update":"Mon Feb 26 04:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.56100000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.60899999999999999}]},{"osd":9,"last update":"Mon Feb 26 04:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77300000000000002,"15min":0.80100000000000005},"min":{"1min":0.47399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.76500000000000001,"15min":0.79600000000000004},"min":{"1min":0.48299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.497}]},{"osd":10,"last update":"Mon Feb 26 04:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.871,"15min":0.877},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.622},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.65400000000000003}]},{"osd":11,"last update":"Mon Feb 26 04:56:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.70599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":559485,"data_compressed":22182,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.54600000000000004,"15min":0.499},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.92400000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.55300000000000005,"15min":0.504},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46899999999999997}]},{"osd":2,"last update":"Mon Feb 26 04:56:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.60599999999999998,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59299999999999997,"15min":0.54100000000000004},"min":{"1min":0.48199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83899999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68000000000000005}]},{"osd":3,"last update":"Mon Feb 26 04:56:23 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56899999999999995,"15min":0.55300000000000005},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90300000000000002,"5min":0.997,"15min":0.997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.626,"15min":0.57499999999999996},"min":{"1min":0.439,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.62}]},{"osd":4,"last update":"Mon Feb 26 04:56:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.45600000000000002,"15min":0.42199999999999999},"min":{"1min":0.28299999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.442},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.46300000000000002,"15min":0.42699999999999999},"min":{"1min":0.30099999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58399999999999996}]},{"osd":5,"last update":"Mon Feb 26 04:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61599999999999999,"15min":0.56299999999999994},"min":{"1min":0.47899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.59999999999999998,"15min":0.55100000000000005},"min":{"1min":0.50900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.754}]},{"osd":6,"last update":"Mon Feb 26 04:56:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.48899999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82599999999999996,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.70799999999999996,"15min":0.66300000000000003},"min":{"1min":0.55300000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97799999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.69299999999999995}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.55200000000000005},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88300000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75600000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.65400000000000003}]},{"osd":8,"last update":"Mon Feb 26 04:56:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73999999999999999,"15min":0.71599999999999997},"min":{"1min":0.52400000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.03,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.73399999999999999,"15min":0.71499999999999997},"min":{"1min":0.59399999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.028,"5min":1.04,"15min":1.04},"last":0.72099999999999997}]},{"osd":9,"last update":"Mon Feb 26 04:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1080000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.81100000000000005}]},{"osd":10,"last update":"Mon Feb 26 04:56:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.504,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0720000000000001,"5min":1.095,"15min":1.095},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95999999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.78600000000000003}]},{"osd":11,"last update":"Mon Feb 26 04:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.87},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.82299999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":497607,"data_compressed":20559,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 04:56:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53400000000000003,"15min":0.50700000000000001},"min":{"1min":0.42499999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79200000000000004,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55200000000000005,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.61199999999999999}]},{"osd":2,"last update":"Mon Feb 26 04:56:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56899999999999995,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.33500000000000002,"15min":0.28999999999999998},"max":{"1min":0.81899999999999995,"5min":1.006,"15min":1.006},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60899999999999999,"15min":0.55700000000000005},"min":{"1min":0.45000000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70799999999999996}]},{"osd":3,"last update":"Mon Feb 26 04:56:28 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.47499999999999998}]},{"osd":4,"last update":"Mon Feb 26 04:56:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.625,"15min":0.56100000000000005},"min":{"1min":0.45700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.55400000000000005},"min":{"1min":0.44700000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72599999999999998}]},{"osd":5,"last update":"Mon Feb 26 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60999999999999999,"15min":0.56200000000000006},"min":{"1min":0.42399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85199999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66600000000000004,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.77300000000000002}]},{"osd":6,"last update":"Mon Feb 26 04:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.374,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.35399999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.50600000000000001}]},{"osd":7,"last update":"Mon Feb 26 04:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.84699999999999998}]},{"osd":8,"last update":"Mon Feb 26 04:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.58999999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.745,"15min":0.754},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93400000000000005,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.69899999999999995}]},{"osd":9,"last update":"Mon Feb 26 04:56:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.46500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.80100000000000005},"min":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.91800000000000004}]},{"osd":10,"last update":"Mon Feb 26 04:56:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.83599999999999997,"15min":0.84199999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.80500000000000005,"15min":0.81899999999999995},"min":{"1min":0.55600000000000005,"5min":0.5,"15min":0.5},"max":{"1min":0.85699999999999998,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.747}]},{"osd":11,"last update":"Mon Feb 26 04:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72799999999999998,"15min":0.72599999999999998},"min":{"1min":0.53200000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.89500000000000002,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78100000000000003,"15min":0.78200000000000003},"min":{"1min":0.629,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.004,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.79400000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":560835,"data_compressed":22182,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.53900000000000003,"15min":0.47299999999999998},"min":{"1min":0.41599999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.96399999999999997,"5min":1.002,"15min":1.002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53000000000000003,"15min":0.47399999999999998},"min":{"1min":0.378,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59199999999999997}]},{"osd":1,"last update":"Mon Feb 26 04:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.50600000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.57899999999999996,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61499999999999999}]},{"osd":3,"last update":"Mon Feb 26 04:56:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.59499999999999997,"15min":0.55300000000000005},"min":{"1min":0.45800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.61099999999999999,"15min":0.57499999999999996},"min":{"1min":0.51700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.78800000000000003}]},{"osd":4,"last update":"Mon Feb 26 04:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.63},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.47499999999999998}]},{"osd":5,"last update":"Mon Feb 26 04:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.29599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74399999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.46800000000000003,"15min":0.44600000000000001},"min":{"1min":0.32800000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77800000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.38800000000000001}]},{"osd":6,"last update":"Mon Feb 26 04:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.60699999999999998,"15min":0.55900000000000005},"min":{"1min":0.54100000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.60799999999999998,"15min":0.56599999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Feb 26 04:56:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.97799999999999998,"5min":1.115,"15min":1.115},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.44900000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.85399999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.51000000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.68899999999999995,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.73699999999999999,"15min":0.71899999999999997},"min":{"1min":0.48699999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.042,"5min":1.056,"15min":1.056},"last":0.71299999999999997}]},{"osd":10,"last update":"Mon Feb 26 04:56:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.72099999999999997,"15min":0.69399999999999995},"min":{"1min":0.57099999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.59199999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.68899999999999995}]},{"osd":11,"last update":"Mon Feb 26 04:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.75},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.76300000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1003520,"data_stored":496413,"data_compressed":20559,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:20 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34699999999999998,"15min":0.33500000000000002},"min":{"1min":0.28199999999999997,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.31},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.35199999999999998,"15min":0.33400000000000002},"min":{"1min":0.28199999999999997,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.33300000000000002}]},{"osd":1,"last update":"Mon Feb 26 04:56:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55600000000000005,"15min":0.52200000000000002},"min":{"1min":0.46300000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.42599999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75800000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54800000000000004}]},{"osd":2,"last update":"Mon Feb 26 04:56:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.55100000000000005},"min":{"1min":0.36699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80900000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.42099999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88400000000000001,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Mon Feb 26 04:56:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.42199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89200000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63800000000000001,"15min":0.61899999999999999},"min":{"1min":0.502,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85899999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.68799999999999994}]},{"osd":5,"last update":"Mon Feb 26 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.56899999999999995},"min":{"1min":0.40699999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53700000000000003}]},{"osd":6,"last update":"Mon Feb 26 04:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.504,"15min":0.47799999999999998},"min":{"1min":0.34000000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71199999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.378},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53500000000000003,"15min":0.51200000000000001},"min":{"1min":0.33700000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66900000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46700000000000003}]},{"osd":7,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66600000000000004,"15min":0.629},"min":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95999999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.76900000000000002}]},{"osd":8,"last update":"Mon Feb 26 04:55:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.874,"5min":0.995,"15min":0.995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.878,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63100000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90500000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58999999999999997}]},{"osd":10,"last update":"Mon Feb 26 04:56:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.89600000000000002,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.62},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.98399999999999999,"5min":1.232,"15min":1.232},"last":0.72999999999999998}]},{"osd":11,"last update":"Mon Feb 26 04:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.80400000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":500869,"data_compressed":20559,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:54:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.51100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79500000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42899999999999999}]},{"osd":1,"last update":"Mon Feb 26 04:56:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.41899999999999998,"15min":0.373},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.53800000000000003,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.42199999999999999,"15min":0.38600000000000001},"min":{"1min":0.29999999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.72499999999999998,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.46600000000000003}]},{"osd":2,"last update":"Mon Feb 26 04:56:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.52700000000000002},"min":{"1min":0.39500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90300000000000002,"5min":1.657,"15min":1.657},"last":0.629},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.57699999999999996,"15min":0.54300000000000004},"min":{"1min":0.44900000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.749,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.66700000000000004}]},{"osd":3,"last update":"Mon Feb 26 04:56:41 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60999999999999999,"15min":0.55900000000000005},"min":{"1min":0.42199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.75800000000000001,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.53900000000000003},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.745,"5min":2.133,"15min":2.133},"last":0.79900000000000004}]},{"osd":5,"last update":"Mon Feb 26 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59299999999999997,"15min":0.55400000000000005},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89000000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63,"15min":0.56399999999999995},"min":{"1min":0.41799999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82199999999999995,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.71799999999999997}]},{"osd":6,"last update":"Mon Feb 26 04:56:36 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.627,"15min":0.58399999999999996},"min":{"1min":0.44900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80300000000000005,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.68100000000000005,"15min":0.65000000000000002},"min":{"1min":0.46800000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85099999999999998,"5min":4.1639999999999997,"15min":4.1639999999999997},"last":0.84499999999999997}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.55300000000000005},"min":{"1min":0.34200000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.55500000000000005},"min":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":4.2300000000000004,"15min":4.2300000000000004},"last":0.55500000000000005}]},{"osd":8,"last update":"Mon Feb 26 04:56:26 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69899999999999995,"15min":0.68100000000000005},"min":{"1min":0.57299999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.66400000000000003,"15min":0.63800000000000001},"min":{"1min":0.505,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.68500000000000005}]},{"osd":9,"last update":"Mon Feb 26 04:56:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.82499999999999996,"15min":0.747},"min":{"1min":0.54600000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94399999999999995,"5min":10.051,"15min":10.051},"last":0.752},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81799999999999995,"15min":0.76100000000000001},"min":{"1min":0.63,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94099999999999995,"5min":6.3360000000000003,"15min":6.3360000000000003},"last":0.89500000000000002}]},{"osd":10,"last update":"Mon Feb 26 04:56:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.78600000000000003,"15min":0.77900000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71599999999999997,"15min":0.70899999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.70299999999999996}]},{"osd":11,"last update":"Mon Feb 26 04:56:26 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":0.78000000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":1092069,"data_compressed":56342,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.495,"15min":0.48499999999999999},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48499999999999999,"15min":0.48599999999999999},"min":{"1min":0.32400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.67400000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.70499999999999996}]},{"osd":1,"last update":"Mon Feb 26 04:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.42999999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75600000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.39500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76700000000000002,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.72399999999999998}]},{"osd":2,"last update":"Mon Feb 26 04:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.36199999999999999},"min":{"1min":0.27600000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.40100000000000002,"15min":0.38100000000000001},"min":{"1min":0.30099999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.48499999999999999}]},{"osd":3,"last update":"Mon Feb 26 04:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52600000000000002,"15min":0.498},"min":{"1min":0.45500000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.621}]},{"osd":4,"last update":"Mon Feb 26 04:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.61599999999999999,"15min":0.58599999999999997},"min":{"1min":0.47699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.57199999999999995,"15min":0.55400000000000005},"min":{"1min":0.41599999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.63700000000000001}]},{"osd":6,"last update":"Mon Feb 26 04:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61199999999999999,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.755}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67400000000000004,"15min":0.65700000000000003},"min":{"1min":0.46500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89600000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69199999999999995}]},{"osd":8,"last update":"Mon Feb 26 04:56:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.434,"5min":0.316,"15min":0.316},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.47199999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76700000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.57799999999999996}]},{"osd":9,"last update":"Mon Feb 26 04:56:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.67600000000000005,"15min":0.67000000000000004},"min":{"1min":0.49199999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.70699999999999996,"15min":0.68500000000000005},"min":{"1min":0.50700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.92200000000000004}]},{"osd":10,"last update":"Mon Feb 26 04:56:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.46700000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.52800000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95899999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.78600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":63356,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64706,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64934,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T04:57:13.628 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-26T04:57:13.629 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-26T04:57:13.629 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-26T04:57:13.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph health --format=json 2024-02-26T04:57:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:57:14.251 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:14 smithi060 ceph-mon[32382]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:57:14.251 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:14 smithi060 ceph-mon[32382]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:14.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:14 smithi156 ceph-mon[35805]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:57:14.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:14 smithi156 ceph-mon[35805]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:14.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:14 smithi191 ceph-mon[36352]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:57:14.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:14 smithi191 ceph-mon[36352]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:15.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.627+0000 7f4560133700 1 -- 172.21.15.60:0/2655317170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4558105e80 msgr2=0x7f455810da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:15.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.627+0000 7f4560133700 1 --2- 172.21.15.60:0/2655317170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4558105e80 0x7f455810da50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f454800b0a0 tx=0x7f454800b3b0 comp rx=0 tx=0).stop 2024-02-26T04:57:15.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.627+0000 7f4560133700 1 -- 172.21.15.60:0/2655317170 shutdown_connections 2024-02-26T04:57:15.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.627+0000 7f4560133700 1 --2- 172.21.15.60:0/2655317170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4558105e80 0x7f455810da50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:15.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.627+0000 7f4560133700 1 --2- 172.21.15.60:0/2655317170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4558100dc0 0x7f45581011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:15.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.627+0000 7f4560133700 1 --2- 172.21.15.60:0/2655317170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558101770 0x7f45581057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:15.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.627+0000 7f4560133700 1 -- 172.21.15.60:0/2655317170 >> 172.21.15.60:0/2655317170 conn(0x7f4558077ae0 msgr2=0x7f4558077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:15.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.627+0000 7f4560133700 1 -- 172.21.15.60:0/2655317170 shutdown_connections 2024-02-26T04:57:15.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.627+0000 7f4560133700 1 -- 172.21.15.60:0/2655317170 wait complete. 2024-02-26T04:57:15.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f4560133700 1 Processor -- start 2024-02-26T04:57:15.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f4560133700 1 -- start start 2024-02-26T04:57:15.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f4560133700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4558100dc0 0x7f45581982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:15.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f4560133700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558101770 0x7f45581987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:15.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f4560133700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4558105e80 0x7f455819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:15.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f4560133700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455819d180 con 0x7f4558100dc0 2024-02-26T04:57:15.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f4560133700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455819d2f0 con 0x7f4558101770 2024-02-26T04:57:15.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f4560133700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455819d4b0 con 0x7f4558105e80 2024-02-26T04:57:15.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f455decf700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4558100dc0 0x7f45581982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:15.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f455decf700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4558100dc0 0x7f45581982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41546/0 (socket says 172.21.15.60:41546) 2024-02-26T04:57:15.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f455decf700 1 -- 172.21.15.60:0/2602321976 learned_addr learned my addr 172.21.15.60:0/2602321976 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:15.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f455e6d0700 1 --2- 172.21.15.60:0/2602321976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4558105e80 0x7f455819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:15.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f455d6ce700 1 --2- 172.21.15.60:0/2602321976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558101770 0x7f45581987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:15.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f455decf700 1 -- 172.21.15.60:0/2602321976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4558105e80 msgr2=0x7f455819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:15.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f455decf700 1 --2- 172.21.15.60:0/2602321976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4558105e80 0x7f455819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:15.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f455decf700 1 -- 172.21.15.60:0/2602321976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558101770 msgr2=0x7f45581987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:15.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f455decf700 1 --2- 172.21.15.60:0/2602321976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558101770 0x7f45581987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:15.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.628+0000 7f455decf700 1 -- 172.21.15.60:0/2602321976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4548009d00 con 0x7f4558100dc0 2024-02-26T04:57:15.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.629+0000 7f455decf700 1 --2- 172.21.15.60:0/2602321976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4558100dc0 0x7f45581982a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f455000d950 tx=0x7f455000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:15.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.629+0000 7f4556ffd700 1 -- 172.21.15.60:0/2602321976 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550009940 con 0x7f4558100dc0 2024-02-26T04:57:15.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.629+0000 7f4560133700 1 -- 172.21.15.60:0/2602321976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455819d790 con 0x7f4558100dc0 2024-02-26T04:57:15.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.629+0000 7f4556ffd700 1 -- 172.21.15.60:0/2602321976 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4550010460 con 0x7f4558100dc0 2024-02-26T04:57:15.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.629+0000 7f4556ffd700 1 -- 172.21.15.60:0/2602321976 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455000f620 con 0x7f4558100dc0 2024-02-26T04:57:15.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.629+0000 7f4560133700 1 -- 172.21.15.60:0/2602321976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45581a4ff0 con 0x7f4558100dc0 2024-02-26T04:57:15.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.630+0000 7f4560133700 1 -- 172.21.15.60:0/2602321976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455810a120 con 0x7f4558100dc0 2024-02-26T04:57:15.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.631+0000 7f4556ffd700 1 -- 172.21.15.60:0/2602321976 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f45500105d0 con 0x7f4558100dc0 2024-02-26T04:57:15.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.632+0000 7f4556ffd700 1 --2- 172.21.15.60:0/2602321976 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f45400a21e0 0x7f45400a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.632+0000 7f455d6ce700 1 --2- 172.21.15.60:0/2602321976 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f45400a21e0 0x7f45400a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.632+0000 7f4556ffd700 1 -- 172.21.15.60:0/2602321976 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f45500ce080 con 0x7f4558100dc0 2024-02-26T04:57:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.632+0000 7f455d6ce700 1 --2- 172.21.15.60:0/2602321976 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f45400a21e0 0x7f45400a46a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f454c005950 tx=0x7f454c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:15.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.635+0000 7f4556ffd700 1 -- 172.21.15.60:0/2602321976 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f455009a6c0 con 0x7f4558100dc0 2024-02-26T04:57:15.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.881+0000 7f4560133700 1 -- 172.21.15.60:0/2602321976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4558102280 con 0x7f4558100dc0 2024-02-26T04:57:15.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.882+0000 7f4556ffd700 1 -- 172.21.15.60:0/2602321976 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f455009a250 con 0x7f4558100dc0 2024-02-26T04:57:15.884 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:15.884 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-26T04:57:15.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.884+0000 7f4560133700 1 -- 172.21.15.60:0/2602321976 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f45400a21e0 msgr2=0x7f45400a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:15.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.884+0000 7f4560133700 1 --2- 172.21.15.60:0/2602321976 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f45400a21e0 0x7f45400a46a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f454c005950 tx=0x7f454c0058e0 comp rx=0 tx=0).stop 2024-02-26T04:57:15.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.884+0000 7f4560133700 1 -- 172.21.15.60:0/2602321976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4558100dc0 msgr2=0x7f45581982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:15.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.884+0000 7f4560133700 1 --2- 172.21.15.60:0/2602321976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4558100dc0 0x7f45581982a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f455000d950 tx=0x7f455000dc60 comp rx=0 tx=0).stop 2024-02-26T04:57:15.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.884+0000 7f4560133700 1 -- 172.21.15.60:0/2602321976 shutdown_connections 2024-02-26T04:57:15.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.884+0000 7f4560133700 1 --2- 172.21.15.60:0/2602321976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4558105e80 0x7f455819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:15.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.884+0000 7f4560133700 1 --2- 172.21.15.60:0/2602321976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4558100dc0 0x7f45581982a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:15.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.884+0000 7f4560133700 1 --2- 172.21.15.60:0/2602321976 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f45400a21e0 0x7f45400a46a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:15.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.884+0000 7f4560133700 1 --2- 172.21.15.60:0/2602321976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558101770 0x7f45581987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:15.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.884+0000 7f4560133700 1 -- 172.21.15.60:0/2602321976 >> 172.21.15.60:0/2602321976 conn(0x7f4558077ae0 msgr2=0x7f455806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:15.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.884+0000 7f4560133700 1 -- 172.21.15.60:0/2602321976 shutdown_connections 2024-02-26T04:57:15.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:15.884+0000 7f4560133700 1 -- 172.21.15.60:0/2602321976 wait complete. 2024-02-26T04:57:16.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:16 smithi060 ceph-mon[32382]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:16.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:16 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2602321976' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-26T04:57:16.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:16 smithi156 ceph-mon[35805]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:16.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:16 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2602321976' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-26T04:57:16.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:16 smithi191 ceph-mon[36352]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:16.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:16 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2602321976' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-26T04:57:16.747 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-26T04:57:16.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:57:16.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 -- 172.21.15.60:0/291311589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d80101c70 msgr2=0x7f2d80108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:16.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 --2- 172.21.15.60:0/291311589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d80101c70 0x7f2d80108500 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009900 tx=0x7f2d7002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:16.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 -- 172.21.15.60:0/291311589 shutdown_connections 2024-02-26T04:57:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 --2- 172.21.15.60:0/291311589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d80101c70 0x7f2d80108500 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 --2- 172.21.15.60:0/291311589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d801012b0 0x7f2d80101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 --2- 172.21.15.60:0/291311589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d801000c0 0x7f2d801004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 -- 172.21.15.60:0/291311589 >> 172.21.15.60:0/291311589 conn(0x7f2d800fb630 msgr2=0x7f2d800fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 -- 172.21.15.60:0/291311589 shutdown_connections 2024-02-26T04:57:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 -- 172.21.15.60:0/291311589 wait complete. 2024-02-26T04:57:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 Processor -- start 2024-02-26T04:57:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 -- start start 2024-02-26T04:57:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d801000c0 0x7f2d80197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d801012b0 0x7f2d801976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d80101c70 0x7f2d8019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8019ec80 con 0x7f2d801000c0 2024-02-26T04:57:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8019edf0 con 0x7f2d801012b0 2024-02-26T04:57:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.857+0000 7f2d86b74640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8019efb0 con 0x7f2d80101c70 2024-02-26T04:57:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d84910640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d801000c0 0x7f2d80197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d84910640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d801000c0 0x7f2d80197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41562/0 (socket says 172.21.15.60:41562) 2024-02-26T04:57:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d84910640 1 -- 172.21.15.60:0/665476788 learned_addr learned my addr 172.21.15.60:0/665476788 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d85111640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d80101c70 0x7f2d8019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d7ffff640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d801012b0 0x7f2d801976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d84910640 1 -- 172.21.15.60:0/665476788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d80101c70 msgr2=0x7f2d8019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d84910640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d80101c70 0x7f2d8019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d84910640 1 -- 172.21.15.60:0/665476788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d801012b0 msgr2=0x7f2d801976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d84910640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d801012b0 0x7f2d801976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d84910640 1 -- 172.21.15.60:0/665476788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d70009560 con 0x7f2d801000c0 2024-02-26T04:57:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d85111640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d80101c70 0x7f2d8019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:16.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d84910640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d801000c0 0x7f2d80197190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800ccb0 tx=0x7f2d7800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:16.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d7dffb640 1 -- 172.21.15.60:0/665476788 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78003b70 con 0x7f2d801000c0 2024-02-26T04:57:16.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d7dffb640 1 -- 172.21.15.60:0/665476788 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d7800ae50 con 0x7f2d801000c0 2024-02-26T04:57:16.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d86b74640 1 -- 172.21.15.60:0/665476788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8019f290 con 0x7f2d801000c0 2024-02-26T04:57:16.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d7dffb640 1 -- 172.21.15.60:0/665476788 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7800f7d0 con 0x7f2d801000c0 2024-02-26T04:57:16.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.858+0000 7f2d86b74640 1 -- 172.21.15.60:0/665476788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8019f7a0 con 0x7f2d801000c0 2024-02-26T04:57:16.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.859+0000 7f2d86b74640 1 -- 172.21.15.60:0/665476788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8010ba90 con 0x7f2d801000c0 2024-02-26T04:57:16.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.860+0000 7f2d7dffb640 1 -- 172.21.15.60:0/665476788 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f2d78004000 con 0x7f2d801000c0 2024-02-26T04:57:16.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.861+0000 7f2d7dffb640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2d680a2150 0x7f2d680a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:16.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.861+0000 7f2d7ffff640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2d680a2150 0x7f2d680a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:16.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.861+0000 7f2d7dffb640 1 -- 172.21.15.60:0/665476788 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f2d780ce6f0 con 0x7f2d801000c0 2024-02-26T04:57:16.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.861+0000 7f2d7ffff640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2d680a2150 0x7f2d680a4610 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2d801986a0 tx=0x7f2d74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:16.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:16.865+0000 7f2d7dffb640 1 -- 172.21.15.60:0/665476788 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d78097200 con 0x7f2d801000c0 2024-02-26T04:57:17.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.048+0000 7f2d86b74640 1 -- 172.21.15.60:0/665476788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d8019fc70 con 0x7f2d801000c0 2024-02-26T04:57:17.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.049+0000 7f2d7dffb640 1 -- 172.21.15.60:0/665476788 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24596 (secure 0 0 0) 0x7f2d78003e60 con 0x7f2d801000c0 2024-02-26T04:57:17.052 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:17.052 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":91,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:56:20.193583+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:57:17.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.051+0000 7f2d86b74640 1 -- 172.21.15.60:0/665476788 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2d680a2150 msgr2=0x7f2d680a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.051+0000 7f2d86b74640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2d680a2150 0x7f2d680a4610 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2d801986a0 tx=0x7f2d74005f90 comp rx=0 tx=0).stop 2024-02-26T04:57:17.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.051+0000 7f2d86b74640 1 -- 172.21.15.60:0/665476788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d801000c0 msgr2=0x7f2d80197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.051+0000 7f2d86b74640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d801000c0 0x7f2d80197190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800ccb0 tx=0x7f2d7800a620 comp rx=0 tx=0).stop 2024-02-26T04:57:17.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.051+0000 7f2d86b74640 1 -- 172.21.15.60:0/665476788 shutdown_connections 2024-02-26T04:57:17.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.051+0000 7f2d86b74640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2d680a2150 0x7f2d680a4610 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.051+0000 7f2d86b74640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d80101c70 0x7f2d8019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.051+0000 7f2d86b74640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d801012b0 0x7f2d801976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.051+0000 7f2d86b74640 1 --2- 172.21.15.60:0/665476788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d801000c0 0x7f2d80197190 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.051+0000 7f2d86b74640 1 -- 172.21.15.60:0/665476788 >> 172.21.15.60:0/665476788 conn(0x7f2d800fb630 msgr2=0x7f2d800fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:17.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.051+0000 7f2d86b74640 1 -- 172.21.15.60:0/665476788 shutdown_connections 2024-02-26T04:57:17.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.051+0000 7f2d86b74640 1 -- 172.21.15.60:0/665476788 wait complete. 2024-02-26T04:57:17.065 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T04:50:44.572921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T04:55:25.530720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T04:55:26.741604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T04:55:30.453569+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T04:57:17.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T04:57:17.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.168+0000 7f9db9fbb640 1 -- 172.21.15.60:0/507941141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db4102430 msgr2=0x7f9db41028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.168+0000 7f9db9fbb640 1 --2- 172.21.15.60:0/507941141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db4102430 0x7f9db41028b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009900 tx=0x7f9da802bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:17.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.168+0000 7f9db9fbb640 1 -- 172.21.15.60:0/507941141 shutdown_connections 2024-02-26T04:57:17.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.168+0000 7f9db9fbb640 1 --2- 172.21.15.60:0/507941141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9db4102df0 0x7f9db4109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.168+0000 7f9db9fbb640 1 --2- 172.21.15.60:0/507941141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db4102430 0x7f9db41028b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.168+0000 7f9db9fbb640 1 --2- 172.21.15.60:0/507941141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9db40ff460 0x7f9db40ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.168+0000 7f9db9fbb640 1 -- 172.21.15.60:0/507941141 >> 172.21.15.60:0/507941141 conn(0x7f9db40fad90 msgr2=0x7f9db40fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:17.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.168+0000 7f9db9fbb640 1 -- 172.21.15.60:0/507941141 shutdown_connections 2024-02-26T04:57:17.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.168+0000 7f9db9fbb640 1 -- 172.21.15.60:0/507941141 wait complete. 2024-02-26T04:57:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.169+0000 7f9db9fbb640 1 Processor -- start 2024-02-26T04:57:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.169+0000 7f9db9fbb640 1 -- start start 2024-02-26T04:57:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.169+0000 7f9db9fbb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9db40ff460 0x7f9db4197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.169+0000 7f9db9fbb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9db4102430 0x7f9db4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.169+0000 7f9db9fbb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db4102df0 0x7f9db419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.169+0000 7f9db9fbb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db419ed00 con 0x7f9db4102df0 2024-02-26T04:57:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.169+0000 7f9db9fbb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db419ee70 con 0x7f9db40ff460 2024-02-26T04:57:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.169+0000 7f9db9fbb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db419f030 con 0x7f9db4102430 2024-02-26T04:57:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.169+0000 7f9db3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db4102df0 0x7f9db419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.169+0000 7f9db3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db4102df0 0x7f9db419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41582/0 (socket says 172.21.15.60:41582) 2024-02-26T04:57:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.169+0000 7f9db3fff640 1 -- 172.21.15.60:0/2126453692 learned_addr learned my addr 172.21.15.60:0/2126453692 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.170+0000 7f9db2ffd640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9db4102430 0x7f9db4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:17.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.170+0000 7f9db37fe640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9db40ff460 0x7f9db4197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:17.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.170+0000 7f9db3fff640 1 -- 172.21.15.60:0/2126453692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9db4102430 msgr2=0x7f9db4197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.170+0000 7f9db3fff640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9db4102430 0x7f9db4197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.170+0000 7f9db3fff640 1 -- 172.21.15.60:0/2126453692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9db40ff460 msgr2=0x7f9db4197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.170+0000 7f9db3fff640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9db40ff460 0x7f9db4197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.170+0000 7f9db3fff640 1 -- 172.21.15.60:0/2126453692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da8009560 con 0x7f9db4102df0 2024-02-26T04:57:17.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.170+0000 7f9db37fe640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9db40ff460 0x7f9db4197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:17.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.170+0000 7f9db3fff640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db4102df0 0x7f9db419e7c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00eaa0 tx=0x7f9dac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:17.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.170+0000 7f9db0ff9640 1 -- 172.21.15.60:0/2126453692 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac00cdb0 con 0x7f9db4102df0 2024-02-26T04:57:17.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.170+0000 7f9db0ff9640 1 -- 172.21.15.60:0/2126453692 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9dac005020 con 0x7f9db4102df0 2024-02-26T04:57:17.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.170+0000 7f9db9fbb640 1 -- 172.21.15.60:0/2126453692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db419f310 con 0x7f9db4102df0 2024-02-26T04:57:17.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.170+0000 7f9db0ff9640 1 -- 172.21.15.60:0/2126453692 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0116b0 con 0x7f9db4102df0 2024-02-26T04:57:17.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.170+0000 7f9db9fbb640 1 -- 172.21.15.60:0/2126453692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db419f840 con 0x7f9db4102df0 2024-02-26T04:57:17.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.171+0000 7f9db9fbb640 1 -- 172.21.15.60:0/2126453692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db4108560 con 0x7f9db4102df0 2024-02-26T04:57:17.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.172+0000 7f9db0ff9640 1 -- 172.21.15.60:0/2126453692 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f9dac005520 con 0x7f9db4102df0 2024-02-26T04:57:17.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.173+0000 7f9db0ff9640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9d9c0a21d0 0x7f9d9c0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:17.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.173+0000 7f9db37fe640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9d9c0a21d0 0x7f9d9c0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:17.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.173+0000 7f9db0ff9640 1 -- 172.21.15.60:0/2126453692 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f9dac0ce370 con 0x7f9db4102df0 2024-02-26T04:57:17.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.173+0000 7f9db37fe640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9d9c0a21d0 0x7f9d9c0a4690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9da40058c0 tx=0x7f9da4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:17.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.177+0000 7f9db0ff9640 1 -- 172.21.15.60:0/2126453692 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9dac096e20 con 0x7f9db4102df0 2024-02-26T04:57:17.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.358+0000 7f9db9fbb640 1 -- 172.21.15.60:0/2126453692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9db402ce40 con 0x7f9db4102df0 2024-02-26T04:57:17.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.358+0000 7f9db0ff9640 1 -- 172.21.15.60:0/2126453692 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9dac01f0a0 con 0x7f9db4102df0 2024-02-26T04:57:17.361 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T04:57:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.360+0000 7f9db9fbb640 1 -- 172.21.15.60:0/2126453692 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9d9c0a21d0 msgr2=0x7f9d9c0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.360+0000 7f9db9fbb640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9d9c0a21d0 0x7f9d9c0a4690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9da40058c0 tx=0x7f9da4005850 comp rx=0 tx=0).stop 2024-02-26T04:57:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.360+0000 7f9db9fbb640 1 -- 172.21.15.60:0/2126453692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db4102df0 msgr2=0x7f9db419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.360+0000 7f9db9fbb640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db4102df0 0x7f9db419e7c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00eaa0 tx=0x7f9dac00ef60 comp rx=0 tx=0).stop 2024-02-26T04:57:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.360+0000 7f9db9fbb640 1 -- 172.21.15.60:0/2126453692 shutdown_connections 2024-02-26T04:57:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.360+0000 7f9db9fbb640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f9d9c0a21d0 0x7f9d9c0a4690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.360+0000 7f9db9fbb640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db4102df0 0x7f9db419e7c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.360+0000 7f9db9fbb640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9db4102430 0x7f9db4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.360+0000 7f9db9fbb640 1 --2- 172.21.15.60:0/2126453692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9db40ff460 0x7f9db4197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.360+0000 7f9db9fbb640 1 -- 172.21.15.60:0/2126453692 >> 172.21.15.60:0/2126453692 conn(0x7f9db40fad90 msgr2=0x7f9db4105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:17.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.360+0000 7f9db9fbb640 1 -- 172.21.15.60:0/2126453692 shutdown_connections 2024-02-26T04:57:17.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.360+0000 7f9db9fbb640 1 -- 172.21.15.60:0/2126453692 wait complete. 2024-02-26T04:57:17.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T04:57:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.476+0000 7f1d228b0640 1 -- 172.21.15.60:0/2361839014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d1c100080 msgr2=0x7f1d1c1004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.476+0000 7f1d228b0640 1 --2- 172.21.15.60:0/2361839014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d1c100080 0x7f1d1c1004a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009900 tx=0x7f1d0c02bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 -- 172.21.15.60:0/2361839014 shutdown_connections 2024-02-26T04:57:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 --2- 172.21.15.60:0/2361839014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d1c101c30 0x7f1d1c1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 --2- 172.21.15.60:0/2361839014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d1c101270 0x7f1d1c1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 --2- 172.21.15.60:0/2361839014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d1c100080 0x7f1d1c1004a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 -- 172.21.15.60:0/2361839014 >> 172.21.15.60:0/2361839014 conn(0x7f1d1c0fb610 msgr2=0x7f1d1c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 -- 172.21.15.60:0/2361839014 shutdown_connections 2024-02-26T04:57:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 -- 172.21.15.60:0/2361839014 wait complete. 2024-02-26T04:57:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 Processor -- start 2024-02-26T04:57:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 -- start start 2024-02-26T04:57:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d1c100080 0x7f1d1c10a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d1c101270 0x7f1d1c108f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d1c101c30 0x7f1d1c1094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c109c20 con 0x7f1d1c101c30 2024-02-26T04:57:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c109d90 con 0x7f1d1c101270 2024-02-26T04:57:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.477+0000 7f1d228b0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c1a1270 con 0x7f1d1c100080 2024-02-26T04:57:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d20e4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d1c101c30 0x7f1d1c1094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d20e4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d1c101c30 0x7f1d1c1094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41608/0 (socket says 172.21.15.60:41608) 2024-02-26T04:57:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d20e4d640 1 -- 172.21.15.60:0/3987946589 learned_addr learned my addr 172.21.15.60:0/3987946589 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d1b7fe640 1 --2- 172.21.15.60:0/3987946589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d1c101270 0x7f1d1c108f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d1bfff640 1 --2- 172.21.15.60:0/3987946589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d1c100080 0x7f1d1c10a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d20e4d640 1 -- 172.21.15.60:0/3987946589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d1c100080 msgr2=0x7f1d1c10a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d20e4d640 1 --2- 172.21.15.60:0/3987946589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d1c100080 0x7f1d1c10a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d20e4d640 1 -- 172.21.15.60:0/3987946589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d1c101270 msgr2=0x7f1d1c108f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d20e4d640 1 --2- 172.21.15.60:0/3987946589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d1c101270 0x7f1d1c108f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d20e4d640 1 -- 172.21.15.60:0/3987946589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c009560 con 0x7f1d1c101c30 2024-02-26T04:57:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d20e4d640 1 --2- 172.21.15.60:0/3987946589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d1c101c30 0x7f1d1c1094c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400db20 tx=0x7f1d1400df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d197fa640 1 -- 172.21.15.60:0/3987946589 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1400bdd0 con 0x7f1d1c101c30 2024-02-26T04:57:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d197fa640 1 -- 172.21.15.60:0/3987946589 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d14003e50 con 0x7f1d1c101c30 2024-02-26T04:57:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d228b0640 1 -- 172.21.15.60:0/3987946589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1c1a1520 con 0x7f1d1c101c30 2024-02-26T04:57:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d197fa640 1 -- 172.21.15.60:0/3987946589 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d140106c0 con 0x7f1d1c101c30 2024-02-26T04:57:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.478+0000 7f1d228b0640 1 -- 172.21.15.60:0/3987946589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1c1a19e0 con 0x7f1d1c101c30 2024-02-26T04:57:17.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.479+0000 7f1d228b0640 1 -- 172.21.15.60:0/3987946589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1c1073f0 con 0x7f1d1c101c30 2024-02-26T04:57:17.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.480+0000 7f1d197fa640 1 -- 172.21.15.60:0/3987946589 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f1d14004220 con 0x7f1d1c101c30 2024-02-26T04:57:17.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.481+0000 7f1d197fa640 1 --2- 172.21.15.60:0/3987946589 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1d040a2150 0x7f1d040a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:17.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.481+0000 7f1d1bfff640 1 --2- 172.21.15.60:0/3987946589 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1d040a2150 0x7f1d040a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:17.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.481+0000 7f1d197fa640 1 -- 172.21.15.60:0/3987946589 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f1d140cdd70 con 0x7f1d1c101c30 2024-02-26T04:57:17.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.482+0000 7f1d1bfff640 1 --2- 172.21.15.60:0/3987946589 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1d040a2150 0x7f1d040a4610 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c005ab0 tx=0x7f1d0c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:17.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.486+0000 7f1d197fa640 1 -- 172.21.15.60:0/3987946589 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1d14096880 con 0x7f1d1c101c30 2024-02-26T04:57:17.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.667+0000 7f1d228b0640 1 -- 172.21.15.60:0/3987946589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d1c02ce40 con 0x7f1d1c101c30 2024-02-26T04:57:17.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.667+0000 7f1d197fa640 1 -- 172.21.15.60:0/3987946589 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d14099f40 con 0x7f1d1c101c30 2024-02-26T04:57:17.670 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:57:17.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.669+0000 7f1d228b0640 1 -- 172.21.15.60:0/3987946589 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1d040a2150 msgr2=0x7f1d040a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.669+0000 7f1d228b0640 1 --2- 172.21.15.60:0/3987946589 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1d040a2150 0x7f1d040a4610 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c005ab0 tx=0x7f1d0c005fd0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.669+0000 7f1d228b0640 1 -- 172.21.15.60:0/3987946589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d1c101c30 msgr2=0x7f1d1c1094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.669+0000 7f1d228b0640 1 --2- 172.21.15.60:0/3987946589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d1c101c30 0x7f1d1c1094c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400db20 tx=0x7f1d1400df20 comp rx=0 tx=0).stop 2024-02-26T04:57:17.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.669+0000 7f1d228b0640 1 -- 172.21.15.60:0/3987946589 shutdown_connections 2024-02-26T04:57:17.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.669+0000 7f1d228b0640 1 --2- 172.21.15.60:0/3987946589 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1d040a2150 0x7f1d040a4610 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.669+0000 7f1d228b0640 1 --2- 172.21.15.60:0/3987946589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d1c101c30 0x7f1d1c1094c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.669+0000 7f1d228b0640 1 --2- 172.21.15.60:0/3987946589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d1c101270 0x7f1d1c108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.669+0000 7f1d228b0640 1 --2- 172.21.15.60:0/3987946589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d1c100080 0x7f1d1c10a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.669+0000 7f1d228b0640 1 -- 172.21.15.60:0/3987946589 >> 172.21.15.60:0/3987946589 conn(0x7f1d1c0fb610 msgr2=0x7f1d1c1045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:17.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.669+0000 7f1d228b0640 1 -- 172.21.15.60:0/3987946589 shutdown_connections 2024-02-26T04:57:17.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.669+0000 7f1d228b0640 1 -- 172.21.15.60:0/3987946589 wait complete. 2024-02-26T04:57:17.683 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T04:57:17.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.786+0000 7f0024e88640 1 -- 172.21.15.60:0/2517291450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0020101c50 msgr2=0x7f00201084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.786+0000 7f0024e88640 1 --2- 172.21.15.60:0/2517291450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0020101c50 0x7f00201084e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0010009ca0 tx=0x7f001002c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.786+0000 7f0024e88640 1 -- 172.21.15.60:0/2517291450 shutdown_connections 2024-02-26T04:57:17.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.786+0000 7f0024e88640 1 --2- 172.21.15.60:0/2517291450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0020101c50 0x7f00201084e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.786+0000 7f0024e88640 1 --2- 172.21.15.60:0/2517291450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0020101290 0x7f0020101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.786+0000 7f0024e88640 1 --2- 172.21.15.60:0/2517291450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00201000a0 0x7f00201004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.786+0000 7f0024e88640 1 -- 172.21.15.60:0/2517291450 >> 172.21.15.60:0/2517291450 conn(0x7f00200fb630 msgr2=0x7f00200fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:17.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.786+0000 7f0024e88640 1 -- 172.21.15.60:0/2517291450 shutdown_connections 2024-02-26T04:57:17.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.786+0000 7f0024e88640 1 -- 172.21.15.60:0/2517291450 wait complete. 2024-02-26T04:57:17.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f0024e88640 1 Processor -- start 2024-02-26T04:57:17.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f0024e88640 1 -- start start 2024-02-26T04:57:17.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f0024e88640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00201000a0 0x7f00201971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:17.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f0024e88640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0020101290 0x7f0020197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:17.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f0024e88640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0020101c50 0x7f002019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:17.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f0024e88640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002019ecc0 con 0x7f0020101c50 2024-02-26T04:57:17.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f0024e88640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002019ee30 con 0x7f0020101290 2024-02-26T04:57:17.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f0024e88640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002019eff0 con 0x7f00201000a0 2024-02-26T04:57:17.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f001ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0020101c50 0x7f002019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:17.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f001ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0020101c50 0x7f002019e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41620/0 (socket says 172.21.15.60:41620) 2024-02-26T04:57:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f001ed9d640 1 -- 172.21.15.60:0/3393022346 learned_addr learned my addr 172.21.15.60:0/3393022346 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f001e59c640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00201000a0 0x7f00201971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f001dd9b640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0020101290 0x7f0020197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f001ed9d640 1 -- 172.21.15.60:0/3393022346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00201000a0 msgr2=0x7f00201971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f001ed9d640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00201000a0 0x7f00201971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f001ed9d640 1 -- 172.21.15.60:0/3393022346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0020101290 msgr2=0x7f0020197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f001ed9d640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0020101290 0x7f0020197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f001ed9d640 1 -- 172.21.15.60:0/3393022346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0010009900 con 0x7f0020101c50 2024-02-26T04:57:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.787+0000 7f001dd9b640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0020101290 0x7f0020197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.788+0000 7f001ed9d640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0020101c50 0x7f002019e780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0010037040 tx=0x7f0010034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.788+0000 7f00177fe640 1 -- 172.21.15.60:0/3393022346 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010032eb0 con 0x7f0020101c50 2024-02-26T04:57:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.788+0000 7f00177fe640 1 -- 172.21.15.60:0/3393022346 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0010003e50 con 0x7f0020101c50 2024-02-26T04:57:17.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.788+0000 7f00177fe640 1 -- 172.21.15.60:0/3393022346 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010040450 con 0x7f0020101c50 2024-02-26T04:57:17.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.788+0000 7f0024e88640 1 -- 172.21.15.60:0/3393022346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002019f270 con 0x7f0020101c50 2024-02-26T04:57:17.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.788+0000 7f0024e88640 1 -- 172.21.15.60:0/3393022346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002019f770 con 0x7f0020101c50 2024-02-26T04:57:17.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.789+0000 7f0024e88640 1 -- 172.21.15.60:0/3393022346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00201073c0 con 0x7f0020101c50 2024-02-26T04:57:17.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.790+0000 7f00177fe640 1 -- 172.21.15.60:0/3393022346 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f001000a040 con 0x7f0020101c50 2024-02-26T04:57:17.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.790+0000 7f00177fe640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f000c0a21d0 0x7f000c0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:17.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.790+0000 7f00177fe640 1 -- 172.21.15.60:0/3393022346 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f00100f0c20 con 0x7f0020101c50 2024-02-26T04:57:17.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.790+0000 7f001e59c640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f000c0a21d0 0x7f000c0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:17.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.791+0000 7f001e59c640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f000c0a21d0 0x7f000c0a4690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0018005fd0 tx=0x7f001800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.795+0000 7f00177fe640 1 -- 172.21.15.60:0/3393022346 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f00100b9840 con 0x7f0020101c50 2024-02-26T04:57:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.977+0000 7f0024e88640 1 -- 172.21.15.60:0/3393022346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f002002ce40 con 0x7f0020101c50 2024-02-26T04:57:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.978+0000 7f00177fe640 1 -- 172.21.15.60:0/3393022346 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f00100bcf00 con 0x7f0020101c50 2024-02-26T04:57:17.980 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T04:57:17.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.980+0000 7f0024e88640 1 -- 172.21.15.60:0/3393022346 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f000c0a21d0 msgr2=0x7f000c0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.980+0000 7f0024e88640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f000c0a21d0 0x7f000c0a4690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0018005fd0 tx=0x7f001800b390 comp rx=0 tx=0).stop 2024-02-26T04:57:17.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.980+0000 7f0024e88640 1 -- 172.21.15.60:0/3393022346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0020101c50 msgr2=0x7f002019e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:17.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.980+0000 7f0024e88640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0020101c50 0x7f002019e780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0010037040 tx=0x7f0010034d40 comp rx=0 tx=0).stop 2024-02-26T04:57:17.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.980+0000 7f0024e88640 1 -- 172.21.15.60:0/3393022346 shutdown_connections 2024-02-26T04:57:17.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.980+0000 7f0024e88640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f000c0a21d0 0x7f000c0a4690 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.980+0000 7f0024e88640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0020101c50 0x7f002019e780 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.980+0000 7f0024e88640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0020101290 0x7f0020197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.980+0000 7f0024e88640 1 --2- 172.21.15.60:0/3393022346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00201000a0 0x7f00201971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:17.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.980+0000 7f0024e88640 1 -- 172.21.15.60:0/3393022346 >> 172.21.15.60:0/3393022346 conn(0x7f00200fb630 msgr2=0x7f0020104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:17.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.980+0000 7f0024e88640 1 -- 172.21.15.60:0/3393022346 shutdown_connections 2024-02-26T04:57:17.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:17.980+0000 7f0024e88640 1 -- 172.21.15.60:0/3393022346 wait complete. 2024-02-26T04:57:17.993 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T04:57:18.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.097+0000 7f474fe08640 1 -- 172.21.15.60:0/2947760364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4748101c30 msgr2=0x7f47481084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.097+0000 7f474fe08640 1 --2- 172.21.15.60:0/2947760364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4748101c30 0x7f47481084c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4740009ca0 tx=0x7f474002c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.097+0000 7f474fe08640 1 -- 172.21.15.60:0/2947760364 shutdown_connections 2024-02-26T04:57:18.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.097+0000 7f474fe08640 1 --2- 172.21.15.60:0/2947760364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4748101c30 0x7f47481084c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474fe08640 1 --2- 172.21.15.60:0/2947760364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4748101270 0x7f47481016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474fe08640 1 --2- 172.21.15.60:0/2947760364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4748100080 0x7f47481004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474fe08640 1 -- 172.21.15.60:0/2947760364 >> 172.21.15.60:0/2947760364 conn(0x7f47480fb610 msgr2=0x7f47480fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474fe08640 1 -- 172.21.15.60:0/2947760364 shutdown_connections 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474fe08640 1 -- 172.21.15.60:0/2947760364 wait complete. 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474fe08640 1 Processor -- start 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474fe08640 1 -- start start 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474fe08640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4748100080 0x7f4748197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474fe08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4748101270 0x7f47481977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474fe08640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4748101c30 0x7f474819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474fe08640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474819ed50 con 0x7f4748101270 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474fe08640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474819eec0 con 0x7f4748101c30 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474fe08640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474819f080 con 0x7f4748100080 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474d3a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4748101270 0x7f47481977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474d3a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4748101270 0x7f47481977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41634/0 (socket says 172.21.15.60:41634) 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.098+0000 7f474d3a3640 1 -- 172.21.15.60:0/2801888861 learned_addr learned my addr 172.21.15.60:0/2801888861 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.099+0000 7f474dba4640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4748100080 0x7f4748197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.099+0000 7f474e3a5640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4748101c30 0x7f474819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.099+0000 7f474d3a3640 1 -- 172.21.15.60:0/2801888861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4748100080 msgr2=0x7f4748197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.099+0000 7f474d3a3640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4748100080 0x7f4748197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.099+0000 7f474d3a3640 1 -- 172.21.15.60:0/2801888861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4748101c30 msgr2=0x7f474819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.099+0000 7f474d3a3640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4748101c30 0x7f474819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.099+0000 7f474d3a3640 1 -- 172.21.15.60:0/2801888861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4740009900 con 0x7f4748101270 2024-02-26T04:57:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.099+0000 7f474e3a5640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4748101c30 0x7f474819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.099+0000 7f474d3a3640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4748101270 0x7f47481977a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f473800d770 tx=0x7f473800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.099+0000 7f473effd640 1 -- 172.21.15.60:0/2801888861 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738009c30 con 0x7f4748101270 2024-02-26T04:57:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.099+0000 7f473effd640 1 -- 172.21.15.60:0/2801888861 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f473800de80 con 0x7f4748101270 2024-02-26T04:57:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.099+0000 7f474fe08640 1 -- 172.21.15.60:0/2801888861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f474819f360 con 0x7f4748101270 2024-02-26T04:57:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.099+0000 7f473effd640 1 -- 172.21.15.60:0/2801888861 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738004e80 con 0x7f4748101270 2024-02-26T04:57:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.099+0000 7f474fe08640 1 -- 172.21.15.60:0/2801888861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f474819f870 con 0x7f4748101270 2024-02-26T04:57:18.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.100+0000 7f474fe08640 1 -- 172.21.15.60:0/2801888861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f474810ba90 con 0x7f4748101270 2024-02-26T04:57:18.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.101+0000 7f473effd640 1 -- 172.21.15.60:0/2801888861 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f4738005020 con 0x7f4748101270 2024-02-26T04:57:18.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.102+0000 7f473effd640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f47340a21d0 0x7f47340a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:18.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.102+0000 7f474dba4640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f47340a21d0 0x7f47340a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:18.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.102+0000 7f473effd640 1 -- 172.21.15.60:0/2801888861 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f47380cdba0 con 0x7f4748101270 2024-02-26T04:57:18.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.102+0000 7f474dba4640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f47340a21d0 0x7f47340a4690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f47481010d0 tx=0x7f474400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:18.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.107+0000 7f473effd640 1 -- 172.21.15.60:0/2801888861 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f47380966b0 con 0x7f4748101270 2024-02-26T04:57:18.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.287+0000 7f474fe08640 1 -- 172.21.15.60:0/2801888861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f474819fd30 con 0x7f4748101270 2024-02-26T04:57:18.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.287+0000 7f473effd640 1 -- 172.21.15.60:0/2801888861 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4738017020 con 0x7f4748101270 2024-02-26T04:57:18.290 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:57:18.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.289+0000 7f474fe08640 1 -- 172.21.15.60:0/2801888861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f47340a21d0 msgr2=0x7f47340a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.289+0000 7f474fe08640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f47340a21d0 0x7f47340a4690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f47481010d0 tx=0x7f474400b390 comp rx=0 tx=0).stop 2024-02-26T04:57:18.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.289+0000 7f474fe08640 1 -- 172.21.15.60:0/2801888861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4748101270 msgr2=0x7f47481977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.289+0000 7f474fe08640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4748101270 0x7f47481977a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f473800d770 tx=0x7f473800dc30 comp rx=0 tx=0).stop 2024-02-26T04:57:18.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.289+0000 7f474fe08640 1 -- 172.21.15.60:0/2801888861 shutdown_connections 2024-02-26T04:57:18.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.289+0000 7f474fe08640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f47340a21d0 0x7f47340a4690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.289+0000 7f474fe08640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4748101c30 0x7f474819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.289+0000 7f474fe08640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4748101270 0x7f47481977a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.289+0000 7f474fe08640 1 --2- 172.21.15.60:0/2801888861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4748100080 0x7f4748197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.289+0000 7f474fe08640 1 -- 172.21.15.60:0/2801888861 >> 172.21.15.60:0/2801888861 conn(0x7f47480fb610 msgr2=0x7f47480fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:18.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.289+0000 7f474fe08640 1 -- 172.21.15.60:0/2801888861 shutdown_connections 2024-02-26T04:57:18.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.289+0000 7f474fe08640 1 -- 172.21.15.60:0/2801888861 wait complete. 2024-02-26T04:57:18.302 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:18 smithi060 ceph-mon[32382]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:57:18.302 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:18 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/665476788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:18.302 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:18 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2126453692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:18.302 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:18 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3987946589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:18.302 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:18 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3393022346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:18.304 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:57:18.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:18 smithi156 ceph-mon[35805]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:57:18.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:18 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/665476788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:18.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:18 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2126453692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:18.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:18 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3987946589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:18.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:18 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3393022346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:18.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.427+0000 7f6b212de640 1 -- 172.21.15.60:0/941732191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b1c106f00 msgr2=0x7f6b1c1092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.427+0000 7f6b212de640 1 --2- 172.21.15.60:0/941732191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b1c106f00 0x7f6b1c1092f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009ca0 tx=0x7f6b0c02c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.427+0000 7f6b212de640 1 -- 172.21.15.60:0/941732191 shutdown_connections 2024-02-26T04:57:18.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.427+0000 7f6b212de640 1 --2- 172.21.15.60:0/941732191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b1c106f00 0x7f6b1c1092f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.427+0000 7f6b212de640 1 --2- 172.21.15.60:0/941732191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1c1045d0 0x7f6b1c1069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.427+0000 7f6b212de640 1 --2- 172.21.15.60:0/941732191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b1c101ca0 0x7f6b1c104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.427+0000 7f6b212de640 1 -- 172.21.15.60:0/941732191 >> 172.21.15.60:0/941732191 conn(0x7f6b1c0fb630 msgr2=0x7f6b1c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.427+0000 7f6b212de640 1 -- 172.21.15.60:0/941732191 shutdown_connections 2024-02-26T04:57:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.427+0000 7f6b212de640 1 -- 172.21.15.60:0/941732191 wait complete. 2024-02-26T04:57:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.427+0000 7f6b212de640 1 Processor -- start 2024-02-26T04:57:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b212de640 1 -- start start 2024-02-26T04:57:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b212de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b1c101ca0 0x7f6b1c197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b212de640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b1c1045d0 0x7f6b1c1976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b212de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1c106f00 0x7f6b1c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b212de640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c19ec70 con 0x7f6b1c101ca0 2024-02-26T04:57:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b212de640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c19ede0 con 0x7f6b1c1045d0 2024-02-26T04:57:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b212de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c19efa0 con 0x7f6b1c106f00 2024-02-26T04:57:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b1affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b1c101ca0 0x7f6b1c197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b1affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b1c101ca0 0x7f6b1c197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41660/0 (socket says 172.21.15.60:41660) 2024-02-26T04:57:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b1affd640 1 -- 172.21.15.60:0/146796918 learned_addr learned my addr 172.21.15.60:0/146796918 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b1b7fe640 1 --2- 172.21.15.60:0/146796918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1c106f00 0x7f6b1c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b1a7fc640 1 --2- 172.21.15.60:0/146796918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b1c1045d0 0x7f6b1c1976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b1affd640 1 -- 172.21.15.60:0/146796918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1c106f00 msgr2=0x7f6b1c19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b1affd640 1 --2- 172.21.15.60:0/146796918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1c106f00 0x7f6b1c19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b1affd640 1 -- 172.21.15.60:0/146796918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b1c1045d0 msgr2=0x7f6b1c1976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b1affd640 1 --2- 172.21.15.60:0/146796918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b1c1045d0 0x7f6b1c1976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b1affd640 1 -- 172.21.15.60:0/146796918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c009900 con 0x7f6b1c101ca0 2024-02-26T04:57:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.428+0000 7f6b1affd640 1 --2- 172.21.15.60:0/146796918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b1c101ca0 0x7f6b1c197180 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400d950 tx=0x7f6b1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.429+0000 7f6b03fff640 1 -- 172.21.15.60:0/146796918 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14009a30 con 0x7f6b1c101ca0 2024-02-26T04:57:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.429+0000 7f6b03fff640 1 -- 172.21.15.60:0/146796918 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b14009b90 con 0x7f6b1c101ca0 2024-02-26T04:57:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.429+0000 7f6b212de640 1 -- 172.21.15.60:0/146796918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c19f280 con 0x7f6b1c101ca0 2024-02-26T04:57:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.429+0000 7f6b03fff640 1 -- 172.21.15.60:0/146796918 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14004a00 con 0x7f6b1c101ca0 2024-02-26T04:57:18.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.429+0000 7f6b212de640 1 -- 172.21.15.60:0/146796918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c19f790 con 0x7f6b1c101ca0 2024-02-26T04:57:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.430+0000 7f6b212de640 1 -- 172.21.15.60:0/146796918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1c0fd120 con 0x7f6b1c101ca0 2024-02-26T04:57:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.431+0000 7f6b03fff640 1 -- 172.21.15.60:0/146796918 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f6b14009d00 con 0x7f6b1c101ca0 2024-02-26T04:57:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.431+0000 7f6b03fff640 1 --2- 172.21.15.60:0/146796918 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6b040a2150 0x7f6b040a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.431+0000 7f6b1a7fc640 1 --2- 172.21.15.60:0/146796918 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6b040a2150 0x7f6b040a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.431+0000 7f6b03fff640 1 -- 172.21.15.60:0/146796918 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f6b140cdd00 con 0x7f6b1c101ca0 2024-02-26T04:57:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.432+0000 7f6b1a7fc640 1 --2- 172.21.15.60:0/146796918 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6b040a2150 0x7f6b040a4610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c198690 tx=0x7f6b10005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:18.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.436+0000 7f6b03fff640 1 -- 172.21.15.60:0/146796918 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b14096810 con 0x7f6b1c101ca0 2024-02-26T04:57:18.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:18 smithi191 ceph-mon[36352]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:57:18.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:18 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/665476788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:18.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:18 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2126453692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:18.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:18 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3987946589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:18.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:18 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3393022346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.658+0000 7f6b212de640 1 -- 172.21.15.60:0/146796918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b1c10cbd0 con 0x7f6b1c101ca0 2024-02-26T04:57:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.659+0000 7f6b03fff640 1 -- 172.21.15.60:0/146796918 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11347 (secure 0 0 0) 0x7f6b14099ed0 con 0x7f6b1c101ca0 2024-02-26T04:57:18.662 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:18.662 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:55:54.140330+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24476,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T04:57:18.662 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-02-26T04:57:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.661+0000 7f6b212de640 1 -- 172.21.15.60:0/146796918 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6b040a2150 msgr2=0x7f6b040a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.661+0000 7f6b212de640 1 --2- 172.21.15.60:0/146796918 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6b040a2150 0x7f6b040a4610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c198690 tx=0x7f6b10005c50 comp rx=0 tx=0).stop 2024-02-26T04:57:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.661+0000 7f6b212de640 1 -- 172.21.15.60:0/146796918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b1c101ca0 msgr2=0x7f6b1c197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.661+0000 7f6b212de640 1 --2- 172.21.15.60:0/146796918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b1c101ca0 0x7f6b1c197180 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400d950 tx=0x7f6b1400de10 comp rx=0 tx=0).stop 2024-02-26T04:57:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.661+0000 7f6b212de640 1 -- 172.21.15.60:0/146796918 shutdown_connections 2024-02-26T04:57:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.661+0000 7f6b212de640 1 --2- 172.21.15.60:0/146796918 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6b040a2150 0x7f6b040a4610 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.661+0000 7f6b212de640 1 --2- 172.21.15.60:0/146796918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1c106f00 0x7f6b1c19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.661+0000 7f6b212de640 1 --2- 172.21.15.60:0/146796918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b1c1045d0 0x7f6b1c1976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.661+0000 7f6b212de640 1 --2- 172.21.15.60:0/146796918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b1c101ca0 0x7f6b1c197180 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.661+0000 7f6b212de640 1 -- 172.21.15.60:0/146796918 >> 172.21.15.60:0/146796918 conn(0x7f6b1c0fb630 msgr2=0x7f6b1c1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.662+0000 7f6b212de640 1 -- 172.21.15.60:0/146796918 shutdown_connections 2024-02-26T04:57:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.662+0000 7f6b212de640 1 -- 172.21.15.60:0/146796918 wait complete. 2024-02-26T04:57:18.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:57:18.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.775+0000 7f695351e640 1 -- 172.21.15.60:0/3592398998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c1016c0 msgr2=0x7f694c101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.775+0000 7f695351e640 1 --2- 172.21.15.60:0/3592398998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c1016c0 0x7f694c101ae0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6948009900 tx=0x7f694802bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:18.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.775+0000 7f695351e640 1 -- 172.21.15.60:0/3592398998 shutdown_connections 2024-02-26T04:57:18.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.775+0000 7f695351e640 1 --2- 172.21.15.60:0/3592398998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f694c100cd0 0x7f694c108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.775+0000 7f695351e640 1 --2- 172.21.15.60:0/3592398998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f694c100310 0x7f694c100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.775+0000 7f695351e640 1 --2- 172.21.15.60:0/3592398998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c1016c0 0x7f694c101ae0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.775+0000 7f695351e640 1 -- 172.21.15.60:0/3592398998 >> 172.21.15.60:0/3592398998 conn(0x7f694c0fada0 msgr2=0x7f694c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:18.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.775+0000 7f695351e640 1 -- 172.21.15.60:0/3592398998 shutdown_connections 2024-02-26T04:57:18.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.775+0000 7f695351e640 1 -- 172.21.15.60:0/3592398998 wait complete. 2024-02-26T04:57:18.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f695351e640 1 Processor -- start 2024-02-26T04:57:18.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f695351e640 1 -- start start 2024-02-26T04:57:18.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f695351e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c100310 0x7f694c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:18.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f695351e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f694c100cd0 0x7f694c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f695351e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f694c1016c0 0x7f694c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f695351e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c19ed30 con 0x7f694c100310 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f695351e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c19eea0 con 0x7f694c100cd0 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f695351e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c19f060 con 0x7f694c1016c0 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f69512ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c100310 0x7f694c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f69512ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c100310 0x7f694c197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41682/0 (socket says 172.21.15.60:41682) 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f69512ba640 1 -- 172.21.15.60:0/1773164114 learned_addr learned my addr 172.21.15.60:0/1773164114 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f6951abb640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f694c1016c0 0x7f694c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f6950ab9640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f694c100cd0 0x7f694c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f69512ba640 1 -- 172.21.15.60:0/1773164114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f694c1016c0 msgr2=0x7f694c19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f69512ba640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f694c1016c0 0x7f694c19e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f69512ba640 1 -- 172.21.15.60:0/1773164114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f694c100cd0 msgr2=0x7f694c197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f69512ba640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f694c100cd0 0x7f694c197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f69512ba640 1 -- 172.21.15.60:0/1773164114 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6948009560 con 0x7f694c100310 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f6950ab9640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f694c100cd0 0x7f694c197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.776+0000 7f69512ba640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c100310 0x7f694c197240 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6948036040 tx=0x7f69480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:18.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.777+0000 7f69427fc640 1 -- 172.21.15.60:0/1773164114 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6948004510 con 0x7f694c100310 2024-02-26T04:57:18.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.777+0000 7f69427fc640 1 -- 172.21.15.60:0/1773164114 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69480054e0 con 0x7f694c100310 2024-02-26T04:57:18.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.777+0000 7f695351e640 1 -- 172.21.15.60:0/1773164114 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694c19f2e0 con 0x7f694c100310 2024-02-26T04:57:18.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.777+0000 7f69427fc640 1 -- 172.21.15.60:0/1773164114 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694803e450 con 0x7f694c100310 2024-02-26T04:57:18.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.777+0000 7f695351e640 1 -- 172.21.15.60:0/1773164114 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694c19f870 con 0x7f694c100310 2024-02-26T04:57:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.778+0000 7f695351e640 1 -- 172.21.15.60:0/1773164114 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694c1073f0 con 0x7f694c100310 2024-02-26T04:57:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.779+0000 7f69427fc640 1 -- 172.21.15.60:0/1773164114 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f6948003710 con 0x7f694c100310 2024-02-26T04:57:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.780+0000 7f69427fc640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f69380a2150 0x7f69380a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.780+0000 7f6950ab9640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f69380a2150 0x7f69380a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.780+0000 7f69427fc640 1 -- 172.21.15.60:0/1773164114 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f69480f0bf0 con 0x7f694c100310 2024-02-26T04:57:18.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.780+0000 7f6950ab9640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f69380a2150 0x7f69380a4610 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f694c198750 tx=0x7f693c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:18.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.783+0000 7f69427fc640 1 -- 172.21.15.60:0/1773164114 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f69480b9810 con 0x7f694c100310 2024-02-26T04:57:18.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.966+0000 7f695351e640 1 -- 172.21.15.60:0/1773164114 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f694c02cdd0 con 0x7f694c100310 2024-02-26T04:57:18.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.967+0000 7f69427fc640 1 -- 172.21.15.60:0/1773164114 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24596 (secure 0 0 0) 0x7f69480bced0 con 0x7f694c100310 2024-02-26T04:57:18.970 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:18.970 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":91,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:56:20.193583+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:57:18.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.970+0000 7f695351e640 1 -- 172.21.15.60:0/1773164114 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f69380a2150 msgr2=0x7f69380a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.970+0000 7f695351e640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f69380a2150 0x7f69380a4610 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f694c198750 tx=0x7f693c005db0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.970+0000 7f695351e640 1 -- 172.21.15.60:0/1773164114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c100310 msgr2=0x7f694c197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:18.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.970+0000 7f695351e640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c100310 0x7f694c197240 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6948036040 tx=0x7f69480042f0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.970+0000 7f695351e640 1 -- 172.21.15.60:0/1773164114 shutdown_connections 2024-02-26T04:57:18.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.970+0000 7f695351e640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f69380a2150 0x7f69380a4610 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.970+0000 7f695351e640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f694c1016c0 0x7f694c19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.970+0000 7f695351e640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f694c100cd0 0x7f694c197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.970+0000 7f695351e640 1 --2- 172.21.15.60:0/1773164114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c100310 0x7f694c197240 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:18.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.970+0000 7f695351e640 1 -- 172.21.15.60:0/1773164114 >> 172.21.15.60:0/1773164114 conn(0x7f694c0fada0 msgr2=0x7f694c1045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:18.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.970+0000 7f695351e640 1 -- 172.21.15.60:0/1773164114 shutdown_connections 2024-02-26T04:57:18.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:18.970+0000 7f695351e640 1 -- 172.21.15.60:0/1773164114 wait complete. 2024-02-26T04:57:18.983 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T04:50:44.572921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T04:55:25.530720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T04:55:26.741604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T04:55:30.453569+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T04:57:18.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T04:57:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/2331686341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd18100090 msgr2=0x7fcd181004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 --2- 172.21.15.60:0/2331686341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd18100090 0x7fcd181004b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009900 tx=0x7fcd0c02bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/2331686341 shutdown_connections 2024-02-26T04:57:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 --2- 172.21.15.60:0/2331686341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd18101c40 0x7fcd181084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 --2- 172.21.15.60:0/2331686341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd18101280 0x7fcd18101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 --2- 172.21.15.60:0/2331686341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd18100090 0x7fcd181004b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/2331686341 >> 172.21.15.60:0/2331686341 conn(0x7fcd180fb620 msgr2=0x7fcd180fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/2331686341 shutdown_connections 2024-02-26T04:57:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/2331686341 wait complete. 2024-02-26T04:57:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 Processor -- start 2024-02-26T04:57:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 -- start start 2024-02-26T04:57:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd18100090 0x7fcd18104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd18101280 0x7fcd18102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd18101c40 0x7fcd18102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd18103300 con 0x7fcd18101c40 2024-02-26T04:57:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd18103470 con 0x7fcd18100090 2024-02-26T04:57:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.088+0000 7fcd1ec0f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd181a1310 con 0x7fcd18101280 2024-02-26T04:57:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd1d1ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd18101c40 0x7fcd18102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd1d1ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd18101c40 0x7fcd18102ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41704/0 (socket says 172.21.15.60:41704) 2024-02-26T04:57:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd1d1ac640 1 -- 172.21.15.60:0/3516299544 learned_addr learned my addr 172.21.15.60:0/3516299544 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd17fff640 1 --2- 172.21.15.60:0/3516299544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd18101280 0x7fcd18102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd1c9ab640 1 --2- 172.21.15.60:0/3516299544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd18100090 0x7fcd18104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd1d1ac640 1 -- 172.21.15.60:0/3516299544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd18101280 msgr2=0x7fcd18102660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd1d1ac640 1 --2- 172.21.15.60:0/3516299544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd18101280 0x7fcd18102660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd1d1ac640 1 -- 172.21.15.60:0/3516299544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd18100090 msgr2=0x7fcd18104020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd1d1ac640 1 --2- 172.21.15.60:0/3516299544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd18100090 0x7fcd18104020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd1d1ac640 1 -- 172.21.15.60:0/3516299544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c009560 con 0x7fcd18101c40 2024-02-26T04:57:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd1d1ac640 1 --2- 172.21.15.60:0/3516299544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd18101c40 0x7fcd18102ba0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000ead0 tx=0x7fcd1000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd15ffb640 1 -- 172.21.15.60:0/3516299544 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000cde0 con 0x7fcd18101c40 2024-02-26T04:57:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd15ffb640 1 -- 172.21.15.60:0/3516299544 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd10005020 con 0x7fcd18101c40 2024-02-26T04:57:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/3516299544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd181a15c0 con 0x7fcd18101c40 2024-02-26T04:57:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd15ffb640 1 -- 172.21.15.60:0/3516299544 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd100116b0 con 0x7fcd18101c40 2024-02-26T04:57:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.089+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/3516299544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd181a1a80 con 0x7fcd18101c40 2024-02-26T04:57:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.090+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/3516299544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1810baa0 con 0x7fcd18101c40 2024-02-26T04:57:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.091+0000 7fcd15ffb640 1 -- 172.21.15.60:0/3516299544 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fcd10005520 con 0x7fcd18101c40 2024-02-26T04:57:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.092+0000 7fcd15ffb640 1 --2- 172.21.15.60:0/3516299544 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcd080a2150 0x7fcd080a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.092+0000 7fcd1c9ab640 1 --2- 172.21.15.60:0/3516299544 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcd080a2150 0x7fcd080a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:19.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.092+0000 7fcd15ffb640 1 -- 172.21.15.60:0/3516299544 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fcd100ce3b0 con 0x7fcd18101c40 2024-02-26T04:57:19.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.092+0000 7fcd1c9ab640 1 --2- 172.21.15.60:0/3516299544 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcd080a2150 0x7fcd080a4610 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c005ab0 tx=0x7fcd0c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.096+0000 7fcd15ffb640 1 -- 172.21.15.60:0/3516299544 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd10096f10 con 0x7fcd18101c40 2024-02-26T04:57:19.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.287+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/3516299544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd181a1ed0 con 0x7fcd18101c40 2024-02-26T04:57:19.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.288+0000 7fcd15ffb640 1 -- 172.21.15.60:0/3516299544 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd1001f0a0 con 0x7fcd18101c40 2024-02-26T04:57:19.290 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T04:57:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.290+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/3516299544 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcd080a2150 msgr2=0x7fcd080a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.290+0000 7fcd1ec0f640 1 --2- 172.21.15.60:0/3516299544 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcd080a2150 0x7fcd080a4610 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c005ab0 tx=0x7fcd0c005fd0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.290+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/3516299544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd18101c40 msgr2=0x7fcd18102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.290+0000 7fcd1ec0f640 1 --2- 172.21.15.60:0/3516299544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd18101c40 0x7fcd18102ba0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000ead0 tx=0x7fcd1000ef90 comp rx=0 tx=0).stop 2024-02-26T04:57:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.290+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/3516299544 shutdown_connections 2024-02-26T04:57:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.290+0000 7fcd1ec0f640 1 --2- 172.21.15.60:0/3516299544 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcd080a2150 0x7fcd080a4610 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.290+0000 7fcd1ec0f640 1 --2- 172.21.15.60:0/3516299544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd18101c40 0x7fcd18102ba0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.290+0000 7fcd1ec0f640 1 --2- 172.21.15.60:0/3516299544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd18101280 0x7fcd18102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.290+0000 7fcd1ec0f640 1 --2- 172.21.15.60:0/3516299544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd18100090 0x7fcd18104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.290+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/3516299544 >> 172.21.15.60:0/3516299544 conn(0x7fcd180fb620 msgr2=0x7fcd180fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.290+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/3516299544 shutdown_connections 2024-02-26T04:57:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.290+0000 7fcd1ec0f640 1 -- 172.21.15.60:0/3516299544 wait complete. 2024-02-26T04:57:19.302 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:19 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2801888861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:19.302 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:19 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/146796918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:19.302 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:19 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1773164114' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:19.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T04:57:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.407+0000 7f4882f1e640 1 -- 172.21.15.60:0/1792874092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c101c20 msgr2=0x7f487c104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.407+0000 7f4882f1e640 1 --2- 172.21.15.60:0/1792874092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c101c20 0x7f487c104010 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4874009ca0 tx=0x7f487402c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.407+0000 7f4882f1e640 1 -- 172.21.15.60:0/1792874092 shutdown_connections 2024-02-26T04:57:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.407+0000 7f4882f1e640 1 --2- 172.21.15.60:0/1792874092 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f487c106e80 0x7f487c1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.407+0000 7f4882f1e640 1 --2- 172.21.15.60:0/1792874092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487c104550 0x7f487c106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.407+0000 7f4882f1e640 1 --2- 172.21.15.60:0/1792874092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c101c20 0x7f487c104010 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.407+0000 7f4882f1e640 1 -- 172.21.15.60:0/1792874092 >> 172.21.15.60:0/1792874092 conn(0x7f487c0fb630 msgr2=0x7f487c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.407+0000 7f4882f1e640 1 -- 172.21.15.60:0/1792874092 shutdown_connections 2024-02-26T04:57:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.407+0000 7f4882f1e640 1 -- 172.21.15.60:0/1792874092 wait complete. 2024-02-26T04:57:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.407+0000 7f4882f1e640 1 Processor -- start 2024-02-26T04:57:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.407+0000 7f4882f1e640 1 -- start start 2024-02-26T04:57:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f4882f1e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487c101c20 0x7f487c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f4882f1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c104550 0x7f487c197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f4882f1e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f487c106e80 0x7f487c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f4882f1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c19ed20 con 0x7f487c104550 2024-02-26T04:57:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f4882f1e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c19ee90 con 0x7f487c101c20 2024-02-26T04:57:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f4882f1e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c19f050 con 0x7f487c106e80 2024-02-26T04:57:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f487bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c104550 0x7f487c197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f487bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c104550 0x7f487c197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41720/0 (socket says 172.21.15.60:41720) 2024-02-26T04:57:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f487bfff640 1 -- 172.21.15.60:0/1302895116 learned_addr learned my addr 172.21.15.60:0/1302895116 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f48814bb640 1 --2- 172.21.15.60:0/1302895116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f487c106e80 0x7f487c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f4880cba640 1 --2- 172.21.15.60:0/1302895116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487c101c20 0x7f487c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f487bfff640 1 -- 172.21.15.60:0/1302895116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f487c106e80 msgr2=0x7f487c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f487bfff640 1 --2- 172.21.15.60:0/1302895116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f487c106e80 0x7f487c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f487bfff640 1 -- 172.21.15.60:0/1302895116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487c101c20 msgr2=0x7f487c197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f487bfff640 1 --2- 172.21.15.60:0/1302895116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487c101c20 0x7f487c197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f487bfff640 1 -- 172.21.15.60:0/1302895116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4874009900 con 0x7f487c104550 2024-02-26T04:57:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f487bfff640 1 --2- 172.21.15.60:0/1302895116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c104550 0x7f487c197770 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f486400c970 tx=0x7f486400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f4879ffb640 1 -- 172.21.15.60:0/1302895116 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864004440 con 0x7f487c104550 2024-02-26T04:57:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f4879ffb640 1 -- 172.21.15.60:0/1302895116 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4864007d40 con 0x7f487c104550 2024-02-26T04:57:19.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f4882f1e640 1 -- 172.21.15.60:0/1302895116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487c19f330 con 0x7f487c104550 2024-02-26T04:57:19.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f4879ffb640 1 -- 172.21.15.60:0/1302895116 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486400f450 con 0x7f487c104550 2024-02-26T04:57:19.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.408+0000 7f4882f1e640 1 -- 172.21.15.60:0/1302895116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487c19f840 con 0x7f487c104550 2024-02-26T04:57:19.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.409+0000 7f4882f1e640 1 -- 172.21.15.60:0/1302895116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487c0fd120 con 0x7f487c104550 2024-02-26T04:57:19.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.410+0000 7f4879ffb640 1 -- 172.21.15.60:0/1302895116 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f4864007860 con 0x7f487c104550 2024-02-26T04:57:19.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.411+0000 7f4879ffb640 1 --2- 172.21.15.60:0/1302895116 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48680a2150 0x7f48680a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:19.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.411+0000 7f4880cba640 1 --2- 172.21.15.60:0/1302895116 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48680a2150 0x7f48680a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:19.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.411+0000 7f4879ffb640 1 -- 172.21.15.60:0/1302895116 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f48640ce300 con 0x7f487c104550 2024-02-26T04:57:19.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.411+0000 7f4880cba640 1 --2- 172.21.15.60:0/1302895116 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48680a2150 0x7f48680a4610 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4874006010 tx=0x7f48740326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.415+0000 7f4879ffb640 1 -- 172.21.15.60:0/1302895116 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4864096eb0 con 0x7f487c104550 2024-02-26T04:57:19.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:19 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2801888861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:19.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:19 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/146796918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:19.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:19 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1773164114' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:19.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:19 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2801888861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:19.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:19 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/146796918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:19.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:19 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1773164114' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:19.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.598+0000 7f4882f1e640 1 -- 172.21.15.60:0/1302895116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f487c10cdc0 con 0x7f487c104550 2024-02-26T04:57:19.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.598+0000 7f4879ffb640 1 -- 172.21.15.60:0/1302895116 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f486409a570 con 0x7f487c104550 2024-02-26T04:57:19.601 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:57:19.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.600+0000 7f4882f1e640 1 -- 172.21.15.60:0/1302895116 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48680a2150 msgr2=0x7f48680a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.600+0000 7f4882f1e640 1 --2- 172.21.15.60:0/1302895116 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48680a2150 0x7f48680a4610 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4874006010 tx=0x7f48740326b0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.600+0000 7f4882f1e640 1 -- 172.21.15.60:0/1302895116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c104550 msgr2=0x7f487c197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.600+0000 7f4882f1e640 1 --2- 172.21.15.60:0/1302895116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c104550 0x7f487c197770 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f486400c970 tx=0x7f486400ce30 comp rx=0 tx=0).stop 2024-02-26T04:57:19.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.600+0000 7f4882f1e640 1 -- 172.21.15.60:0/1302895116 shutdown_connections 2024-02-26T04:57:19.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.600+0000 7f4882f1e640 1 --2- 172.21.15.60:0/1302895116 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48680a2150 0x7f48680a4610 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.600+0000 7f4882f1e640 1 --2- 172.21.15.60:0/1302895116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f487c106e80 0x7f487c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.600+0000 7f4882f1e640 1 --2- 172.21.15.60:0/1302895116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c104550 0x7f487c197770 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.600+0000 7f4882f1e640 1 --2- 172.21.15.60:0/1302895116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487c101c20 0x7f487c197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.600+0000 7f4882f1e640 1 -- 172.21.15.60:0/1302895116 >> 172.21.15.60:0/1302895116 conn(0x7f487c0fb630 msgr2=0x7f487c102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:19.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.600+0000 7f4882f1e640 1 -- 172.21.15.60:0/1302895116 shutdown_connections 2024-02-26T04:57:19.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.601+0000 7f4882f1e640 1 -- 172.21.15.60:0/1302895116 wait complete. 2024-02-26T04:57:19.614 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T04:57:19.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.717+0000 7f19f7374640 1 -- 172.21.15.60:0/694533106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f0102450 msgr2=0x7f19f01028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.717+0000 7f19f7374640 1 --2- 172.21.15.60:0/694533106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f0102450 0x7f19f01028d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009900 tx=0x7f19e002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:19.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.717+0000 7f19f7374640 1 -- 172.21.15.60:0/694533106 shutdown_connections 2024-02-26T04:57:19.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.717+0000 7f19f7374640 1 --2- 172.21.15.60:0/694533106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19f0102e10 0x7f19f01096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.717+0000 7f19f7374640 1 --2- 172.21.15.60:0/694533106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f0102450 0x7f19f01028d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.717+0000 7f19f7374640 1 --2- 172.21.15.60:0/694533106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f00ff480 0x7f19f00ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.717+0000 7f19f7374640 1 -- 172.21.15.60:0/694533106 >> 172.21.15.60:0/694533106 conn(0x7f19f00fadb0 msgr2=0x7f19f00fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:19.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.717+0000 7f19f7374640 1 -- 172.21.15.60:0/694533106 shutdown_connections 2024-02-26T04:57:19.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.717+0000 7f19f7374640 1 -- 172.21.15.60:0/694533106 wait complete. 2024-02-26T04:57:19.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f7374640 1 Processor -- start 2024-02-26T04:57:19.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f7374640 1 -- start start 2024-02-26T04:57:19.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f7374640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f00ff480 0x7f19f0197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:19.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f7374640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19f0102450 0x7f19f01977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f7374640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f0102e10 0x7f19f019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f7374640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f019ed50 con 0x7f19f0102e10 2024-02-26T04:57:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f7374640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f019eec0 con 0x7f19f00ff480 2024-02-26T04:57:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f7374640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f019f080 con 0x7f19f0102450 2024-02-26T04:57:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f5911640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f0102e10 0x7f19f019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f5911640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f0102e10 0x7f19f019e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50082/0 (socket says 172.21.15.60:50082) 2024-02-26T04:57:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f5911640 1 -- 172.21.15.60:0/3770493077 learned_addr learned my addr 172.21.15.60:0/3770493077 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f5110640 1 --2- 172.21.15.60:0/3770493077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f00ff480 0x7f19f0197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f490f640 1 --2- 172.21.15.60:0/3770493077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19f0102450 0x7f19f01977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f5911640 1 -- 172.21.15.60:0/3770493077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19f0102450 msgr2=0x7f19f01977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f5911640 1 --2- 172.21.15.60:0/3770493077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19f0102450 0x7f19f01977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f5911640 1 -- 172.21.15.60:0/3770493077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f00ff480 msgr2=0x7f19f0197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f5911640 1 --2- 172.21.15.60:0/3770493077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f00ff480 0x7f19f0197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.718+0000 7f19f5911640 1 -- 172.21.15.60:0/3770493077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0009560 con 0x7f19f0102e10 2024-02-26T04:57:19.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.719+0000 7f19f5911640 1 --2- 172.21.15.60:0/3770493077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f0102e10 0x7f19f019e810 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f19e800cad0 tx=0x7f19e800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:19.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.719+0000 7f19e67fc640 1 -- 172.21.15.60:0/3770493077 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e800ae20 con 0x7f19f0102e10 2024-02-26T04:57:19.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.719+0000 7f19e67fc640 1 -- 172.21.15.60:0/3770493077 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19e8005020 con 0x7f19f0102e10 2024-02-26T04:57:19.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.719+0000 7f19f7374640 1 -- 172.21.15.60:0/3770493077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f019f360 con 0x7f19f0102e10 2024-02-26T04:57:19.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.719+0000 7f19e67fc640 1 -- 172.21.15.60:0/3770493077 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e800f6b0 con 0x7f19f0102e10 2024-02-26T04:57:19.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.719+0000 7f19f7374640 1 -- 172.21.15.60:0/3770493077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f019f840 con 0x7f19f0102e10 2024-02-26T04:57:19.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.720+0000 7f19f7374640 1 -- 172.21.15.60:0/3770493077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f0108580 con 0x7f19f0102e10 2024-02-26T04:57:19.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.721+0000 7f19e67fc640 1 -- 172.21.15.60:0/3770493077 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f19e8005520 con 0x7f19f0102e10 2024-02-26T04:57:19.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.721+0000 7f19e67fc640 1 --2- 172.21.15.60:0/3770493077 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f19dc0a2150 0x7f19dc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:19.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.722+0000 7f19f5110640 1 --2- 172.21.15.60:0/3770493077 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f19dc0a2150 0x7f19dc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:19.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.722+0000 7f19e67fc640 1 -- 172.21.15.60:0/3770493077 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f19e80ce480 con 0x7f19f0102e10 2024-02-26T04:57:19.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.722+0000 7f19f5110640 1 --2- 172.21.15.60:0/3770493077 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f19dc0a2150 0x7f19dc0a4610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f19f00695b0 tx=0x7f19ec00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:19.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.726+0000 7f19e67fc640 1 -- 172.21.15.60:0/3770493077 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f19e8097040 con 0x7f19f0102e10 2024-02-26T04:57:19.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.907+0000 7f19f7374640 1 -- 172.21.15.60:0/3770493077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f19f002ce40 con 0x7f19f0102e10 2024-02-26T04:57:19.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.908+0000 7f19e67fc640 1 -- 172.21.15.60:0/3770493077 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f19e801d0a0 con 0x7f19f0102e10 2024-02-26T04:57:19.911 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T04:57:19.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.910+0000 7f19f7374640 1 -- 172.21.15.60:0/3770493077 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f19dc0a2150 msgr2=0x7f19dc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.910+0000 7f19f7374640 1 --2- 172.21.15.60:0/3770493077 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f19dc0a2150 0x7f19dc0a4610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f19f00695b0 tx=0x7f19ec00b390 comp rx=0 tx=0).stop 2024-02-26T04:57:19.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.910+0000 7f19f7374640 1 -- 172.21.15.60:0/3770493077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f0102e10 msgr2=0x7f19f019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:19.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.910+0000 7f19f7374640 1 --2- 172.21.15.60:0/3770493077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f0102e10 0x7f19f019e810 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f19e800cad0 tx=0x7f19e800cf90 comp rx=0 tx=0).stop 2024-02-26T04:57:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.910+0000 7f19f7374640 1 -- 172.21.15.60:0/3770493077 shutdown_connections 2024-02-26T04:57:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.910+0000 7f19f7374640 1 --2- 172.21.15.60:0/3770493077 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f19dc0a2150 0x7f19dc0a4610 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.910+0000 7f19f7374640 1 --2- 172.21.15.60:0/3770493077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f0102e10 0x7f19f019e810 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.910+0000 7f19f7374640 1 --2- 172.21.15.60:0/3770493077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19f0102450 0x7f19f01977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.910+0000 7f19f7374640 1 --2- 172.21.15.60:0/3770493077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f00ff480 0x7f19f0197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.910+0000 7f19f7374640 1 -- 172.21.15.60:0/3770493077 >> 172.21.15.60:0/3770493077 conn(0x7f19f00fadb0 msgr2=0x7f19f0105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.910+0000 7f19f7374640 1 -- 172.21.15.60:0/3770493077 shutdown_connections 2024-02-26T04:57:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:19.910+0000 7f19f7374640 1 -- 172.21.15.60:0/3770493077 wait complete. 2024-02-26T04:57:19.923 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T04:57:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.026+0000 7f03a7220640 1 -- 172.21.15.60:0/991174606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a0100360 msgr2=0x7f03a01007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.026+0000 7f03a7220640 1 --2- 172.21.15.60:0/991174606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a0100360 0x7f03a01007e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f038c009930 tx=0x7f038c02bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.026+0000 7f03a7220640 1 -- 172.21.15.60:0/991174606 shutdown_connections 2024-02-26T04:57:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.026+0000 7f03a7220640 1 --2- 172.21.15.60:0/991174606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03a0100d20 0x7f03a01085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.026+0000 7f03a7220640 1 --2- 172.21.15.60:0/991174606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a0100360 0x7f03a01007e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.026+0000 7f03a7220640 1 --2- 172.21.15.60:0/991174606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a0101630 0x7f03a0101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.026+0000 7f03a7220640 1 -- 172.21.15.60:0/991174606 >> 172.21.15.60:0/991174606 conn(0x7f03a00fadb0 msgr2=0x7f03a00fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.026+0000 7f03a7220640 1 -- 172.21.15.60:0/991174606 shutdown_connections 2024-02-26T04:57:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.026+0000 7f03a7220640 1 -- 172.21.15.60:0/991174606 wait complete. 2024-02-26T04:57:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a7220640 1 Processor -- start 2024-02-26T04:57:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a7220640 1 -- start start 2024-02-26T04:57:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a7220640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a0100360 0x7f03a0196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a7220640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03a0100d20 0x7f03a01966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a7220640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a0101630 0x7f03a019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a7220640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a019dc80 con 0x7f03a0100360 2024-02-26T04:57:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a7220640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a019ddf0 con 0x7f03a0100d20 2024-02-26T04:57:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a7220640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a019dfb0 con 0x7f03a0101630 2024-02-26T04:57:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a4fbc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a0100360 0x7f03a0196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a4fbc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a0100360 0x7f03a0196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50092/0 (socket says 172.21.15.60:50092) 2024-02-26T04:57:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a4fbc640 1 -- 172.21.15.60:0/2518942407 learned_addr learned my addr 172.21.15.60:0/2518942407 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f039ffff640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03a0100d20 0x7f03a01966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a4fbc640 1 -- 172.21.15.60:0/2518942407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a0101630 msgr2=0x7f03a019d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a57bd640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a0101630 0x7f03a019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a4fbc640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a0101630 0x7f03a019d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a4fbc640 1 -- 172.21.15.60:0/2518942407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03a0100d20 msgr2=0x7f03a01966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a4fbc640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03a0100d20 0x7f03a01966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a4fbc640 1 -- 172.21.15.60:0/2518942407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c009560 con 0x7f03a0100360 2024-02-26T04:57:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a57bd640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a0101630 0x7f03a019d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f039ffff640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03a0100d20 0x7f03a01966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.027+0000 7f03a4fbc640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a0100360 0x7f03a0196190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0394007920 tx=0x7f0394007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.028+0000 7f039dffb640 1 -- 172.21.15.60:0/2518942407 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394011070 con 0x7f03a0100360 2024-02-26T04:57:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.028+0000 7f039dffb640 1 -- 172.21.15.60:0/2518942407 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f039400c2e0 con 0x7f03a0100360 2024-02-26T04:57:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.028+0000 7f03a7220640 1 -- 172.21.15.60:0/2518942407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a00ff2b0 con 0x7f03a0100360 2024-02-26T04:57:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.028+0000 7f039dffb640 1 -- 172.21.15.60:0/2518942407 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394004a00 con 0x7f03a0100360 2024-02-26T04:57:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.028+0000 7f03a7220640 1 -- 172.21.15.60:0/2518942407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a00ff7c0 con 0x7f03a0100360 2024-02-26T04:57:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.029+0000 7f03a7220640 1 -- 172.21.15.60:0/2518942407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a00683c0 con 0x7f03a0100360 2024-02-26T04:57:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.030+0000 7f039dffb640 1 -- 172.21.15.60:0/2518942407 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f0394004ba0 con 0x7f03a0100360 2024-02-26T04:57:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.030+0000 7f039dffb640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f03900a21d0 0x7f03900a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.031+0000 7f039ffff640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f03900a21d0 0x7f03900a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.031+0000 7f039dffb640 1 -- 172.21.15.60:0/2518942407 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f03940cf2c0 con 0x7f03a0100360 2024-02-26T04:57:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.031+0000 7f039ffff640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f03900a21d0 0x7f03900a4690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f038c036040 tx=0x7f038c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.035+0000 7f039dffb640 1 -- 172.21.15.60:0/2518942407 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f039404fd20 con 0x7f03a0100360 2024-02-26T04:57:20.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.177+0000 7f039dffb640 1 -- 172.21.15.60:0/2518942407 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f03940981a0 con 0x7f03a0100360 2024-02-26T04:57:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:20 smithi060 ceph-mon[32382]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-02-26T04:57:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-02-26T04:57:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-02-26T04:57:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-02-26T04:57:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-02-26T04:57:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-02-26T04:57:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-02-26T04:57:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:20 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-02-26T04:57:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:20 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3516299544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:20 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1302895116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:20 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3770493077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:20.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.227+0000 7f03a7220640 1 -- 172.21.15.60:0/2518942407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f03a010bd80 con 0x7f03a0100360 2024-02-26T04:57:20.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.228+0000 7f039dffb640 1 -- 172.21.15.60:0/2518942407 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f03a010bd80 con 0x7f03a0100360 2024-02-26T04:57:20.231 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:57:20.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.230+0000 7f03a7220640 1 -- 172.21.15.60:0/2518942407 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f03900a21d0 msgr2=0x7f03900a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.230+0000 7f03a7220640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f03900a21d0 0x7f03900a4690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f038c036040 tx=0x7f038c000bc0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.230+0000 7f03a7220640 1 -- 172.21.15.60:0/2518942407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a0100360 msgr2=0x7f03a0196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.230+0000 7f03a7220640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a0100360 0x7f03a0196190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0394007920 tx=0x7f0394007de0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.230+0000 7f03a7220640 1 -- 172.21.15.60:0/2518942407 shutdown_connections 2024-02-26T04:57:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.230+0000 7f03a7220640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f03900a21d0 0x7f03900a4690 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.230+0000 7f03a7220640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a0101630 0x7f03a019d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.230+0000 7f03a7220640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03a0100d20 0x7f03a01966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.230+0000 7f03a7220640 1 --2- 172.21.15.60:0/2518942407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a0100360 0x7f03a0196190 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.230+0000 7f03a7220640 1 -- 172.21.15.60:0/2518942407 >> 172.21.15.60:0/2518942407 conn(0x7f03a00fadb0 msgr2=0x7f03a0104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.230+0000 7f03a7220640 1 -- 172.21.15.60:0/2518942407 shutdown_connections 2024-02-26T04:57:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.230+0000 7f03a7220640 1 -- 172.21.15.60:0/2518942407 wait complete. 2024-02-26T04:57:20.244 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:57:20.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.344+0000 7ffae291e640 1 -- 172.21.15.60:0/1108082450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffadc1000a0 msgr2=0x7ffadc1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.344+0000 7ffae291e640 1 --2- 172.21.15.60:0/1108082450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffadc1000a0 0x7ffadc1004c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009900 tx=0x7ffacc02bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:20.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.344+0000 7ffae291e640 1 -- 172.21.15.60:0/1108082450 shutdown_connections 2024-02-26T04:57:20.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.344+0000 7ffae291e640 1 --2- 172.21.15.60:0/1108082450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffadc101c50 0x7ffadc1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.344+0000 7ffae291e640 1 --2- 172.21.15.60:0/1108082450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffadc101290 0x7ffadc101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.344+0000 7ffae291e640 1 --2- 172.21.15.60:0/1108082450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffadc1000a0 0x7ffadc1004c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.344+0000 7ffae291e640 1 -- 172.21.15.60:0/1108082450 >> 172.21.15.60:0/1108082450 conn(0x7ffadc0fb610 msgr2=0x7ffadc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.345+0000 7ffae291e640 1 -- 172.21.15.60:0/1108082450 shutdown_connections 2024-02-26T04:57:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.345+0000 7ffae291e640 1 -- 172.21.15.60:0/1108082450 wait complete. 2024-02-26T04:57:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.345+0000 7ffae291e640 1 Processor -- start 2024-02-26T04:57:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.345+0000 7ffae291e640 1 -- start start 2024-02-26T04:57:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.345+0000 7ffae291e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffadc1000a0 0x7ffadc196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.345+0000 7ffae291e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffadc101290 0x7ffadc196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.345+0000 7ffae291e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffadc101c50 0x7ffadc19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.345+0000 7ffae291e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc19dc30 con 0x7ffadc101290 2024-02-26T04:57:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.345+0000 7ffae291e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc19dda0 con 0x7ffadc101c50 2024-02-26T04:57:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.345+0000 7ffae291e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc19df60 con 0x7ffadc1000a0 2024-02-26T04:57:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffadb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffadc101290 0x7ffadc196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffadb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffadc101290 0x7ffadc196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50104/0 (socket says 172.21.15.60:50104) 2024-02-26T04:57:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffadb7fe640 1 -- 172.21.15.60:0/2827875113 learned_addr learned my addr 172.21.15.60:0/2827875113 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffadbfff640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffadc1000a0 0x7ffadc196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffae0ebb640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffadc101c50 0x7ffadc19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffadb7fe640 1 -- 172.21.15.60:0/2827875113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffadc1000a0 msgr2=0x7ffadc196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffadb7fe640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffadc1000a0 0x7ffadc196140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffadb7fe640 1 -- 172.21.15.60:0/2827875113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffadc101c50 msgr2=0x7ffadc19d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffadb7fe640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffadc101c50 0x7ffadc19d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffadb7fe640 1 -- 172.21.15.60:0/2827875113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc009560 con 0x7ffadc101290 2024-02-26T04:57:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffadbfff640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffadc1000a0 0x7ffadc196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffae0ebb640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffadc101c50 0x7ffadc19d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffadb7fe640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffadc101290 0x7ffadc196680 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ffad000c920 tx=0x7ffad000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffad97fa640 1 -- 172.21.15.60:0/2827875113 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad0007c70 con 0x7ffadc101290 2024-02-26T04:57:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffad97fa640 1 -- 172.21.15.60:0/2827875113 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffad0007dd0 con 0x7ffadc101290 2024-02-26T04:57:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffae291e640 1 -- 172.21.15.60:0/2827875113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffadc19e240 con 0x7ffadc101290 2024-02-26T04:57:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffad97fa640 1 -- 172.21.15.60:0/2827875113 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad0018860 con 0x7ffadc101290 2024-02-26T04:57:20.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.346+0000 7ffae291e640 1 -- 172.21.15.60:0/2827875113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffadc19e750 con 0x7ffadc101290 2024-02-26T04:57:20.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.347+0000 7ffae291e640 1 -- 172.21.15.60:0/2827875113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffadc107360 con 0x7ffadc101290 2024-02-26T04:57:20.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.348+0000 7ffad97fa640 1 -- 172.21.15.60:0/2827875113 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ffad000fac0 con 0x7ffadc101290 2024-02-26T04:57:20.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.349+0000 7ffad97fa640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffac40a20c0 0x7ffac40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.349+0000 7ffadbfff640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffac40a20c0 0x7ffac40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.349+0000 7ffad97fa640 1 -- 172.21.15.60:0/2827875113 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ffad00ce310 con 0x7ffadc101290 2024-02-26T04:57:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.349+0000 7ffadbfff640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffac40a20c0 0x7ffac40a4580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffacc006010 tx=0x7ffacc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.353+0000 7ffad97fa640 1 -- 172.21.15.60:0/2827875113 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffad0096e40 con 0x7ffadc101290 2024-02-26T04:57:20.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:20 smithi156 ceph-mon[35805]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:20.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-02-26T04:57:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-02-26T04:57:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-02-26T04:57:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-02-26T04:57:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-02-26T04:57:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-02-26T04:57:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-02-26T04:57:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:20 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-02-26T04:57:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:20 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3516299544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:20 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1302895116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:20.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:20 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3770493077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:20.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:20 smithi191 ceph-mon[36352]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:20.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-02-26T04:57:20.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-02-26T04:57:20.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-02-26T04:57:20.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-02-26T04:57:20.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-02-26T04:57:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-02-26T04:57:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-02-26T04:57:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:20 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-02-26T04:57:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:20 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3516299544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:20 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1302895116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:20.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:20 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3770493077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:20.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.575+0000 7ffae291e640 1 -- 172.21.15.60:0/2827875113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffadc10bab0 con 0x7ffadc101290 2024-02-26T04:57:20.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.576+0000 7ffad97fa640 1 -- 172.21.15.60:0/2827875113 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11347 (secure 0 0 0) 0x7ffad009a500 con 0x7ffadc101290 2024-02-26T04:57:20.578 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:20.578 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:55:54.140330+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24476,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T04:57:20.579 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-02-26T04:57:20.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.578+0000 7ffae291e640 1 -- 172.21.15.60:0/2827875113 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffac40a20c0 msgr2=0x7ffac40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.578+0000 7ffae291e640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffac40a20c0 0x7ffac40a4580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffacc006010 tx=0x7ffacc000bc0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.578+0000 7ffae291e640 1 -- 172.21.15.60:0/2827875113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffadc101290 msgr2=0x7ffadc196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.578+0000 7ffae291e640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffadc101290 0x7ffadc196680 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ffad000c920 tx=0x7ffad000cde0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.578+0000 7ffae291e640 1 -- 172.21.15.60:0/2827875113 shutdown_connections 2024-02-26T04:57:20.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.578+0000 7ffae291e640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffac40a20c0 0x7ffac40a4580 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.578+0000 7ffae291e640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffadc101c50 0x7ffadc19d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.578+0000 7ffae291e640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffadc101290 0x7ffadc196680 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.578+0000 7ffae291e640 1 --2- 172.21.15.60:0/2827875113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffadc1000a0 0x7ffadc196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.578+0000 7ffae291e640 1 -- 172.21.15.60:0/2827875113 >> 172.21.15.60:0/2827875113 conn(0x7ffadc0fb610 msgr2=0x7ffadc104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:20.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.578+0000 7ffae291e640 1 -- 172.21.15.60:0/2827875113 shutdown_connections 2024-02-26T04:57:20.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.578+0000 7ffae291e640 1 -- 172.21.15.60:0/2827875113 wait complete. 2024-02-26T04:57:20.592 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:57:20.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.692+0000 7fe92bd1e640 1 -- 172.21.15.60:0/2189047362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe924101310 msgr2=0x7fe924101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.692+0000 7fe92bd1e640 1 --2- 172.21.15.60:0/2189047362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe924101310 0x7fe924101790 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe914009900 tx=0x7fe91402bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:20.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.692+0000 7fe92bd1e640 1 -- 172.21.15.60:0/2189047362 shutdown_connections 2024-02-26T04:57:20.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.692+0000 7fe92bd1e640 1 --2- 172.21.15.60:0/2189047362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe924101cd0 0x7fe924108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.692+0000 7fe92bd1e640 1 --2- 172.21.15.60:0/2189047362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe924101310 0x7fe924101790 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.692+0000 7fe92bd1e640 1 --2- 172.21.15.60:0/2189047362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe924100120 0x7fe924100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.692+0000 7fe92bd1e640 1 -- 172.21.15.60:0/2189047362 >> 172.21.15.60:0/2189047362 conn(0x7fe9240fb6b0 msgr2=0x7fe9240fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.692+0000 7fe92bd1e640 1 -- 172.21.15.60:0/2189047362 shutdown_connections 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.692+0000 7fe92bd1e640 1 -- 172.21.15.60:0/2189047362 wait complete. 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.692+0000 7fe92bd1e640 1 Processor -- start 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe92bd1e640 1 -- start start 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe92bd1e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe924100120 0x7fe9241971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe92bd1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe924101310 0x7fe9241976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe92bd1e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe924101cd0 0x7fe92419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe92bd1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92419eca0 con 0x7fe924101310 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe92bd1e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92419ee10 con 0x7fe924101cd0 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe92bd1e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92419efd0 con 0x7fe924100120 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe9292b9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe924101310 0x7fe9241976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe9292b9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe924101310 0x7fe9241976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50122/0 (socket says 172.21.15.60:50122) 2024-02-26T04:57:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe9292b9640 1 -- 172.21.15.60:0/903088421 learned_addr learned my addr 172.21.15.60:0/903088421 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe929aba640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe924100120 0x7fe9241971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe92a2bb640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe924101cd0 0x7fe92419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe9292b9640 1 -- 172.21.15.60:0/903088421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe924100120 msgr2=0x7fe9241971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe9292b9640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe924100120 0x7fe9241971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe9292b9640 1 -- 172.21.15.60:0/903088421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe924101cd0 msgr2=0x7fe92419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe9292b9640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe924101cd0 0x7fe92419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe9292b9640 1 -- 172.21.15.60:0/903088421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe914009560 con 0x7fe924101310 2024-02-26T04:57:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe92a2bb640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe924101cd0 0x7fe92419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe9292b9640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe924101310 0x7fe9241976f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe914006010 tx=0x7fe9140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.693+0000 7fe91affd640 1 -- 172.21.15.60:0/903088421 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140044e0 con 0x7fe924101310 2024-02-26T04:57:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.694+0000 7fe91affd640 1 -- 172.21.15.60:0/903088421 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9140054e0 con 0x7fe924101310 2024-02-26T04:57:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.694+0000 7fe92bd1e640 1 -- 172.21.15.60:0/903088421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92419f250 con 0x7fe924101310 2024-02-26T04:57:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.694+0000 7fe91affd640 1 -- 172.21.15.60:0/903088421 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91403e450 con 0x7fe924101310 2024-02-26T04:57:20.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.694+0000 7fe92bd1e640 1 -- 172.21.15.60:0/903088421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92419f760 con 0x7fe924101310 2024-02-26T04:57:20.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.694+0000 7fe92bd1e640 1 -- 172.21.15.60:0/903088421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe924107440 con 0x7fe924101310 2024-02-26T04:57:20.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.695+0000 7fe91affd640 1 -- 172.21.15.60:0/903088421 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe9140035f0 con 0x7fe924101310 2024-02-26T04:57:20.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.696+0000 7fe91affd640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe9100a20c0 0x7fe9100a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:20.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.696+0000 7fe929aba640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe9100a20c0 0x7fe9100a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:20.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.696+0000 7fe91affd640 1 -- 172.21.15.60:0/903088421 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fe9140f03f0 con 0x7fe924101310 2024-02-26T04:57:20.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.696+0000 7fe929aba640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe9100a20c0 0x7fe9100a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe920005fd0 tx=0x7fe92000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:20.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.700+0000 7fe91affd640 1 -- 172.21.15.60:0/903088421 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe9140b9020 con 0x7fe924101310 2024-02-26T04:57:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.882+0000 7fe92bd1e640 1 -- 172.21.15.60:0/903088421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe92410ce10 con 0x7fe924101310 2024-02-26T04:57:20.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.883+0000 7fe91affd640 1 -- 172.21.15.60:0/903088421 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24972 (secure 0 0 0) 0x7fe9140bc6e0 con 0x7fe924101310 2024-02-26T04:57:20.886 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:20.886 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":92,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:57:20.162055+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:57:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.885+0000 7fe92bd1e640 1 -- 172.21.15.60:0/903088421 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe9100a20c0 msgr2=0x7fe9100a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.885+0000 7fe92bd1e640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe9100a20c0 0x7fe9100a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe920005fd0 tx=0x7fe92000b390 comp rx=0 tx=0).stop 2024-02-26T04:57:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.885+0000 7fe92bd1e640 1 -- 172.21.15.60:0/903088421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe924101310 msgr2=0x7fe9241976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.885+0000 7fe92bd1e640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe924101310 0x7fe9241976f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe914006010 tx=0x7fe9140042f0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.885+0000 7fe92bd1e640 1 -- 172.21.15.60:0/903088421 shutdown_connections 2024-02-26T04:57:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.885+0000 7fe92bd1e640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe9100a20c0 0x7fe9100a4580 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.885+0000 7fe92bd1e640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe924101cd0 0x7fe92419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.885+0000 7fe92bd1e640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe924101310 0x7fe9241976f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.885+0000 7fe92bd1e640 1 --2- 172.21.15.60:0/903088421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe924100120 0x7fe9241971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.885+0000 7fe92bd1e640 1 -- 172.21.15.60:0/903088421 >> 172.21.15.60:0/903088421 conn(0x7fe9240fb6b0 msgr2=0x7fe924104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.885+0000 7fe92bd1e640 1 -- 172.21.15.60:0/903088421 shutdown_connections 2024-02-26T04:57:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.885+0000 7fe92bd1e640 1 -- 172.21.15.60:0/903088421 wait complete. 2024-02-26T04:57:20.899 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:57:21.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.999+0000 7fd453220640 1 -- 172.21.15.60:0/246110995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c102d40 msgr2=0x7fd44c1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.999+0000 7fd453220640 1 --2- 172.21.15.60:0/246110995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c102d40 0x7fd44c1095d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009900 tx=0x7fd43c02bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:21.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.999+0000 7fd453220640 1 -- 172.21.15.60:0/246110995 shutdown_connections 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.999+0000 7fd453220640 1 --2- 172.21.15.60:0/246110995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c102d40 0x7fd44c1095d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.999+0000 7fd453220640 1 --2- 172.21.15.60:0/246110995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd44c102380 0x7fd44c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.999+0000 7fd453220640 1 --2- 172.21.15.60:0/246110995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd44c101190 0x7fd44c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.999+0000 7fd453220640 1 -- 172.21.15.60:0/246110995 >> 172.21.15.60:0/246110995 conn(0x7fd44c0fc760 msgr2=0x7fd44c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.999+0000 7fd453220640 1 -- 172.21.15.60:0/246110995 shutdown_connections 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:20.999+0000 7fd453220640 1 -- 172.21.15.60:0/246110995 wait complete. 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd453220640 1 Processor -- start 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd453220640 1 -- start start 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd453220640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd44c101190 0x7fd44c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd453220640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd44c102380 0x7fd44c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd453220640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c102d40 0x7fd44c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd453220640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c19ed40 con 0x7fd44c102d40 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd453220640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c19eeb0 con 0x7fd44c101190 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd453220640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c19f070 con 0x7fd44c102380 2024-02-26T04:57:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd4517bd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c102d40 0x7fd44c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd4517bd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c102d40 0x7fd44c19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50144/0 (socket says 172.21.15.60:50144) 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd4517bd640 1 -- 172.21.15.60:0/4116962152 learned_addr learned my addr 172.21.15.60:0/4116962152 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd44bfff640 1 --2- 172.21.15.60:0/4116962152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd44c102380 0x7fd44c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd450fbc640 1 --2- 172.21.15.60:0/4116962152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd44c101190 0x7fd44c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd4517bd640 1 -- 172.21.15.60:0/4116962152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd44c102380 msgr2=0x7fd44c197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd4517bd640 1 --2- 172.21.15.60:0/4116962152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd44c102380 0x7fd44c197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd4517bd640 1 -- 172.21.15.60:0/4116962152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd44c101190 msgr2=0x7fd44c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd4517bd640 1 --2- 172.21.15.60:0/4116962152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd44c101190 0x7fd44c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd4517bd640 1 -- 172.21.15.60:0/4116962152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c009560 con 0x7fd44c102d40 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.000+0000 7fd4517bd640 1 --2- 172.21.15.60:0/4116962152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c102d40 0x7fd44c19e800 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd43c005ab0 tx=0x7fd43c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.001+0000 7fd449ffb640 1 -- 172.21.15.60:0/4116962152 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c004aa0 con 0x7fd44c102d40 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.001+0000 7fd449ffb640 1 -- 172.21.15.60:0/4116962152 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd43c0052d0 con 0x7fd44c102d40 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.001+0000 7fd453220640 1 -- 172.21.15.60:0/4116962152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44c19f2f0 con 0x7fd44c102d40 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.001+0000 7fd449ffb640 1 -- 172.21.15.60:0/4116962152 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c003e80 con 0x7fd44c102d40 2024-02-26T04:57:21.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.001+0000 7fd453220640 1 -- 172.21.15.60:0/4116962152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c19f850 con 0x7fd44c102d40 2024-02-26T04:57:21.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.002+0000 7fd453220640 1 -- 172.21.15.60:0/4116962152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44c108450 con 0x7fd44c102d40 2024-02-26T04:57:21.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.003+0000 7fd449ffb640 1 -- 172.21.15.60:0/4116962152 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd43c0035f0 con 0x7fd44c102d40 2024-02-26T04:57:21.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.003+0000 7fd449ffb640 1 --2- 172.21.15.60:0/4116962152 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd4380a2140 0x7fd4380a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.003+0000 7fd449ffb640 1 -- 172.21.15.60:0/4116962152 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fd43c0f0a50 con 0x7fd44c102d40 2024-02-26T04:57:21.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.003+0000 7fd450fbc640 1 --2- 172.21.15.60:0/4116962152 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd4380a2140 0x7fd4380a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.004+0000 7fd450fbc640 1 --2- 172.21.15.60:0/4116962152 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd4380a2140 0x7fd4380a4600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd44c1021e0 tx=0x7fd444009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.008+0000 7fd449ffb640 1 -- 172.21.15.60:0/4116962152 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd43c0b95e0 con 0x7fd44c102d40 2024-02-26T04:57:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.236+0000 7fd453220640 1 -- 172.21.15.60:0/4116962152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd44c02ce40 con 0x7fd44c102d40 2024-02-26T04:57:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.237+0000 7fd449ffb640 1 -- 172.21.15.60:0/4116962152 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11347 (secure 0 0 0) 0x7fd43c0bcca0 con 0x7fd44c102d40 2024-02-26T04:57:21.240 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:21.240 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:55:54.140330+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24476,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T04:57:21.240 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-02-26T04:57:21.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.239+0000 7fd453220640 1 -- 172.21.15.60:0/4116962152 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd4380a2140 msgr2=0x7fd4380a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.239+0000 7fd453220640 1 --2- 172.21.15.60:0/4116962152 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd4380a2140 0x7fd4380a4600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd44c1021e0 tx=0x7fd444009870 comp rx=0 tx=0).stop 2024-02-26T04:57:21.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.239+0000 7fd453220640 1 -- 172.21.15.60:0/4116962152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c102d40 msgr2=0x7fd44c19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.239+0000 7fd453220640 1 --2- 172.21.15.60:0/4116962152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c102d40 0x7fd44c19e800 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd43c005ab0 tx=0x7fd43c004270 comp rx=0 tx=0).stop 2024-02-26T04:57:21.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.239+0000 7fd453220640 1 -- 172.21.15.60:0/4116962152 shutdown_connections 2024-02-26T04:57:21.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.239+0000 7fd453220640 1 --2- 172.21.15.60:0/4116962152 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd4380a2140 0x7fd4380a4600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.239+0000 7fd453220640 1 --2- 172.21.15.60:0/4116962152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c102d40 0x7fd44c19e800 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.239+0000 7fd453220640 1 --2- 172.21.15.60:0/4116962152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd44c102380 0x7fd44c197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.239+0000 7fd453220640 1 --2- 172.21.15.60:0/4116962152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd44c101190 0x7fd44c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.239+0000 7fd453220640 1 -- 172.21.15.60:0/4116962152 >> 172.21.15.60:0/4116962152 conn(0x7fd44c0fc760 msgr2=0x7fd44c105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.239+0000 7fd453220640 1 -- 172.21.15.60:0/4116962152 shutdown_connections 2024-02-26T04:57:21.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.239+0000 7fd453220640 1 -- 172.21.15.60:0/4116962152 wait complete. 2024-02-26T04:57:21.253 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-26T04:55:27.553789+0000', 'modified': '2024-02-26T04:55:54.140330+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24524, 'mds_1': 24476, 'mds_2': 24472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6835/1642858086', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 1642858086}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 1642858086}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/2819944290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 2819944290}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 2819944290}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6837/3000497710', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 3000497710}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 3000497710}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.191:6839/640686517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6838', 'nonce': 640686517}, {'type': 'v1', 'addr': '172.21.15.191:6839', 'nonce': 640686517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.156:6837/707575792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 707575792}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 707575792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.156:6839/1768629491', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 1768629491}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 1768629491}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-26T04:57:21.253 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-02-26T04:57:21.253 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-26T04:57:21.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.354+0000 7fd94582d640 1 -- 172.21.15.60:0/3100791283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd940101280 msgr2=0x7fd940101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.354+0000 7fd94582d640 1 --2- 172.21.15.60:0/3100791283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd940101280 0x7fd940101700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd934009900 tx=0x7fd93402bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:21.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.354+0000 7fd94582d640 1 -- 172.21.15.60:0/3100791283 shutdown_connections 2024-02-26T04:57:21.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.354+0000 7fd94582d640 1 --2- 172.21.15.60:0/3100791283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd940101c40 0x7fd9401084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.354+0000 7fd94582d640 1 --2- 172.21.15.60:0/3100791283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd940101280 0x7fd940101700 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.354+0000 7fd94582d640 1 --2- 172.21.15.60:0/3100791283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd940100090 0x7fd9401004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.354+0000 7fd94582d640 1 -- 172.21.15.60:0/3100791283 >> 172.21.15.60:0/3100791283 conn(0x7fd9400fb620 msgr2=0x7fd9400fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.354+0000 7fd94582d640 1 -- 172.21.15.60:0/3100791283 shutdown_connections 2024-02-26T04:57:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.354+0000 7fd94582d640 1 -- 172.21.15.60:0/3100791283 wait complete. 2024-02-26T04:57:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.355+0000 7fd94582d640 1 Processor -- start 2024-02-26T04:57:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.355+0000 7fd94582d640 1 -- start start 2024-02-26T04:57:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.355+0000 7fd94582d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd940100090 0x7fd94006a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.355+0000 7fd94582d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd940101280 0x7fd94019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.355+0000 7fd94582d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd940101c40 0x7fd94019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.355+0000 7fd94582d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94019edb0 con 0x7fd940101c40 2024-02-26T04:57:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.355+0000 7fd94582d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94019ef20 con 0x7fd940101280 2024-02-26T04:57:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.355+0000 7fd94582d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94019f0e0 con 0x7fd940100090 2024-02-26T04:57:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.355+0000 7fd93f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd940101c40 0x7fd94019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd93f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd940101c40 0x7fd94019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50160/0 (socket says 172.21.15.60:50160) 2024-02-26T04:57:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd93f7fe640 1 -- 172.21.15.60:0/346879181 learned_addr learned my addr 172.21.15.60:0/346879181 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd940101280 0x7fd94019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd93effd640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd940100090 0x7fd94006a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd93f7fe640 1 -- 172.21.15.60:0/346879181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd940100090 msgr2=0x7fd94006a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd93f7fe640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd940100090 0x7fd94006a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd93f7fe640 1 -- 172.21.15.60:0/346879181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd940101280 msgr2=0x7fd94019bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd93f7fe640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd940101280 0x7fd94019bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd93f7fe640 1 -- 172.21.15.60:0/346879181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd934009560 con 0x7fd940101c40 2024-02-26T04:57:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd93effd640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd940100090 0x7fd94006a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd93f7fe640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd940101c40 0x7fd94019e620 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd93000eaa0 tx=0x7fd93000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd94482b640 1 -- 172.21.15.60:0/346879181 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93000cdb0 con 0x7fd940101c40 2024-02-26T04:57:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd94482b640 1 -- 172.21.15.60:0/346879181 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd930005020 con 0x7fd940101c40 2024-02-26T04:57:21.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd94582d640 1 -- 172.21.15.60:0/346879181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94019f3c0 con 0x7fd940101c40 2024-02-26T04:57:21.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd94482b640 1 -- 172.21.15.60:0/346879181 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9300116b0 con 0x7fd940101c40 2024-02-26T04:57:21.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.356+0000 7fd94582d640 1 -- 172.21.15.60:0/346879181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94019f900 con 0x7fd940101c40 2024-02-26T04:57:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.358+0000 7fd94482b640 1 -- 172.21.15.60:0/346879181 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd930005520 con 0x7fd940101c40 2024-02-26T04:57:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd948603740 1 Processor -- start 2024-02-26T04:57:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd948603740 1 -- start start 2024-02-26T04:57:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd948603740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f71f50 0x563991f72370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd948603740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f72940 0x563991f77440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd948603740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f77ab0 0x563991f79f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd948603740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563991f2dca0 con 0x563991f71f50 2024-02-26T04:57:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd948603740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563991ee0ef0 con 0x563991f77ab0 2024-02-26T04:57:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd948603740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563991ede420 con 0x563991f72940 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd94482b640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd9280a20c0 0x7fd9280a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd93e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f71f50 0x563991f72370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd93e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f71f50 0x563991f72370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50172/0 (socket says 172.21.15.60:50172) 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd93e7fc640 1 -- 172.21.15.60:0/119040644 learned_addr learned my addr 172.21.15.60:0/119040644 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd93effd640 1 --2- 172.21.15.60:0/119040644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f72940 0x563991f77440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd93f7fe640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd9280a20c0 0x7fd9280a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd94482b640 1 -- 172.21.15.60:0/346879181 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fd9300ce3a0 con 0x7fd940101c40 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd93e7fc640 1 -- 172.21.15.60:0/119040644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f72940 msgr2=0x563991f77440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/119040644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f72940 0x563991f77440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd93e7fc640 1 -- 172.21.15.60:0/119040644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f77ab0 msgr2=0x563991f79f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/119040644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f77ab0 0x563991f79f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd93e7fc640 1 -- 172.21.15.60:0/119040644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563991e8eef0 con 0x563991f71f50 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.359+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/119040644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f71f50 0x563991f72370 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd93402c170 tx=0x7fd934005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd8cc73bd4e4a00 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.360+0000 7fd93f7fe640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd9280a20c0 0x7fd9280a4580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd93000cfd0 tx=0x7fd930069e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.360+0000 7fd925ffb640 1 -- 172.21.15.60:0/119040644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9340048c0 con 0x563991f71f50 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.360+0000 7fd925ffb640 1 -- 172.21.15.60:0/119040644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd934004510 con 0x563991f71f50 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.360+0000 7fd925ffb640 1 -- 172.21.15.60:0/119040644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934005500 con 0x563991f71f50 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.360+0000 7fd948603740 1 -- 172.21.15.60:0/119040644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f71f50 msgr2=0x563991f72370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.360+0000 7fd948603740 1 --2- 172.21.15.60:0/119040644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f71f50 0x563991f72370 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd93402c170 tx=0x7fd934005e80 comp rx=0 tx=0).stop 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.360+0000 7fd948603740 1 -- 172.21.15.60:0/119040644 shutdown_connections 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.360+0000 7fd948603740 1 --2- 172.21.15.60:0/119040644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f77ab0 0x563991f79f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.360+0000 7fd948603740 1 --2- 172.21.15.60:0/119040644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f72940 0x563991f77440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.360+0000 7fd948603740 1 --2- 172.21.15.60:0/119040644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f71f50 0x563991f72370 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.360+0000 7fd948603740 1 -- 172.21.15.60:0/119040644 >> 172.21.15.60:0/119040644 conn(0x563991f6d4c0 msgr2=0x563991f6d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.360+0000 7fd948603740 1 -- 172.21.15.60:0/119040644 shutdown_connections 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.360+0000 7fd948603740 1 -- 172.21.15.60:0/119040644 wait complete. 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.361+0000 7fd948603740 1 -- start start 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.361+0000 7fd948603740 1 Processor -- start 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.361+0000 7fd948603740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f71f50 0x563991f7c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.361+0000 7fd948603740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f72940 0x563991f7c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.361+0000 7fd948603740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f77ab0 0x563991f83660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.361+0000 7fd948603740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563991ed9c00 con 0x563991f77ab0 2024-02-26T04:57:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.361+0000 7fd948603740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563991e7ed00 con 0x563991f72940 2024-02-26T04:57:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.361+0000 7fd948603740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563991bad310 con 0x563991f71f50 2024-02-26T04:57:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.361+0000 7fd948603740 10 client.0 _ll_register_callbacks cb 0x563991e86d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.361+0000 7fd93e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f77ab0 0x563991f83660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.361+0000 7fd93effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f72940 0x563991f7c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.361+0000 7fd93e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f77ab0 0x563991f83660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50188/0 (socket says 172.21.15.60:50188) 2024-02-26T04:57:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.361+0000 7fd93e7fc640 1 -- 172.21.15.60:0/2100477471 learned_addr learned my addr 172.21.15.60:0/2100477471 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.361+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/2100477471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f71f50 0x563991f7c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd93e7fc640 1 -- 172.21.15.60:0/2100477471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f71f50 msgr2=0x563991f7c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/2100477471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f71f50 0x563991f7c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd93e7fc640 1 -- 172.21.15.60:0/2100477471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f72940 msgr2=0x563991f7c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/2100477471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f72940 0x563991f7c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd93e7fc640 1 -- 172.21.15.60:0/2100477471 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd940108dd0 con 0x563991f77ab0 2024-02-26T04:57:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/2100477471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f77ab0 0x563991f83660 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd934006010 tx=0x7fd93402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7d1f44192c8c28 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd91effd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd91effd640 1 -- 172.21.15.60:0/2100477471 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934040460 con 0x563991f77ab0 2024-02-26T04:57:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd91effd640 1 -- 172.21.15.60:0/2100477471 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fd9340405c0 con 0x563991f77ab0 2024-02-26T04:57:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd948603740 1 -- 172.21.15.60:0/2100477471 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563991bd96c0 con 0x563991f77ab0 2024-02-26T04:57:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd91effd640 1 -- 172.21.15.60:0/2100477471 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd934040780 con 0x563991f77ab0 2024-02-26T04:57:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd948603740 10 client.15210 fetch_fsmap learned FSMap version 22 2024-02-26T04:57:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd948603740 1 -- 172.21.15.60:0/2100477471 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x563991ed8280 con 0x563991f77ab0 2024-02-26T04:57:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd91effd640 1 -- 172.21.15.60:0/2100477471 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563991bd96c0 con 0x563991f77ab0 2024-02-26T04:57:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd91effd640 1 -- 172.21.15.60:0/2100477471 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd934034ab0 con 0x563991f77ab0 2024-02-26T04:57:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd948603740 10 client.15210 fetch_fsmap finished waiting for FSMap version 22 2024-02-26T04:57:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd948603740 10 client.15210 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T04:57:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd948603740 20 client.15210 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd948603740 1 --2- 172.21.15.60:0/2100477471 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x563991f87fe0 0x563991f8ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd948603740 4 client.15210 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:57:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.362+0000 7fd948603740 1 -- 172.21.15.60:0/2100477471 --> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563991c2a270 con 0x563991f87fe0 2024-02-26T04:57:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.363+0000 7fd93effd640 1 --2- 172.21.15.60:0/2100477471 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x563991f87fe0 0x563991f8ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.363+0000 7fd91effd640 10 client.15210 ms_handle_connect on v2:172.21.15.191:6838/640686517 2024-02-26T04:57:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.363+0000 7fd93effd640 1 --2- 172.21.15.60:0/2100477471 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x563991f87fe0 0x563991f8ca80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4dd7c25468d0c05f server_cookie=a82727e8a4c03c61 in_seq=0 out_seq=0 2024-02-26T04:57:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.364+0000 7fd91effd640 1 -- 172.21.15.60:0/2100477471 <== mds.2 v2:172.21.15.191:6838/640686517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563991c2a270 con 0x563991f87fe0 2024-02-26T04:57:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.364+0000 7fd91effd640 10 client.15210 handle_command_reply: tid=0 2024-02-26T04:57:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.364+0000 7fd948603740 1 client.15210 shutdown 2024-02-26T04:57:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.364+0000 7fd948603740 1 -- 172.21.15.60:0/2100477471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f77ab0 msgr2=0x563991f83660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.364+0000 7fd948603740 1 --2- 172.21.15.60:0/2100477471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f77ab0 0x563991f83660 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd934006010 tx=0x7fd93402c7e0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.364+0000 7fd948603740 1 -- 172.21.15.60:0/2100477471 shutdown_connections 2024-02-26T04:57:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.364+0000 7fd948603740 1 --2- 172.21.15.60:0/2100477471 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x563991f87fe0 0x563991f8ca80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.364+0000 7fd948603740 1 --2- 172.21.15.60:0/2100477471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f77ab0 0x563991f83660 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.364+0000 7fd948603740 1 --2- 172.21.15.60:0/2100477471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f72940 0x563991f7c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.364+0000 7fd948603740 1 --2- 172.21.15.60:0/2100477471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f71f50 0x563991f7c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.364+0000 7fd91effd640 0 client.15210 ms_handle_reset on v2:172.21.15.191:6838/640686517 2024-02-26T04:57:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.364+0000 7fd948603740 1 -- 172.21.15.60:0/2100477471 >> 172.21.15.60:0/2100477471 conn(0x563991f6d4c0 msgr2=0x563991f6c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.364+0000 7fd948603740 1 -- 172.21.15.60:0/2100477471 shutdown_connections 2024-02-26T04:57:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.364+0000 7fd948603740 1 -- 172.21.15.60:0/2100477471 wait complete. 2024-02-26T04:57:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.365+0000 7fd948603740 20 client.15210 trim_cache size 0 max 16384 2024-02-26T04:57:21.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.380+0000 7fd948603740 1 Processor -- start 2024-02-26T04:57:21.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.380+0000 7fd948603740 1 -- start start 2024-02-26T04:57:21.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.380+0000 7fd948603740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f71f50 0x563991f6d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd948603740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f72940 0x563991f79ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd948603740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f77ab0 0x563991f911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd948603740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563991e5f460 con 0x563991f77ab0 2024-02-26T04:57:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd948603740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563991f7ae80 con 0x563991f72940 2024-02-26T04:57:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd948603740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563991f73a50 con 0x563991f71f50 2024-02-26T04:57:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd93e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f77ab0 0x563991f911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd93e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f77ab0 0x563991f911b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50200/0 (socket says 172.21.15.60:50200) 2024-02-26T04:57:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd93e7fc640 1 -- 172.21.15.60:0/3709032662 learned_addr learned my addr 172.21.15.60:0/3709032662 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd93e7fc640 1 -- 172.21.15.60:0/3709032662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f71f50 msgr2=0x563991f6d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/3709032662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f71f50 0x563991f6d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd93effd640 1 --2- 172.21.15.60:0/3709032662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f72940 0x563991f79ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd93e7fc640 1 -- 172.21.15.60:0/3709032662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f72940 msgr2=0x563991f79ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/3709032662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f72940 0x563991f79ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd93e7fc640 1 -- 172.21.15.60:0/3709032662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563991e749f0 con 0x563991f77ab0 2024-02-26T04:57:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd93effd640 1 --2- 172.21.15.60:0/3709032662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f72940 0x563991f79ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/3709032662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f77ab0 0x563991f911b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd9340059b0 tx=0x7fd934005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4769af0bbc2b9a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd9257fa640 1 -- 172.21.15.60:0/3709032662 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93402ccb0 con 0x563991f77ab0 2024-02-26T04:57:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd9257fa640 1 -- 172.21.15.60:0/3709032662 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd93404f070 con 0x563991f77ab0 2024-02-26T04:57:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.381+0000 7fd9257fa640 1 -- 172.21.15.60:0/3709032662 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934034440 con 0x563991f77ab0 2024-02-26T04:57:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.382+0000 7fd948603740 1 -- 172.21.15.60:0/3709032662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f77ab0 msgr2=0x563991f911b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.382+0000 7fd948603740 1 --2- 172.21.15.60:0/3709032662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f77ab0 0x563991f911b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd9340059b0 tx=0x7fd934005780 comp rx=0 tx=0).stop 2024-02-26T04:57:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.382+0000 7fd948603740 1 -- 172.21.15.60:0/3709032662 shutdown_connections 2024-02-26T04:57:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.382+0000 7fd948603740 1 --2- 172.21.15.60:0/3709032662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f77ab0 0x563991f911b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.382+0000 7fd948603740 1 --2- 172.21.15.60:0/3709032662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f72940 0x563991f79ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.382+0000 7fd948603740 1 --2- 172.21.15.60:0/3709032662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f71f50 0x563991f6d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.382+0000 7fd948603740 1 -- 172.21.15.60:0/3709032662 >> 172.21.15.60:0/3709032662 conn(0x563991f6d4c0 msgr2=0x563991f90aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.382+0000 7fd948603740 1 -- 172.21.15.60:0/3709032662 shutdown_connections 2024-02-26T04:57:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.382+0000 7fd948603740 1 -- 172.21.15.60:0/3709032662 wait complete. 2024-02-26T04:57:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.382+0000 7fd948603740 1 -- start start 2024-02-26T04:57:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.382+0000 7fd948603740 1 Processor -- start 2024-02-26T04:57:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd948603740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f71f50 0x563991fdaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd948603740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f72940 0x563991fdb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd948603740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f77ab0 0x563991fe2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd948603740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563991fe25b0 con 0x563991f72940 2024-02-26T04:57:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd948603740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563991fe26f0 con 0x563991f71f50 2024-02-26T04:57:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd948603740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563991fe2860 con 0x563991f77ab0 2024-02-26T04:57:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd948603740 10 client.0 _ll_register_callbacks cb 0x7fd934009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd93effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f72940 0x563991fdb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd93effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f72940 0x563991fdb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50212/0 (socket says 172.21.15.60:50212) 2024-02-26T04:57:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd93effd640 1 -- 172.21.15.60:0/2750858263 learned_addr learned my addr 172.21.15.60:0/2750858263 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/2750858263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f71f50 0x563991fdaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/2750858263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f77ab0 0x563991fe2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd93effd640 1 -- 172.21.15.60:0/2750858263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f77ab0 msgr2=0x563991fe2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd93effd640 1 --2- 172.21.15.60:0/2750858263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f77ab0 0x563991fe2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd93effd640 1 -- 172.21.15.60:0/2750858263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f71f50 msgr2=0x563991fdaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd93effd640 1 --2- 172.21.15.60:0/2750858263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f71f50 0x563991fdaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd93effd640 1 -- 172.21.15.60:0/2750858263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563991ed8280 con 0x563991f72940 2024-02-26T04:57:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/2750858263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f77ab0 0x563991fe2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd93effd640 1 --2- 172.21.15.60:0/2750858263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f72940 0x563991fdb030 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd938004810 tx=0x7fd93800a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611fda3e5fe47bca server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd91effd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd91effd640 1 -- 172.21.15.60:0/2750858263 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93801a070 con 0x563991f72940 2024-02-26T04:57:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd91effd640 1 -- 172.21.15.60:0/2750858263 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fd93801a760 con 0x563991f72940 2024-02-26T04:57:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd948603740 1 -- 172.21.15.60:0/2750858263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563991f88970 con 0x563991f72940 2024-02-26T04:57:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd91effd640 1 -- 172.21.15.60:0/2750858263 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd93801a920 con 0x563991f72940 2024-02-26T04:57:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd948603740 10 client.15216 fetch_fsmap learned FSMap version 22 2024-02-26T04:57:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.383+0000 7fd948603740 1 -- 172.21.15.60:0/2750858263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x563991e9a750 con 0x563991f72940 2024-02-26T04:57:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.384+0000 7fd91effd640 1 -- 172.21.15.60:0/2750858263 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563991f88970 con 0x563991f72940 2024-02-26T04:57:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.384+0000 7fd91effd640 1 -- 172.21.15.60:0/2750858263 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd938011040 con 0x563991f72940 2024-02-26T04:57:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.384+0000 7fd948603740 10 client.15216 fetch_fsmap finished waiting for FSMap version 22 2024-02-26T04:57:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.384+0000 7fd948603740 10 client.15216 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T04:57:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.384+0000 7fd948603740 20 client.15216 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.384+0000 7fd948603740 1 --2- 172.21.15.60:0/2750858263 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x563991fe7c40 0x563991fe40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.384+0000 7fd948603740 4 client.15216 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T04:57:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.384+0000 7fd948603740 1 -- 172.21.15.60:0/2750858263 --> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fd934004510 con 0x563991fe7c40 2024-02-26T04:57:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.384+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/2750858263 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x563991fe7c40 0x563991fe40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd91effd640 10 client.15216 ms_handle_connect on v2:172.21.15.191:6838/640686517 2024-02-26T04:57:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd93e7fc640 1 --2- 172.21.15.60:0/2750858263 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x563991fe7c40 0x563991fe40e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b8e55ed8d1f2622 server_cookie=d121e670ccbbd6ac in_seq=0 out_seq=0 2024-02-26T04:57:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd91effd640 1 -- 172.21.15.60:0/2750858263 <== mds.2 v2:172.21.15.191:6838/640686517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7fd934004510 con 0x563991fe7c40 2024-02-26T04:57:21.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd91effd640 10 client.15216 handle_command_reply: tid=0 2024-02-26T04:57:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd948603740 1 client.15216 shutdown 2024-02-26T04:57:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd948603740 1 -- 172.21.15.60:0/2750858263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f72940 msgr2=0x563991fdb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd948603740 1 --2- 172.21.15.60:0/2750858263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f72940 0x563991fdb030 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd938004810 tx=0x7fd93800a6e0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd948603740 1 -- 172.21.15.60:0/2750858263 shutdown_connections 2024-02-26T04:57:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd948603740 1 --2- 172.21.15.60:0/2750858263 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x563991fe7c40 0x563991fe40e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd948603740 1 --2- 172.21.15.60:0/2750858263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563991f77ab0 0x563991fe2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd91effd640 0 client.15216 ms_handle_reset on v2:172.21.15.191:6838/640686517 2024-02-26T04:57:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd948603740 1 --2- 172.21.15.60:0/2750858263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563991f72940 0x563991fdb030 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd948603740 1 --2- 172.21.15.60:0/2750858263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563991f71f50 0x563991fdaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd948603740 1 -- 172.21.15.60:0/2750858263 >> 172.21.15.60:0/2750858263 conn(0x563991f6d4c0 msgr2=0x563991f91330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd948603740 1 -- 172.21.15.60:0/2750858263 shutdown_connections 2024-02-26T04:57:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd948603740 1 -- 172.21.15.60:0/2750858263 wait complete. 2024-02-26T04:57:21.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.385+0000 7fd948603740 20 client.15216 trim_cache size 0 max 16384 2024-02-26T04:57:21.395 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:57:21.395 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "f12e2206-d461-11ee-95c3-87774f69a715", 2024-02-26T04:57:21.395 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 2, 2024-02-26T04:57:21.395 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24472, 2024-02-26T04:57:21.395 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-02-26T04:57:21.395 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-02-26T04:57:21.395 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-02-26T04:57:21.395 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 87.845768765000003, 2024-02-26T04:57:21.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 22, 2024-02-26T04:57:21.395 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 93, 2024-02-26T04:57:21.395 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 89, 2024-02-26T04:57:21.396 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 127.912794448 2024-02-26T04:57:21.396 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:57:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.386+0000 7fd94582d640 1 -- 172.21.15.60:0/346879181 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd9280a20c0 msgr2=0x7fd9280a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.386+0000 7fd94582d640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd9280a20c0 0x7fd9280a4580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd93000cfd0 tx=0x7fd930069e80 comp rx=0 tx=0).stop 2024-02-26T04:57:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.386+0000 7fd94582d640 1 -- 172.21.15.60:0/346879181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd940101c40 msgr2=0x7fd94019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.386+0000 7fd94582d640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd940101c40 0x7fd94019e620 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd93000eaa0 tx=0x7fd93000ef60 comp rx=0 tx=0).stop 2024-02-26T04:57:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.386+0000 7fd94582d640 1 -- 172.21.15.60:0/346879181 shutdown_connections 2024-02-26T04:57:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.386+0000 7fd94582d640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd9280a20c0 0x7fd9280a4580 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.386+0000 7fd94582d640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd940101c40 0x7fd94019e620 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.386+0000 7fd94582d640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd940101280 0x7fd94019bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.386+0000 7fd94582d640 1 --2- 172.21.15.60:0/346879181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd940100090 0x7fd94006a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.386+0000 7fd94582d640 1 -- 172.21.15.60:0/346879181 >> 172.21.15.60:0/346879181 conn(0x7fd9400fb620 msgr2=0x7fd940104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.386+0000 7fd94582d640 1 -- 172.21.15.60:0/346879181 shutdown_connections 2024-02-26T04:57:21.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.386+0000 7fd94582d640 1 -- 172.21.15.60:0/346879181 wait complete. 2024-02-26T04:57:21.400 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-02-26T04:57:21.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-02-26T04:57:21.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-02-26T04:57:21.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-02-26T04:57:21.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-02-26T04:57:21.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-02-26T04:57:21.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-02-26T04:57:21.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-02-26T04:57:21.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:21 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-02-26T04:57:21.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:21 smithi156 ceph-mon[35805]: osdmap e92: 12 total, 12 up, 12 in 2024-02-26T04:57:21.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:21 smithi156 ceph-mon[35805]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-02-26T04:57:21.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:21 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2518942407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:21.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:21 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2827875113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:21.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:21 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/903088421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:21.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-02-26T04:57:21.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-02-26T04:57:21.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-02-26T04:57:21.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-02-26T04:57:21.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-02-26T04:57:21.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-02-26T04:57:21.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-02-26T04:57:21.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:21 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-02-26T04:57:21.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:21 smithi060 ceph-mon[32382]: osdmap e92: 12 total, 12 up, 12 in 2024-02-26T04:57:21.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:21 smithi060 ceph-mon[32382]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-02-26T04:57:21.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:21 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2518942407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:21.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:21 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2827875113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:21.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:21 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/903088421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:21.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.502+0000 7fe2a39aa640 1 -- 172.21.15.60:0/673440811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe29c101c80 msgr2=0x7fe29c104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.502+0000 7fe2a39aa640 1 --2- 172.21.15.60:0/673440811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe29c101c80 0x7fe29c104070 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe298009900 tx=0x7fe29802bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:21.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.502+0000 7fe2a39aa640 1 -- 172.21.15.60:0/673440811 shutdown_connections 2024-02-26T04:57:21.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.502+0000 7fe2a39aa640 1 --2- 172.21.15.60:0/673440811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe29c106ee0 0x7fe29c1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.502+0000 7fe2a39aa640 1 --2- 172.21.15.60:0/673440811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe29c1045b0 0x7fe29c1069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.502+0000 7fe2a39aa640 1 --2- 172.21.15.60:0/673440811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe29c101c80 0x7fe29c104070 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.502+0000 7fe2a39aa640 1 -- 172.21.15.60:0/673440811 >> 172.21.15.60:0/673440811 conn(0x7fe29c0fb630 msgr2=0x7fe29c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.502+0000 7fe2a39aa640 1 -- 172.21.15.60:0/673440811 shutdown_connections 2024-02-26T04:57:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.502+0000 7fe2a39aa640 1 -- 172.21.15.60:0/673440811 wait complete. 2024-02-26T04:57:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.502+0000 7fe2a39aa640 1 Processor -- start 2024-02-26T04:57:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.502+0000 7fe2a39aa640 1 -- start start 2024-02-26T04:57:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a39aa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe29c101c80 0x7fe29c1015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a39aa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe29c1045b0 0x7fe29c0ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a39aa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe29c106ee0 0x7fe29c100170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a39aa640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c1008d0 con 0x7fe29c101c80 2024-02-26T04:57:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a39aa640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c100a40 con 0x7fe29c106ee0 2024-02-26T04:57:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a39aa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c1a1300 con 0x7fe29c1045b0 2024-02-26T04:57:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a1746640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe29c101c80 0x7fe29c1015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a1746640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe29c101c80 0x7fe29c1015f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50230/0 (socket says 172.21.15.60:50230) 2024-02-26T04:57:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a1746640 1 -- 172.21.15.60:0/1376235619 learned_addr learned my addr 172.21.15.60:0/1376235619 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a1f47640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe29c106ee0 0x7fe29c100170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a0f45640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe29c1045b0 0x7fe29c0ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a1746640 1 -- 172.21.15.60:0/1376235619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe29c1045b0 msgr2=0x7fe29c0ffc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a1746640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe29c1045b0 0x7fe29c0ffc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a1746640 1 -- 172.21.15.60:0/1376235619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe29c106ee0 msgr2=0x7fe29c100170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a1746640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe29c106ee0 0x7fe29c100170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a1746640 1 -- 172.21.15.60:0/1376235619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe298009560 con 0x7fe29c101c80 2024-02-26T04:57:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a0f45640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe29c1045b0 0x7fe29c0ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:21.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a1746640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe29c101c80 0x7fe29c1015f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe298036040 tx=0x7fe2980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2927fc640 1 -- 172.21.15.60:0/1376235619 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2980044e0 con 0x7fe29c101c80 2024-02-26T04:57:21.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2927fc640 1 -- 172.21.15.60:0/1376235619 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2980054e0 con 0x7fe29c101c80 2024-02-26T04:57:21.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a39aa640 1 -- 172.21.15.60:0/1376235619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29c1a1550 con 0x7fe29c101c80 2024-02-26T04:57:21.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2927fc640 1 -- 172.21.15.60:0/1376235619 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe298003ec0 con 0x7fe29c101c80 2024-02-26T04:57:21.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.503+0000 7fe2a39aa640 1 -- 172.21.15.60:0/1376235619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c1a1a60 con 0x7fe29c101c80 2024-02-26T04:57:21.511 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-02-26T04:57:21.511 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-02-26T04:57:21.511 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-02-26T04:57:21.511 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-02-26T04:57:21.511 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-02-26T04:57:21.511 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-02-26T04:57:21.511 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-02-26T04:57:21.511 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:21 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-02-26T04:57:21.511 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:21 smithi191 ceph-mon[36352]: osdmap e92: 12 total, 12 up, 12 in 2024-02-26T04:57:21.511 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:21 smithi191 ceph-mon[36352]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-02-26T04:57:21.512 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:21 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2518942407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:21.512 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:21 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2827875113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:21.512 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:21 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/903088421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:21.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.505+0000 7fe2927fc640 1 -- 172.21.15.60:0/1376235619 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe2980035f0 con 0x7fe29c101c80 2024-02-26T04:57:21.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2a6778740 1 Processor -- start 2024-02-26T04:57:21.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2a6778740 1 -- start start 2024-02-26T04:57:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2a6778740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c387bf50 0x5639c387c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2a6778740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 0x5639c3881440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2a6778740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c3881ab0 0x5639c3883f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2a6778740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c3837ca0 con 0x5639c3881ab0 2024-02-26T04:57:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2a6778740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c37eaef0 con 0x5639c387c940 2024-02-26T04:57:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2a6778740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c37e8420 con 0x5639c387bf50 2024-02-26T04:57:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2927fc640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe2880a20c0 0x7fe2880a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2a0f45640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c3881ab0 0x5639c3883f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2a1f47640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe2880a20c0 0x7fe2880a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2a0f45640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c3881ab0 0x5639c3883f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50240/0 (socket says 172.21.15.60:50240) 2024-02-26T04:57:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2a0f45640 1 -- 172.21.15.60:0/4171243538 learned_addr learned my addr 172.21.15.60:0/4171243538 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2927fc640 1 -- 172.21.15.60:0/1376235619 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fe2980f0590 con 0x7fe29c101c80 2024-02-26T04:57:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2a1746640 1 --2- 172.21.15.60:0/4171243538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 0x5639c3881440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.506+0000 7fe2a1f47640 1 --2- 172.21.15.60:0/4171243538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c387bf50 0x5639c387c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.507+0000 7fe2a0f45640 1 -- 172.21.15.60:0/4171243538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c387bf50 msgr2=0x5639c387c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.507+0000 7fe2a0f45640 1 --2- 172.21.15.60:0/4171243538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c387bf50 0x5639c387c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.507+0000 7fe2a0f45640 1 -- 172.21.15.60:0/4171243538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 msgr2=0x5639c3881440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.507+0000 7fe2a0f45640 1 --2- 172.21.15.60:0/4171243538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 0x5639c3881440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.507+0000 7fe2a0f45640 1 -- 172.21.15.60:0/4171243538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639c3798ef0 con 0x5639c3881ab0 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.507+0000 7fe2a1f47640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe2880a20c0 0x7fe2880a4580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe294005fd0 tx=0x7fe294013b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.507+0000 7fe2a0f45640 1 --2- 172.21.15.60:0/4171243538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c3881ab0 0x5639c3883f40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009b70 tx=0x7fe28c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be89432d51108449 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.507+0000 7fe2877fe640 1 -- 172.21.15.60:0/4171243538 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c009e60 con 0x5639c3881ab0 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.507+0000 7fe2877fe640 1 -- 172.21.15.60:0/4171243538 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe28c00ee90 con 0x5639c3881ab0 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.507+0000 7fe2877fe640 1 -- 172.21.15.60:0/4171243538 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0116d0 con 0x5639c3881ab0 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 -- 172.21.15.60:0/4171243538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c3881ab0 msgr2=0x5639c3883f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 --2- 172.21.15.60:0/4171243538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c3881ab0 0x5639c3883f40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009b70 tx=0x7fe28c00e6c0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 -- 172.21.15.60:0/4171243538 shutdown_connections 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 --2- 172.21.15.60:0/4171243538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c3881ab0 0x5639c3883f40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 --2- 172.21.15.60:0/4171243538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 0x5639c3881440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 --2- 172.21.15.60:0/4171243538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c387bf50 0x5639c387c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 -- 172.21.15.60:0/4171243538 >> 172.21.15.60:0/4171243538 conn(0x5639c38774c0 msgr2=0x5639c38778c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 -- 172.21.15.60:0/4171243538 shutdown_connections 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 -- 172.21.15.60:0/4171243538 wait complete. 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 -- start start 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 Processor -- start 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c387bf50 0x5639c38860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 0x5639c3886610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c3881ab0 0x5639c388d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c37e3c00 con 0x5639c3881ab0 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c3788d00 con 0x5639c387c940 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c34b7310 con 0x5639c387bf50 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.508+0000 7fe2a6778740 10 client.0 _ll_register_callbacks cb 0x5639c3790d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe2a1f47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c3881ab0 0x5639c388d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe2a1f47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c3881ab0 0x5639c388d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50252/0 (socket says 172.21.15.60:50252) 2024-02-26T04:57:21.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe2a1f47640 1 -- 172.21.15.60:0/392031216 learned_addr learned my addr 172.21.15.60:0/392031216 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe2a0f45640 1 --2- 172.21.15.60:0/392031216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 0x5639c3886610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe2a1746640 1 --2- 172.21.15.60:0/392031216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c387bf50 0x5639c38860d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe2a1f47640 1 -- 172.21.15.60:0/392031216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c387bf50 msgr2=0x5639c38860d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe2a1f47640 1 --2- 172.21.15.60:0/392031216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c387bf50 0x5639c38860d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe2a1f47640 1 -- 172.21.15.60:0/392031216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 msgr2=0x5639c3886610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe2a1f47640 1 --2- 172.21.15.60:0/392031216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 0x5639c3886610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe2a1f47640 1 -- 172.21.15.60:0/392031216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28c009820 con 0x5639c3881ab0 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe2a1f47640 1 --2- 172.21.15.60:0/392031216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c3881ab0 0x5639c388d650 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe294011ce0 tx=0x7fe294019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e45b2e38aef654 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe284ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe284ff9640 1 -- 172.21.15.60:0/392031216 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294011d10 con 0x5639c3881ab0 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe284ff9640 1 -- 172.21.15.60:0/392031216 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fe294011e70 con 0x5639c3881ab0 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe2a6778740 1 -- 172.21.15.60:0/392031216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639c34e36c0 con 0x5639c3881ab0 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe284ff9640 1 -- 172.21.15.60:0/392031216 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe294028410 con 0x5639c3881ab0 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe2a6778740 10 client.15228 fetch_fsmap learned FSMap version 22 2024-02-26T04:57:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe2a6778740 1 -- 172.21.15.60:0/392031216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5639c37e2280 con 0x5639c3881ab0 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.509+0000 7fe284ff9640 1 -- 172.21.15.60:0/392031216 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5639c34e36c0 con 0x5639c3881ab0 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.510+0000 7fe284ff9640 1 -- 172.21.15.60:0/392031216 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe29402a4d0 con 0x5639c3881ab0 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.510+0000 7fe2a6778740 10 client.15228 fetch_fsmap finished waiting for FSMap version 22 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.510+0000 7fe2a6778740 10 client.15228 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.510+0000 7fe2a6778740 20 client.15228 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.510+0000 7fe2a6778740 1 --2- 172.21.15.60:0/392031216 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5639c3892ab0 0x5639c3891cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.510+0000 7fe2a6778740 4 client.15228 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.510+0000 7fe2a6778740 1 -- 172.21.15.60:0/392031216 --> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639c3534270 con 0x5639c3892ab0 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.510+0000 7fe2a1746640 1 --2- 172.21.15.60:0/392031216 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5639c3892ab0 0x5639c3891cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.510+0000 7fe284ff9640 10 client.15228 ms_handle_connect on v2:172.21.15.156:6836/707575792 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.510+0000 7fe2a1746640 1 --2- 172.21.15.60:0/392031216 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5639c3892ab0 0x5639c3891cd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d78d230b87ce995 server_cookie=87abb40e14c78cc1 in_seq=0 out_seq=0 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.511+0000 7fe284ff9640 1 -- 172.21.15.60:0/392031216 <== mds.1 v2:172.21.15.156:6836/707575792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639c3534270 con 0x5639c3892ab0 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.511+0000 7fe284ff9640 10 client.15228 handle_command_reply: tid=0 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.511+0000 7fe2a6778740 1 client.15228 shutdown 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.511+0000 7fe2a6778740 1 -- 172.21.15.60:0/392031216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c3881ab0 msgr2=0x5639c388d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.511+0000 7fe2a6778740 1 --2- 172.21.15.60:0/392031216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c3881ab0 0x5639c388d650 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe294011ce0 tx=0x7fe294019000 comp rx=0 tx=0).stop 2024-02-26T04:57:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.511+0000 7fe2a6778740 1 -- 172.21.15.60:0/392031216 shutdown_connections 2024-02-26T04:57:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.511+0000 7fe2a6778740 1 --2- 172.21.15.60:0/392031216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c3881ab0 0x5639c388d650 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.511+0000 7fe2a6778740 1 --2- 172.21.15.60:0/392031216 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5639c3892ab0 0x5639c3891cd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.511+0000 7fe2a6778740 1 --2- 172.21.15.60:0/392031216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 0x5639c3886610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.511+0000 7fe2a6778740 1 --2- 172.21.15.60:0/392031216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c387bf50 0x5639c38860d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.511+0000 7fe284ff9640 0 client.15228 ms_handle_reset on v2:172.21.15.156:6836/707575792 2024-02-26T04:57:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.511+0000 7fe2a6778740 1 -- 172.21.15.60:0/392031216 >> 172.21.15.60:0/392031216 conn(0x5639c38774c0 msgr2=0x5639c38767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.511+0000 7fe2a6778740 1 -- 172.21.15.60:0/392031216 shutdown_connections 2024-02-26T04:57:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.511+0000 7fe2a6778740 1 -- 172.21.15.60:0/392031216 wait complete. 2024-02-26T04:57:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.512+0000 7fe2a6778740 20 client.15228 trim_cache size 0 max 16384 2024-02-26T04:57:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.527+0000 7fe2a6778740 1 Processor -- start 2024-02-26T04:57:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.527+0000 7fe2a6778740 1 -- start start 2024-02-26T04:57:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.527+0000 7fe2a6778740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c38774c0 0x5639c387d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.527+0000 7fe2a6778740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c387bf50 0x5639c387d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.527+0000 7fe2a6778740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 0x5639c388d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.527+0000 7fe2a6778740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c3769460 con 0x5639c387bf50 2024-02-26T04:57:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.527+0000 7fe2a6778740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c3534270 con 0x5639c387c940 2024-02-26T04:57:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.527+0000 7fe2a6778740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c34e36c0 con 0x5639c38774c0 2024-02-26T04:57:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.527+0000 7fe2a0f45640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c387bf50 0x5639c387d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.527+0000 7fe2a0f45640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c387bf50 0x5639c387d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50258/0 (socket says 172.21.15.60:50258) 2024-02-26T04:57:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.527+0000 7fe2a0f45640 1 -- 172.21.15.60:0/3645817689 learned_addr learned my addr 172.21.15.60:0/3645817689 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.528+0000 7fe2a1f47640 1 --2- 172.21.15.60:0/3645817689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 0x5639c388d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.528+0000 7fe2a0f45640 1 -- 172.21.15.60:0/3645817689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c38774c0 msgr2=0x5639c387d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.528+0000 7fe2a0f45640 1 --2- 172.21.15.60:0/3645817689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c38774c0 0x5639c387d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.528+0000 7fe2a0f45640 1 -- 172.21.15.60:0/3645817689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 msgr2=0x5639c388d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.528+0000 7fe2a0f45640 1 --2- 172.21.15.60:0/3645817689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 0x5639c388d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.528+0000 7fe2a0f45640 1 -- 172.21.15.60:0/3645817689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639c377e9f0 con 0x5639c387bf50 2024-02-26T04:57:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.528+0000 7fe2a1f47640 1 --2- 172.21.15.60:0/3645817689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 0x5639c388d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.528+0000 7fe2a0f45640 1 --2- 172.21.15.60:0/3645817689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c387bf50 0x5639c387d570 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009b10 tx=0x7fe28c009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbad6484fa91eb25 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.528+0000 7fe286ffd640 1 -- 172.21.15.60:0/3645817689 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c021040 con 0x5639c387bf50 2024-02-26T04:57:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.528+0000 7fe286ffd640 1 -- 172.21.15.60:0/3645817689 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe28c005040 con 0x5639c387bf50 2024-02-26T04:57:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.528+0000 7fe286ffd640 1 -- 172.21.15.60:0/3645817689 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c004a80 con 0x5639c387bf50 2024-02-26T04:57:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.528+0000 7fe2a6778740 1 -- 172.21.15.60:0/3645817689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c387bf50 msgr2=0x5639c387d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.528+0000 7fe2a6778740 1 --2- 172.21.15.60:0/3645817689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c387bf50 0x5639c387d570 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009b10 tx=0x7fe28c009d10 comp rx=0 tx=0).stop 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 -- 172.21.15.60:0/3645817689 shutdown_connections 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 --2- 172.21.15.60:0/3645817689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387c940 0x5639c388d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 --2- 172.21.15.60:0/3645817689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c387bf50 0x5639c387d570 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 --2- 172.21.15.60:0/3645817689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c38774c0 0x5639c387d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 -- 172.21.15.60:0/3645817689 >> 172.21.15.60:0/3645817689 conn(0x5639c3877c10 msgr2=0x5639c389af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 -- 172.21.15.60:0/3645817689 shutdown_connections 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 -- 172.21.15.60:0/3645817689 wait complete. 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 -- start start 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 Processor -- start 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c38774c0 0x5639c388a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387bf50 0x5639c388a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c387c940 0x5639c38e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c388d810 con 0x5639c387c940 2024-02-26T04:57:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c388d980 con 0x5639c387bf50 2024-02-26T04:57:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c38e8bf0 con 0x5639c38774c0 2024-02-26T04:57:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a6778740 10 client.0 _ll_register_callbacks cb 0x7fe28c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a1f47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c387c940 0x5639c38e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a1f47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c387c940 0x5639c38e86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50264/0 (socket says 172.21.15.60:50264) 2024-02-26T04:57:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.529+0000 7fe2a1f47640 1 -- 172.21.15.60:0/11045002 learned_addr learned my addr 172.21.15.60:0/11045002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a0f45640 1 --2- 172.21.15.60:0/11045002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387bf50 0x5639c388a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a1f47640 1 -- 172.21.15.60:0/11045002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c38774c0 msgr2=0x5639c388a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a1f47640 1 --2- 172.21.15.60:0/11045002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c38774c0 0x5639c388a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a1f47640 1 -- 172.21.15.60:0/11045002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387bf50 msgr2=0x5639c388a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a1f47640 1 --2- 172.21.15.60:0/11045002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387bf50 0x5639c388a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a1f47640 1 -- 172.21.15.60:0/11045002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639c3798ef0 con 0x5639c387c940 2024-02-26T04:57:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a0f45640 1 --2- 172.21.15.60:0/11045002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387bf50 0x5639c388a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a1f47640 1 --2- 172.21.15.60:0/11045002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c387c940 0x5639c38e86b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe294011cb0 tx=0x7fe294012cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a158b54188679c20 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe284ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe284ff9640 1 -- 172.21.15.60:0/11045002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294009e60 con 0x5639c387c940 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe284ff9640 1 -- 172.21.15.60:0/11045002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fe294011d10 con 0x5639c387c940 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a6778740 1 -- 172.21.15.60:0/11045002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe28c00ee90 con 0x5639c387c940 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe284ff9640 1 -- 172.21.15.60:0/11045002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe294011ed0 con 0x5639c387c940 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a6778740 10 client.15234 fetch_fsmap learned FSMap version 22 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a6778740 1 -- 172.21.15.60:0/11045002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5639c37a4750 con 0x5639c387c940 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe284ff9640 1 -- 172.21.15.60:0/11045002 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe28c00ee90 con 0x5639c387c940 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe284ff9640 1 -- 172.21.15.60:0/11045002 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe294004c00 con 0x5639c387c940 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a6778740 10 client.15234 fetch_fsmap finished waiting for FSMap version 22 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a6778740 10 client.15234 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a6778740 20 client.15234 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a6778740 1 --2- 172.21.15.60:0/11045002 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5639c3881ab0 0x5639c38ea1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a6778740 4 client.15234 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T04:57:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.530+0000 7fe2a6778740 1 -- 172.21.15.60:0/11045002 --> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5639c3520610 con 0x5639c3881ab0 2024-02-26T04:57:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.531+0000 7fe2a1746640 1 --2- 172.21.15.60:0/11045002 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5639c3881ab0 0x5639c38ea1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.531+0000 7fe2a1746640 1 --2- 172.21.15.60:0/11045002 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5639c3881ab0 0x5639c38ea1c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=448d9e1fc8db2085 server_cookie=4c8d5a90f635b609 in_seq=0 out_seq=0 2024-02-26T04:57:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.531+0000 7fe284ff9640 10 client.15234 ms_handle_connect on v2:172.21.15.156:6836/707575792 2024-02-26T04:57:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.531+0000 7fe284ff9640 1 -- 172.21.15.60:0/11045002 <== mds.1 v2:172.21.15.156:6836/707575792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5639c3520610 con 0x5639c3881ab0 2024-02-26T04:57:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.531+0000 7fe284ff9640 10 client.15234 handle_command_reply: tid=0 2024-02-26T04:57:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.531+0000 7fe2a6778740 1 client.15234 shutdown 2024-02-26T04:57:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.531+0000 7fe2a6778740 1 -- 172.21.15.60:0/11045002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c387c940 msgr2=0x5639c38e86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.531+0000 7fe2a6778740 1 --2- 172.21.15.60:0/11045002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c387c940 0x5639c38e86b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe294011cb0 tx=0x7fe294012cb0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a6778740 1 -- 172.21.15.60:0/11045002 shutdown_connections 2024-02-26T04:57:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a6778740 1 --2- 172.21.15.60:0/11045002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639c387c940 0x5639c38e86b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a6778740 1 --2- 172.21.15.60:0/11045002 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5639c3881ab0 0x5639c38ea1c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a6778740 1 --2- 172.21.15.60:0/11045002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639c387bf50 0x5639c388a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a6778740 1 --2- 172.21.15.60:0/11045002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5639c38774c0 0x5639c388a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe284ff9640 0 client.15234 ms_handle_reset on v2:172.21.15.156:6836/707575792 2024-02-26T04:57:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a6778740 1 -- 172.21.15.60:0/11045002 >> 172.21.15.60:0/11045002 conn(0x5639c3877c10 msgr2=0x5639c387d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a6778740 1 -- 172.21.15.60:0/11045002 shutdown_connections 2024-02-26T04:57:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a6778740 1 -- 172.21.15.60:0/11045002 wait complete. 2024-02-26T04:57:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a6778740 20 client.15234 trim_cache size 0 max 16384 2024-02-26T04:57:21.541 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:57:21.541 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "f12e2206-d461-11ee-95c3-87774f69a715", 2024-02-26T04:57:21.541 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 1, 2024-02-26T04:57:21.541 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24476, 2024-02-26T04:57:21.541 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-02-26T04:57:21.541 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-02-26T04:57:21.542 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-02-26T04:57:21.542 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 88.998920798, 2024-02-26T04:57:21.542 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 22, 2024-02-26T04:57:21.542 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 93, 2024-02-26T04:57:21.542 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 89, 2024-02-26T04:57:21.542 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 130.83858385900001 2024-02-26T04:57:21.542 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:57:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a39aa640 1 -- 172.21.15.60:0/1376235619 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe2880a20c0 msgr2=0x7fe2880a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a39aa640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe2880a20c0 0x7fe2880a4580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe294005fd0 tx=0x7fe294013b40 comp rx=0 tx=0).stop 2024-02-26T04:57:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a39aa640 1 -- 172.21.15.60:0/1376235619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe29c101c80 msgr2=0x7fe29c1015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a39aa640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe29c101c80 0x7fe29c1015f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe298036040 tx=0x7fe2980042f0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a39aa640 1 -- 172.21.15.60:0/1376235619 shutdown_connections 2024-02-26T04:57:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a39aa640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe2880a20c0 0x7fe2880a4580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a39aa640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe29c106ee0 0x7fe29c100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a39aa640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe29c1045b0 0x7fe29c0ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a39aa640 1 --2- 172.21.15.60:0/1376235619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe29c101c80 0x7fe29c1015f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a39aa640 1 -- 172.21.15.60:0/1376235619 >> 172.21.15.60:0/1376235619 conn(0x7fe29c0fb630 msgr2=0x7fe29c1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a39aa640 1 -- 172.21.15.60:0/1376235619 shutdown_connections 2024-02-26T04:57:21.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.532+0000 7fe2a39aa640 1 -- 172.21.15.60:0/1376235619 wait complete. 2024-02-26T04:57:21.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-26T04:57:21.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.646+0000 7ff51fde0640 1 -- 172.21.15.60:0/3466631718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff518106f00 msgr2=0x7ff5181092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.646+0000 7ff51fde0640 1 --2- 172.21.15.60:0/3466631718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff518106f00 0x7ff5181092f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff510009900 tx=0x7ff51002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:21.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.646+0000 7ff51fde0640 1 -- 172.21.15.60:0/3466631718 shutdown_connections 2024-02-26T04:57:21.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.646+0000 7ff51fde0640 1 --2- 172.21.15.60:0/3466631718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff518106f00 0x7ff5181092f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.646+0000 7ff51fde0640 1 --2- 172.21.15.60:0/3466631718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5181045d0 0x7ff5181069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.646+0000 7ff51fde0640 1 --2- 172.21.15.60:0/3466631718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff518101ca0 0x7ff518104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.646+0000 7ff51fde0640 1 -- 172.21.15.60:0/3466631718 >> 172.21.15.60:0/3466631718 conn(0x7ff5180fb610 msgr2=0x7ff5180fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.647+0000 7ff51fde0640 1 -- 172.21.15.60:0/3466631718 shutdown_connections 2024-02-26T04:57:21.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.647+0000 7ff51fde0640 1 -- 172.21.15.60:0/3466631718 wait complete. 2024-02-26T04:57:21.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.647+0000 7ff51fde0640 1 Processor -- start 2024-02-26T04:57:21.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.647+0000 7ff51fde0640 1 -- start start 2024-02-26T04:57:21.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.647+0000 7ff51fde0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff518101ca0 0x7ff518197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.647+0000 7ff51fde0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5181045d0 0x7ff518197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.647+0000 7ff51fde0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff518106f00 0x7ff51819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.647+0000 7ff51fde0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51819ec40 con 0x7ff5181045d0 2024-02-26T04:57:21.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.647+0000 7ff51fde0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51819edb0 con 0x7ff518106f00 2024-02-26T04:57:21.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.647+0000 7ff51fde0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51819ef70 con 0x7ff518101ca0 2024-02-26T04:57:21.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.647+0000 7ff51d37b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5181045d0 0x7ff518197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.647+0000 7ff51d37b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5181045d0 0x7ff518197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50280/0 (socket says 172.21.15.60:50280) 2024-02-26T04:57:21.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.647+0000 7ff51d37b640 1 -- 172.21.15.60:0/2027429871 learned_addr learned my addr 172.21.15.60:0/2027429871 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.648+0000 7ff51d37b640 1 -- 172.21.15.60:0/2027429871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff518101ca0 msgr2=0x7ff518197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.648+0000 7ff51d37b640 1 --2- 172.21.15.60:0/2027429871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff518101ca0 0x7ff518197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.648+0000 7ff51d37b640 1 -- 172.21.15.60:0/2027429871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff518106f00 msgr2=0x7ff51819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.648+0000 7ff51d37b640 1 --2- 172.21.15.60:0/2027429871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff518106f00 0x7ff51819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.648+0000 7ff51d37b640 1 -- 172.21.15.60:0/2027429871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff510009560 con 0x7ff5181045d0 2024-02-26T04:57:21.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.648+0000 7ff51d37b640 1 --2- 172.21.15.60:0/2027429871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5181045d0 0x7ff518197690 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff50800c970 tx=0x7ff50800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.648+0000 7ff50effd640 1 -- 172.21.15.60:0/2027429871 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508004440 con 0x7ff5181045d0 2024-02-26T04:57:21.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.648+0000 7ff50effd640 1 -- 172.21.15.60:0/2027429871 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff508007d40 con 0x7ff5181045d0 2024-02-26T04:57:21.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.648+0000 7ff51fde0640 1 -- 172.21.15.60:0/2027429871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51819f250 con 0x7ff5181045d0 2024-02-26T04:57:21.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.648+0000 7ff50effd640 1 -- 172.21.15.60:0/2027429871 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50800f450 con 0x7ff5181045d0 2024-02-26T04:57:21.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.648+0000 7ff51fde0640 1 -- 172.21.15.60:0/2027429871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51819f7b0 con 0x7ff5181045d0 2024-02-26T04:57:21.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.651+0000 7ff50effd640 1 -- 172.21.15.60:0/2027429871 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff508007860 con 0x7ff5181045d0 2024-02-26T04:57:21.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.651+0000 7ff522bb1740 1 Processor -- start 2024-02-26T04:57:21.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.651+0000 7ff522bb1740 1 -- start start 2024-02-26T04:57:21.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.651+0000 7ff522bb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b2f50 0x5579293b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.651+0000 7ff522bb1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b3940 0x5579293b8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.651+0000 7ff522bb1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b8ab0 0x5579293baf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.651+0000 7ff522bb1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792936eca0 con 0x5579293b3940 2024-02-26T04:57:21.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.651+0000 7ff522bb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557929321ef0 con 0x5579293b2f50 2024-02-26T04:57:21.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.651+0000 7ff522bb1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792931f420 con 0x5579293b8ab0 2024-02-26T04:57:21.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff51db7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b3940 0x5579293b8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff51db7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b3940 0x5579293b8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50282/0 (socket says 172.21.15.60:50282) 2024-02-26T04:57:21.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff51db7c640 1 -- 172.21.15.60:0/977035398 learned_addr learned my addr 172.21.15.60:0/977035398 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff51e37d640 1 --2- 172.21.15.60:0/977035398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b2f50 0x5579293b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff51d37b640 1 --2- 172.21.15.60:0/977035398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b8ab0 0x5579293baf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff50effd640 1 --2- 172.21.15.60:0/2027429871 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff5040a2140 0x7ff5040a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff51e37d640 1 --2- 172.21.15.60:0/2027429871 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff5040a2140 0x7ff5040a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff51db7c640 1 -- 172.21.15.60:0/977035398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b8ab0 msgr2=0x5579293baf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff51db7c640 1 --2- 172.21.15.60:0/977035398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b8ab0 0x5579293baf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff51db7c640 1 -- 172.21.15.60:0/977035398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b2f50 msgr2=0x5579293b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff51db7c640 1 --2- 172.21.15.60:0/977035398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b2f50 0x5579293b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff51db7c640 1 -- 172.21.15.60:0/977035398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579292cfef0 con 0x5579293b3940 2024-02-26T04:57:21.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff50effd640 1 -- 172.21.15.60:0/2027429871 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff5080ce210 con 0x7ff5181045d0 2024-02-26T04:57:21.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff51db7c640 1 --2- 172.21.15.60:0/977035398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b3940 0x5579293b8440 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff514009f10 tx=0x7ff51400d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c0ba5307111fc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff51e37d640 1 --2- 172.21.15.60:0/2027429871 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff5040a2140 0x7ff5040a4600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff51002c6a0 tx=0x7ff510005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff503fff640 1 -- 172.21.15.60:0/977035398 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514010660 con 0x5579293b3940 2024-02-26T04:57:21.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff503fff640 1 -- 172.21.15.60:0/977035398 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5140099f0 con 0x5579293b3940 2024-02-26T04:57:21.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.652+0000 7ff503fff640 1 -- 172.21.15.60:0/977035398 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51400fcf0 con 0x5579293b3940 2024-02-26T04:57:21.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.653+0000 7ff522bb1740 1 -- 172.21.15.60:0/977035398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b3940 msgr2=0x5579293b8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.653+0000 7ff522bb1740 1 --2- 172.21.15.60:0/977035398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b3940 0x5579293b8440 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff514009f10 tx=0x7ff51400d6b0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.653+0000 7ff522bb1740 1 -- 172.21.15.60:0/977035398 shutdown_connections 2024-02-26T04:57:21.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.653+0000 7ff522bb1740 1 --2- 172.21.15.60:0/977035398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b8ab0 0x5579293baf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.653+0000 7ff522bb1740 1 --2- 172.21.15.60:0/977035398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b3940 0x5579293b8440 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.653+0000 7ff522bb1740 1 --2- 172.21.15.60:0/977035398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b2f50 0x5579293b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.653+0000 7ff522bb1740 1 -- 172.21.15.60:0/977035398 >> 172.21.15.60:0/977035398 conn(0x5579293ae4c0 msgr2=0x5579293ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.653+0000 7ff522bb1740 1 -- 172.21.15.60:0/977035398 shutdown_connections 2024-02-26T04:57:21.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.653+0000 7ff522bb1740 1 -- 172.21.15.60:0/977035398 wait complete. 2024-02-26T04:57:21.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff522bb1740 1 -- start start 2024-02-26T04:57:21.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff522bb1740 1 Processor -- start 2024-02-26T04:57:21.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff522bb1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b2f50 0x5579293bd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff522bb1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b3940 0x5579293bd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff522bb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b8ab0 0x5579293c4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff522bb1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792931ac00 con 0x5579293b2f50 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff522bb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579292bfd00 con 0x5579293b8ab0 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff522bb1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557928fee310 con 0x5579293b3940 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff522bb1740 10 client.0 _ll_register_callbacks cb 0x5579292c7d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff51db7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b2f50 0x5579293bd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff51db7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b2f50 0x5579293bd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50296/0 (socket says 172.21.15.60:50296) 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff51db7c640 1 -- 172.21.15.60:0/361014751 learned_addr learned my addr 172.21.15.60:0/361014751 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff51d37b640 1 --2- 172.21.15.60:0/361014751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b3940 0x5579293bd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff51e37d640 1 --2- 172.21.15.60:0/361014751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b8ab0 0x5579293c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff51db7c640 1 -- 172.21.15.60:0/361014751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b3940 msgr2=0x5579293bd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.654+0000 7ff51db7c640 1 --2- 172.21.15.60:0/361014751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b3940 0x5579293bd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff51db7c640 1 -- 172.21.15.60:0/361014751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b8ab0 msgr2=0x5579293c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff51db7c640 1 --2- 172.21.15.60:0/361014751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b8ab0 0x5579293c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff51db7c640 1 -- 172.21.15.60:0/361014751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff514009b70 con 0x5579293b2f50 2024-02-26T04:57:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff51e37d640 1 --2- 172.21.15.60:0/361014751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b8ab0 0x5579293c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff51db7c640 1 --2- 172.21.15.60:0/361014751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b2f50 0x5579293bd1b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff514000f80 tx=0x7ff51400f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fbdb73701c4363 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff5017fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff5017fa640 1 -- 172.21.15.60:0/361014751 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51401f070 con 0x5579293b2f50 2024-02-26T04:57:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff5017fa640 1 -- 172.21.15.60:0/361014751 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff51400fcc0 con 0x5579293b2f50 2024-02-26T04:57:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff522bb1740 1 -- 172.21.15.60:0/361014751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55792906b270 con 0x5579293b2f50 2024-02-26T04:57:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff5017fa640 1 -- 172.21.15.60:0/361014751 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff51400fe80 con 0x5579293b2f50 2024-02-26T04:57:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff522bb1740 10 client.15246 fetch_fsmap learned FSMap version 22 2024-02-26T04:57:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff522bb1740 1 -- 172.21.15.60:0/361014751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x557929319280 con 0x5579293b2f50 2024-02-26T04:57:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff5017fa640 1 -- 172.21.15.60:0/361014751 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55792906b270 con 0x5579293b2f50 2024-02-26T04:57:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff5017fa640 1 -- 172.21.15.60:0/361014751 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff514010460 con 0x5579293b2f50 2024-02-26T04:57:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff522bb1740 10 client.15246 fetch_fsmap finished waiting for FSMap version 22 2024-02-26T04:57:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff522bb1740 10 client.15246 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T04:57:21.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff522bb1740 20 client.15246 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:21.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff522bb1740 1 --2- 172.21.15.60:0/361014751 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5579293c8cd0 0x5579293cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff522bb1740 4 client.15246 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:57:21.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.655+0000 7ff522bb1740 1 -- 172.21.15.60:0/361014751 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557929057610 con 0x5579293c8cd0 2024-02-26T04:57:21.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.656+0000 7ff51d37b640 1 --2- 172.21.15.60:0/361014751 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5579293c8cd0 0x5579293cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.656+0000 7ff51d37b640 1 --2- 172.21.15.60:0/361014751 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5579293c8cd0 0x5579293cdd40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d1b1f0c4abccbcf server_cookie=800348ea2f45207e in_seq=0 out_seq=0 2024-02-26T04:57:21.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.656+0000 7ff5017fa640 10 client.15246 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:21.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.657+0000 7ff5017fa640 1 -- 172.21.15.60:0/361014751 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557929057610 con 0x5579293c8cd0 2024-02-26T04:57:21.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.657+0000 7ff5017fa640 10 client.15246 handle_command_reply: tid=0 2024-02-26T04:57:21.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.657+0000 7ff522bb1740 1 client.15246 shutdown 2024-02-26T04:57:21.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.657+0000 7ff522bb1740 1 -- 172.21.15.60:0/361014751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b2f50 msgr2=0x5579293bd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.657+0000 7ff522bb1740 1 --2- 172.21.15.60:0/361014751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b2f50 0x5579293bd1b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff514000f80 tx=0x7ff51400f650 comp rx=0 tx=0).stop 2024-02-26T04:57:21.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.657+0000 7ff522bb1740 1 -- 172.21.15.60:0/361014751 shutdown_connections 2024-02-26T04:57:21.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.657+0000 7ff522bb1740 1 --2- 172.21.15.60:0/361014751 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5579293c8cd0 0x5579293cdd40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.657+0000 7ff522bb1740 1 --2- 172.21.15.60:0/361014751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b8ab0 0x5579293c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.657+0000 7ff522bb1740 1 --2- 172.21.15.60:0/361014751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b3940 0x5579293bd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.657+0000 7ff5017fa640 0 client.15246 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:21.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.657+0000 7ff522bb1740 1 --2- 172.21.15.60:0/361014751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b2f50 0x5579293bd1b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.657+0000 7ff522bb1740 1 -- 172.21.15.60:0/361014751 >> 172.21.15.60:0/361014751 conn(0x5579293ae4c0 msgr2=0x5579293ad7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.657+0000 7ff522bb1740 1 -- 172.21.15.60:0/361014751 shutdown_connections 2024-02-26T04:57:21.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.657+0000 7ff522bb1740 1 -- 172.21.15.60:0/361014751 wait complete. 2024-02-26T04:57:21.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.658+0000 7ff522bb1740 20 client.15246 trim_cache size 0 max 16384 2024-02-26T04:57:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.673+0000 7ff522bb1740 1 Processor -- start 2024-02-26T04:57:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.673+0000 7ff522bb1740 1 -- start start 2024-02-26T04:57:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.673+0000 7ff522bb1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b2f50 0x5579293b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.673+0000 7ff522bb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b3940 0x5579293c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.673+0000 7ff522bb1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b8ab0 0x5579293acf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.673+0000 7ff522bb1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579292a0460 con 0x5579293b8ab0 2024-02-26T04:57:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.673+0000 7ff522bb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579293cb8a0 con 0x5579293b3940 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.673+0000 7ff522bb1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792906b270 con 0x5579293b2f50 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.674+0000 7ff51e37d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b8ab0 0x5579293acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.674+0000 7ff51e37d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b8ab0 0x5579293acf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50300/0 (socket says 172.21.15.60:50300) 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.674+0000 7ff51e37d640 1 -- 172.21.15.60:0/1632961652 learned_addr learned my addr 172.21.15.60:0/1632961652 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.674+0000 7ff51d37b640 1 --2- 172.21.15.60:0/1632961652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b3940 0x5579293c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.674+0000 7ff51e37d640 1 -- 172.21.15.60:0/1632961652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b2f50 msgr2=0x5579293b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.674+0000 7ff51e37d640 1 --2- 172.21.15.60:0/1632961652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b2f50 0x5579293b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.674+0000 7ff51e37d640 1 -- 172.21.15.60:0/1632961652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b3940 msgr2=0x5579293c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.674+0000 7ff51e37d640 1 --2- 172.21.15.60:0/1632961652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b3940 0x5579293c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.674+0000 7ff51e37d640 1 -- 172.21.15.60:0/1632961652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579292b59f0 con 0x5579293b8ab0 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.674+0000 7ff51e37d640 1 --2- 172.21.15.60:0/1632961652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b8ab0 0x5579293acf10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff510009900 tx=0x7ff510043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5731896b8da51b9c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.674+0000 7ff5037fe640 1 -- 172.21.15.60:0/1632961652 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510048050 con 0x5579293b8ab0 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.674+0000 7ff5037fe640 1 -- 172.21.15.60:0/1632961652 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff510049040 con 0x5579293b8ab0 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.674+0000 7ff5037fe640 1 -- 172.21.15.60:0/1632961652 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510003a40 con 0x5579293b8ab0 2024-02-26T04:57:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 -- 172.21.15.60:0/1632961652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b8ab0 msgr2=0x5579293acf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 --2- 172.21.15.60:0/1632961652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b8ab0 0x5579293acf10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff510009900 tx=0x7ff510043000 comp rx=0 tx=0).stop 2024-02-26T04:57:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 -- 172.21.15.60:0/1632961652 shutdown_connections 2024-02-26T04:57:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 --2- 172.21.15.60:0/1632961652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b8ab0 0x5579293acf10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 --2- 172.21.15.60:0/1632961652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b3940 0x5579293c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 --2- 172.21.15.60:0/1632961652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b2f50 0x5579293b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 -- 172.21.15.60:0/1632961652 >> 172.21.15.60:0/1632961652 conn(0x5579293ae4c0 msgr2=0x5579293d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 -- 172.21.15.60:0/1632961652 shutdown_connections 2024-02-26T04:57:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 -- 172.21.15.60:0/1632961652 wait complete. 2024-02-26T04:57:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 -- start start 2024-02-26T04:57:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 Processor -- start 2024-02-26T04:57:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b2f50 0x5579293c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b3940 0x5579293c2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b8ab0 0x5579293c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579293c3570 con 0x5579293b2f50 2024-02-26T04:57:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579293c36e0 con 0x5579293b8ab0 2024-02-26T04:57:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.675+0000 7ff522bb1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579293c38a0 con 0x5579293b3940 2024-02-26T04:57:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff522bb1740 10 client.0 _ll_register_callbacks cb 0x7ff5140096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff51db7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b2f50 0x5579293c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff51db7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b2f50 0x5579293c1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50302/0 (socket says 172.21.15.60:50302) 2024-02-26T04:57:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff51db7c640 1 -- 172.21.15.60:0/1316663488 learned_addr learned my addr 172.21.15.60:0/1316663488 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff51e37d640 1 --2- 172.21.15.60:0/1316663488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b8ab0 0x5579293c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff51db7c640 1 -- 172.21.15.60:0/1316663488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b3940 msgr2=0x5579293c2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff51db7c640 1 --2- 172.21.15.60:0/1316663488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b3940 0x5579293c2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff51db7c640 1 -- 172.21.15.60:0/1316663488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b8ab0 msgr2=0x5579293c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff51db7c640 1 --2- 172.21.15.60:0/1316663488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b8ab0 0x5579293c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff51db7c640 1 -- 172.21.15.60:0/1316663488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff518109eb0 con 0x5579293b2f50 2024-02-26T04:57:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff51e37d640 1 --2- 172.21.15.60:0/1316663488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b8ab0 0x5579293c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff51db7c640 1 --2- 172.21.15.60:0/1316663488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b2f50 0x5579293c1ae0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff51400e010 tx=0x7ff514010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8a41367d5777e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff5017fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff5017fa640 1 -- 172.21.15.60:0/1316663488 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51401f040 con 0x5579293b2f50 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff522bb1740 1 -- 172.21.15.60:0/1316663488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579293ca8f0 con 0x5579293b2f50 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff5017fa640 1 -- 172.21.15.60:0/1316663488 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff514010a80 con 0x5579293b2f50 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff5017fa640 1 -- 172.21.15.60:0/1316663488 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff51400dbb0 con 0x5579293b2f50 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff522bb1740 10 client.15252 fetch_fsmap learned FSMap version 22 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.676+0000 7ff522bb1740 1 -- 172.21.15.60:0/1316663488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5579292db750 con 0x5579293b2f50 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.677+0000 7ff5017fa640 1 -- 172.21.15.60:0/1316663488 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5579293ca8f0 con 0x5579293b2f50 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.677+0000 7ff5017fa640 1 -- 172.21.15.60:0/1316663488 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff514003a90 con 0x5579293b2f50 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.677+0000 7ff522bb1740 10 client.15252 fetch_fsmap finished waiting for FSMap version 22 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.677+0000 7ff522bb1740 10 client.15252 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.677+0000 7ff522bb1740 20 client.15252 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.677+0000 7ff522bb1740 1 --2- 172.21.15.60:0/1316663488 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x557929425590 0x557929421a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:21.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.677+0000 7ff522bb1740 4 client.15252 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.677+0000 7ff522bb1740 1 -- 172.21.15.60:0/1316663488 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5579293ca290 con 0x557929425590 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.677+0000 7ff51d37b640 1 --2- 172.21.15.60:0/1316663488 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x557929425590 0x557929421a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.677+0000 7ff5017fa640 10 client.15252 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.677+0000 7ff51d37b640 1 --2- 172.21.15.60:0/1316663488 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x557929425590 0x557929421a10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5001dcc7948540d5 server_cookie=8a490b8f5a0a1f5f in_seq=0 out_seq=0 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff5017fa640 1 -- 172.21.15.60:0/1316663488 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5579293ca290 con 0x557929425590 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff5017fa640 10 client.15252 handle_command_reply: tid=0 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff522bb1740 1 client.15252 shutdown 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff522bb1740 1 -- 172.21.15.60:0/1316663488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b2f50 msgr2=0x5579293c1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff522bb1740 1 --2- 172.21.15.60:0/1316663488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b2f50 0x5579293c1ae0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff51400e010 tx=0x7ff514010710 comp rx=0 tx=0).stop 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff522bb1740 1 -- 172.21.15.60:0/1316663488 shutdown_connections 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff522bb1740 1 --2- 172.21.15.60:0/1316663488 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x557929425590 0x557929421a10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff522bb1740 1 --2- 172.21.15.60:0/1316663488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5579293b8ab0 0x5579293c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff522bb1740 1 --2- 172.21.15.60:0/1316663488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5579293b3940 0x5579293c2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff5017fa640 0 client.15252 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff522bb1740 1 --2- 172.21.15.60:0/1316663488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5579293b2f50 0x5579293c1ae0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff522bb1740 1 -- 172.21.15.60:0/1316663488 >> 172.21.15.60:0/1316663488 conn(0x5579293ae4c0 msgr2=0x5579293d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff522bb1740 1 -- 172.21.15.60:0/1316663488 shutdown_connections 2024-02-26T04:57:21.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff522bb1740 1 -- 172.21.15.60:0/1316663488 wait complete. 2024-02-26T04:57:21.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff522bb1740 20 client.15252 trim_cache size 0 max 16384 2024-02-26T04:57:21.687 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:57:21.687 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "f12e2206-d461-11ee-95c3-87774f69a715", 2024-02-26T04:57:21.687 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-02-26T04:57:21.687 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24524, 2024-02-26T04:57:21.687 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-02-26T04:57:21.687 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-02-26T04:57:21.688 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-02-26T04:57:21.688 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 113.49333320700001, 2024-02-26T04:57:21.688 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 22, 2024-02-26T04:57:21.688 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 93, 2024-02-26T04:57:21.688 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 89, 2024-02-26T04:57:21.688 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 119.72885047 2024-02-26T04:57:21.688 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:57:21.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff51fde0640 1 -- 172.21.15.60:0/2027429871 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff5040a2140 msgr2=0x7ff5040a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff51fde0640 1 --2- 172.21.15.60:0/2027429871 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff5040a2140 0x7ff5040a4600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff51002c6a0 tx=0x7ff510005fd0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff51fde0640 1 -- 172.21.15.60:0/2027429871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5181045d0 msgr2=0x7ff518197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:21.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff51fde0640 1 --2- 172.21.15.60:0/2027429871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5181045d0 0x7ff518197690 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff50800c970 tx=0x7ff50800ce30 comp rx=0 tx=0).stop 2024-02-26T04:57:21.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff51fde0640 1 -- 172.21.15.60:0/2027429871 shutdown_connections 2024-02-26T04:57:21.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff51fde0640 1 --2- 172.21.15.60:0/2027429871 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff5040a2140 0x7ff5040a4600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff51fde0640 1 --2- 172.21.15.60:0/2027429871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff518106f00 0x7ff51819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff51fde0640 1 --2- 172.21.15.60:0/2027429871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5181045d0 0x7ff518197690 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff51fde0640 1 --2- 172.21.15.60:0/2027429871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff518101ca0 0x7ff518197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:21.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.678+0000 7ff51fde0640 1 -- 172.21.15.60:0/2027429871 >> 172.21.15.60:0/2027429871 conn(0x7ff5180fb610 msgr2=0x7ff5181025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:21.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.679+0000 7ff51fde0640 1 -- 172.21.15.60:0/2027429871 shutdown_connections 2024-02-26T04:57:21.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:21.679+0000 7ff51fde0640 1 -- 172.21.15.60:0/2027429871 wait complete. 2024-02-26T04:57:21.692 INFO:tasks.cephadm:Setup complete, yielding 2024-02-26T04:57:21.692 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-26T04:57:21.701 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T04:57:21.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- bash -c 'ceph orch status' 2024-02-26T04:57:21.943 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:57:22.329 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:22 smithi060 ceph-mon[32382]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-02-26T04:57:22.329 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:22 smithi060 ceph-mon[32382]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-26T04:57:22.329 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:22 smithi060 ceph-mon[32382]: osdmap e93: 12 total, 12 up, 12 in 2024-02-26T04:57:22.329 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:22 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4116962152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:22.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:22 smithi156 ceph-mon[35805]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-02-26T04:57:22.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:22 smithi156 ceph-mon[35805]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-26T04:57:22.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:22 smithi156 ceph-mon[35805]: osdmap e93: 12 total, 12 up, 12 in 2024-02-26T04:57:22.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:22 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4116962152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:22.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:22 smithi191 ceph-mon[36352]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-02-26T04:57:22.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:22 smithi191 ceph-mon[36352]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-26T04:57:22.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:22 smithi191 ceph-mon[36352]: osdmap e93: 12 total, 12 up, 12 in 2024-02-26T04:57:22.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:22 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4116962152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:23.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.835+0000 7f4919064700 1 -- 172.21.15.60:0/1294750681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4914101ca0 msgr2=0x7f49141020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:23.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.835+0000 7f4919064700 1 --2- 172.21.15.60:0/1294750681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4914101ca0 0x7f49141020c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f490c00b0d0 tx=0x7f490c00b3e0 comp rx=0 tx=0).stop 2024-02-26T04:57:23.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.835+0000 7f4919064700 1 -- 172.21.15.60:0/1294750681 shutdown_connections 2024-02-26T04:57:23.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.835+0000 7f4919064700 1 --2- 172.21.15.60:0/1294750681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4914103840 0x7f491410a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:23.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.835+0000 7f4919064700 1 --2- 172.21.15.60:0/1294750681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4914101ca0 0x7f49141020c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:23.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.835+0000 7f4919064700 1 --2- 172.21.15.60:0/1294750681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4914102ea0 0x7f4914103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:23.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.835+0000 7f4919064700 1 -- 172.21.15.60:0/1294750681 >> 172.21.15.60:0/1294750681 conn(0x7f49140fd220 msgr2=0x7f49140ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:23.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.835+0000 7f4919064700 1 -- 172.21.15.60:0/1294750681 shutdown_connections 2024-02-26T04:57:23.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.835+0000 7f4919064700 1 -- 172.21.15.60:0/1294750681 wait complete. 2024-02-26T04:57:23.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f4919064700 1 Processor -- start 2024-02-26T04:57:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f4919064700 1 -- start start 2024-02-26T04:57:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f4919064700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4914101ca0 0x7f4914075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f4919064700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4914102ea0 0x7f4914074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f4919064700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4914103840 0x7f49140745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f4919064700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4914074d20 con 0x7f4914101ca0 2024-02-26T04:57:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f4919064700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4914074e90 con 0x7f4914103840 2024-02-26T04:57:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f4919064700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49141a1e70 con 0x7f4914102ea0 2024-02-26T04:57:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f4912d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4914101ca0 0x7f4914075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f4912d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4914101ca0 0x7f4914075a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50328/0 (socket says 172.21.15.60:50328) 2024-02-26T04:57:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f4912d9d700 1 -- 172.21.15.60:0/2981481049 learned_addr learned my addr 172.21.15.60:0/2981481049 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f491259c700 1 --2- 172.21.15.60:0/2981481049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4914102ea0 0x7f4914074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f491359e700 1 --2- 172.21.15.60:0/2981481049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4914103840 0x7f49140745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f4912d9d700 1 -- 172.21.15.60:0/2981481049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4914102ea0 msgr2=0x7f4914074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:23.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.836+0000 7f4912d9d700 1 --2- 172.21.15.60:0/2981481049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4914102ea0 0x7f4914074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:23.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.837+0000 7f4912d9d700 1 -- 172.21.15.60:0/2981481049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4914103840 msgr2=0x7f49140745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:23.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.837+0000 7f4912d9d700 1 --2- 172.21.15.60:0/2981481049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4914103840 0x7f49140745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:23.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.837+0000 7f4912d9d700 1 -- 172.21.15.60:0/2981481049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c009d00 con 0x7f4914101ca0 2024-02-26T04:57:23.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.837+0000 7f4912d9d700 1 --2- 172.21.15.60:0/2981481049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4914101ca0 0x7f4914075a10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f490c000f80 tx=0x7f490c009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:23.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.837+0000 7f490bfff700 1 -- 172.21.15.60:0/2981481049 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c00e040 con 0x7f4914101ca0 2024-02-26T04:57:23.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.837+0000 7f4919064700 1 -- 172.21.15.60:0/2981481049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49141a20f0 con 0x7f4914101ca0 2024-02-26T04:57:23.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.837+0000 7f490bfff700 1 -- 172.21.15.60:0/2981481049 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f490c01dd70 con 0x7f4914101ca0 2024-02-26T04:57:23.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.837+0000 7f490bfff700 1 -- 172.21.15.60:0/2981481049 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c01be80 con 0x7f4914101ca0 2024-02-26T04:57:23.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.837+0000 7f4919064700 1 -- 172.21.15.60:0/2981481049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49141a25b0 con 0x7f4914101ca0 2024-02-26T04:57:23.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.838+0000 7f4919064700 1 -- 172.21.15.60:0/2981481049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4914109100 con 0x7f4914101ca0 2024-02-26T04:57:23.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.839+0000 7f490bfff700 1 -- 172.21.15.60:0/2981481049 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f490c01b570 con 0x7f4914101ca0 2024-02-26T04:57:23.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.840+0000 7f490bfff700 1 --2- 172.21.15.60:0/2981481049 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49000a2150 0x7f49000a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:23.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.840+0000 7f491259c700 1 --2- 172.21.15.60:0/2981481049 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49000a2150 0x7f49000a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:23.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.840+0000 7f490bfff700 1 -- 172.21.15.60:0/2981481049 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f490c0cfc80 con 0x7f4914101ca0 2024-02-26T04:57:23.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.840+0000 7f491259c700 1 --2- 172.21.15.60:0/2981481049 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49000a2150 0x7f49000a4610 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005950 tx=0x7f48fc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:23.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:23.844+0000 7f490bfff700 1 -- 172.21.15.60:0/2981481049 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f490c0988a0 con 0x7f4914101ca0 2024-02-26T04:57:24.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:24.063+0000 7f4919064700 1 -- 172.21.15.60:0/2981481049 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f491402d7d0 con 0x7f49000a2150 2024-02-26T04:57:24.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:24.064+0000 7f490bfff700 1 -- 172.21.15.60:0/2981481049 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f491402d7d0 con 0x7f49000a2150 2024-02-26T04:57:24.066 INFO:teuthology.orchestra.run.smithi060.stdout:Backend: cephadm 2024-02-26T04:57:24.066 INFO:teuthology.orchestra.run.smithi060.stdout:Available: Yes 2024-02-26T04:57:24.066 INFO:teuthology.orchestra.run.smithi060.stdout:Paused: No 2024-02-26T04:57:24.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:24.066+0000 7f4919064700 1 -- 172.21.15.60:0/2981481049 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49000a2150 msgr2=0x7f49000a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:24.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:24.066+0000 7f4919064700 1 --2- 172.21.15.60:0/2981481049 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49000a2150 0x7f49000a4610 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005950 tx=0x7f48fc00b410 comp rx=0 tx=0).stop 2024-02-26T04:57:24.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:24.066+0000 7f4919064700 1 -- 172.21.15.60:0/2981481049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4914101ca0 msgr2=0x7f4914075a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:24.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:24.066+0000 7f4919064700 1 --2- 172.21.15.60:0/2981481049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4914101ca0 0x7f4914075a10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f490c000f80 tx=0x7f490c009380 comp rx=0 tx=0).stop 2024-02-26T04:57:24.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:24.066+0000 7f4919064700 1 -- 172.21.15.60:0/2981481049 shutdown_connections 2024-02-26T04:57:24.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:24.066+0000 7f4919064700 1 --2- 172.21.15.60:0/2981481049 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f49000a2150 0x7f49000a4610 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:24.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:24.066+0000 7f4919064700 1 --2- 172.21.15.60:0/2981481049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4914103840 0x7f49140745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:24.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:24.066+0000 7f4919064700 1 --2- 172.21.15.60:0/2981481049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4914101ca0 0x7f4914075a10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:24.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:24.066+0000 7f4919064700 1 --2- 172.21.15.60:0/2981481049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4914102ea0 0x7f4914074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:24.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:24.066+0000 7f4919064700 1 -- 172.21.15.60:0/2981481049 >> 172.21.15.60:0/2981481049 conn(0x7f49140fd220 msgr2=0x7f4914106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:24.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:24.066+0000 7f4919064700 1 -- 172.21.15.60:0/2981481049 shutdown_connections 2024-02-26T04:57:24.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:24.066+0000 7f4919064700 1 -- 172.21.15.60:0/2981481049 wait complete. 2024-02-26T04:57:24.443 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:24 smithi060 ceph-mon[32382]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-02-26T04:57:24.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:24 smithi191 ceph-mon[36352]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-02-26T04:57:24.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:24 smithi156 ceph-mon[35805]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-02-26T04:57:24.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-26T04:57:25.135 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:57:25.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:25 smithi060 ceph-mon[32382]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:25.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:25 smithi191 ceph-mon[36352]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:25.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:25 smithi156 ceph-mon[35805]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:26.462 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:26 smithi060 ceph-mon[32382]: pgmap v356: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-02-26T04:57:26.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:26 smithi191 ceph-mon[36352]: pgmap v356: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-02-26T04:57:26.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:26 smithi156 ceph-mon[35805]: pgmap v356: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-02-26T04:57:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.042+0000 7ffa48912700 1 -- 172.21.15.60:0/4122184590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa44102730 msgr2=0x7ffa44108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.042+0000 7ffa48912700 1 --2- 172.21.15.60:0/4122184590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa44102730 0x7ffa44108fe0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00b0d0 tx=0x7ffa3c00b3e0 comp rx=0 tx=0).stop 2024-02-26T04:57:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.042+0000 7ffa48912700 1 -- 172.21.15.60:0/4122184590 shutdown_connections 2024-02-26T04:57:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.042+0000 7ffa48912700 1 --2- 172.21.15.60:0/4122184590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa44102730 0x7ffa44108fe0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.042+0000 7ffa48912700 1 --2- 172.21.15.60:0/4122184590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa44100b90 0x7ffa44100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.042+0000 7ffa48912700 1 --2- 172.21.15.60:0/4122184590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa44101d90 0x7ffa441021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.042+0000 7ffa48912700 1 -- 172.21.15.60:0/4122184590 >> 172.21.15.60:0/4122184590 conn(0x7ffa440fc130 msgr2=0x7ffa440fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.042+0000 7ffa48912700 1 -- 172.21.15.60:0/4122184590 shutdown_connections 2024-02-26T04:57:27.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.042+0000 7ffa48912700 1 -- 172.21.15.60:0/4122184590 wait complete. 2024-02-26T04:57:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.043+0000 7ffa48912700 1 Processor -- start 2024-02-26T04:57:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.043+0000 7ffa48912700 1 -- start start 2024-02-26T04:57:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.043+0000 7ffa48912700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa44100b90 0x7ffa44196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.043+0000 7ffa48912700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa44101d90 0x7ffa441971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.043+0000 7ffa48912700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa44102730 0x7ffa4419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.043+0000 7ffa48912700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4419e800 con 0x7ffa44100b90 2024-02-26T04:57:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.043+0000 7ffa48912700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4419e940 con 0x7ffa44102730 2024-02-26T04:57:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.043+0000 7ffa48912700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4419eb00 con 0x7ffa44101d90 2024-02-26T04:57:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.043+0000 7ffa4259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa44100b90 0x7ffa44196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.043+0000 7ffa4259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa44100b90 0x7ffa44196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50356/0 (socket says 172.21.15.60:50356) 2024-02-26T04:57:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.043+0000 7ffa4259c700 1 -- 172.21.15.60:0/4262999199 learned_addr learned my addr 172.21.15.60:0/4262999199 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.043+0000 7ffa41d9b700 1 --2- 172.21.15.60:0/4262999199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa44101d90 0x7ffa441971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.043+0000 7ffa42d9d700 1 --2- 172.21.15.60:0/4262999199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa44102730 0x7ffa4419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.044+0000 7ffa4259c700 1 -- 172.21.15.60:0/4262999199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa44101d90 msgr2=0x7ffa441971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.044+0000 7ffa4259c700 1 --2- 172.21.15.60:0/4262999199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa44101d90 0x7ffa441971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.044+0000 7ffa4259c700 1 -- 172.21.15.60:0/4262999199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa44102730 msgr2=0x7ffa4419e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.044+0000 7ffa4259c700 1 --2- 172.21.15.60:0/4262999199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa44102730 0x7ffa4419e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.044+0000 7ffa4259c700 1 -- 172.21.15.60:0/4262999199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c009d00 con 0x7ffa44100b90 2024-02-26T04:57:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.044+0000 7ffa4259c700 1 --2- 172.21.15.60:0/4262999199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa44100b90 0x7ffa44196c80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400c970 tx=0x7ffa3400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.044+0000 7ffa3b7fe700 1 -- 172.21.15.60:0/4262999199 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34004220 con 0x7ffa44100b90 2024-02-26T04:57:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.044+0000 7ffa3b7fe700 1 -- 172.21.15.60:0/4262999199 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa34007d80 con 0x7ffa44100b90 2024-02-26T04:57:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.044+0000 7ffa48912700 1 -- 172.21.15.60:0/4262999199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4419ede0 con 0x7ffa44100b90 2024-02-26T04:57:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.044+0000 7ffa3b7fe700 1 -- 172.21.15.60:0/4262999199 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34003d00 con 0x7ffa44100b90 2024-02-26T04:57:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.044+0000 7ffa48912700 1 -- 172.21.15.60:0/4262999199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4419f280 con 0x7ffa44100b90 2024-02-26T04:57:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.045+0000 7ffa48912700 1 -- 172.21.15.60:0/4262999199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa44068930 con 0x7ffa44100b90 2024-02-26T04:57:27.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.046+0000 7ffa3b7fe700 1 -- 172.21.15.60:0/4262999199 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ffa340078a0 con 0x7ffa44100b90 2024-02-26T04:57:27.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.047+0000 7ffa3b7fe700 1 --2- 172.21.15.60:0/4262999199 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa300a2150 0x7ffa300a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:27.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.047+0000 7ffa41d9b700 1 --2- 172.21.15.60:0/4262999199 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa300a2150 0x7ffa300a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:27.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.047+0000 7ffa3b7fe700 1 -- 172.21.15.60:0/4262999199 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ffa340ce0d0 con 0x7ffa44100b90 2024-02-26T04:57:27.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.047+0000 7ffa41d9b700 1 --2- 172.21.15.60:0/4262999199 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa300a2150 0x7ffa300a4610 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffa44198190 tx=0x7ffa2c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:27.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.050+0000 7ffa3b7fe700 1 -- 172.21.15.60:0/4262999199 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffa34096c00 con 0x7ffa44100b90 2024-02-26T04:57:27.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.265+0000 7ffa48912700 1 -- 172.21.15.60:0/4262999199 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa4402d950 con 0x7ffa300a2150 2024-02-26T04:57:27.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.278+0000 7ffa3b7fe700 1 -- 172.21.15.60:0/4262999199 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4092 (secure 0 0 0) 0x7ffa4402d950 con 0x7ffa300a2150 2024-02-26T04:57:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-26T04:57:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:mds.a smithi060 running (2m) 24s ago 2m 13.9M - 19.0.0-1514-g25296a8f a4519cb49bed b8df58cbd1ae 2024-02-26T04:57:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:mds.b smithi156 running (2m) 59s ago 2m 28.2M - 19.0.0-1514-g25296a8f a4519cb49bed 4b36e180c947 2024-02-26T04:57:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:mds.c smithi191 running (2m) 59s ago 2m 18.4M - 19.0.0-1514-g25296a8f a4519cb49bed 6288967d3c30 2024-02-26T04:57:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:mds.d smithi060 running (2m) 24s ago 2m 16.3M - 19.0.0-1514-g25296a8f a4519cb49bed 5caf4150ff03 2024-02-26T04:57:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:mds.e smithi156 running (2m) 59s ago 2m 17.1M - 19.0.0-1514-g25296a8f a4519cb49bed c6fb7835692b 2024-02-26T04:57:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:mds.f smithi191 running (2m) 59s ago 2m 18.1M - 19.0.0-1514-g25296a8f a4519cb49bed 6fac6472ec87 2024-02-26T04:57:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:mds.g smithi060 running (2m) 24s ago 2m 15.2M - 19.0.0-1514-g25296a8f a4519cb49bed eebc6ae4d6a2 2024-02-26T04:57:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:mds.h smithi156 running (2m) 59s ago 2m 18.5M - 19.0.0-1514-g25296a8f a4519cb49bed d4a9ec314a77 2024-02-26T04:57:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:mds.i smithi191 running (2m) 59s ago 2m 19.2M - 19.0.0-1514-g25296a8f a4519cb49bed e344c5b8d29b 2024-02-26T04:57:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:mds.j smithi060 running (2m) 24s ago 2m 13.7M - 19.0.0-1514-g25296a8f a4519cb49bed e1af7e57f0a7 2024-02-26T04:57:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:mds.k smithi156 running (2m) 59s ago 2m 13.0M - 19.0.0-1514-g25296a8f a4519cb49bed a5348b564836 2024-02-26T04:57:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:mds.l smithi191 running (2m) 59s ago 2m 13.0M - 19.0.0-1514-g25296a8f a4519cb49bed 1639a982b18c 2024-02-26T04:57:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:mgr.x smithi060 *:9283,8765 running (9m) 24s ago 9m 512M - 19.0.0-1514-g25296a8f a4519cb49bed 81ee7493b7f6 2024-02-26T04:57:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:mgr.y smithi156 *:8443,8765 running (7m) 59s ago 7m 452M - 19.0.0-1514-g25296a8f a4519cb49bed 5d25ce3df1b7 2024-02-26T04:57:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:mgr.z smithi191 *:8443,8765 running (7m) 59s ago 7m 452M - 19.0.0-1514-g25296a8f a4519cb49bed 4bee1c9c1140 2024-02-26T04:57:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:mon.a smithi060 running (9m) 24s ago 10m 50.2M 2048M 19.0.0-1514-g25296a8f a4519cb49bed 5b57dca8a033 2024-02-26T04:57:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:mon.b smithi156 running (8m) 59s ago 8m 39.6M 2048M 19.0.0-1514-g25296a8f a4519cb49bed f61d55a72bbf 2024-02-26T04:57:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:mon.c smithi191 running (8m) 59s ago 8m 42.7M 2048M 19.0.0-1514-g25296a8f a4519cb49bed 462f964553c9 2024-02-26T04:57:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 smithi060 running (7m) 24s ago 7m 56.3M 4273M 19.0.0-1514-g25296a8f a4519cb49bed 4164e8fbb764 2024-02-26T04:57:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 smithi156 running (7m) 59s ago 7m 46.7M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 93efe0a029c6 2024-02-26T04:57:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 smithi191 running (6m) 59s ago 6m 57.0M 4270M 19.0.0-1514-g25296a8f a4519cb49bed c3eb7d483b68 2024-02-26T04:57:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 smithi060 running (6m) 24s ago 6m 57.9M 4273M 19.0.0-1514-g25296a8f a4519cb49bed c647fc46e512 2024-02-26T04:57:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 smithi156 running (5m) 59s ago 5m 42.1M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 7638ce1b098b 2024-02-26T04:57:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 smithi191 running (5m) 59s ago 5m 44.9M 4270M 19.0.0-1514-g25296a8f a4519cb49bed a7b3fb8367d8 2024-02-26T04:57:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 smithi060 running (5m) 24s ago 5m 53.1M 4273M 19.0.0-1514-g25296a8f a4519cb49bed c0e8320004ad 2024-02-26T04:57:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 smithi156 running (4m) 59s ago 4m 54.2M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 2b20654fe2ea 2024-02-26T04:57:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8 smithi191 running (4m) 59s ago 4m 53.1M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 0977986a3ab1 2024-02-26T04:57:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9 smithi060 running (3m) 24s ago 3m 52.4M 4273M 19.0.0-1514-g25296a8f a4519cb49bed 9678427e6b8c 2024-02-26T04:57:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10 smithi156 running (3m) 59s ago 3m 42.7M 4270M 19.0.0-1514-g25296a8f a4519cb49bed 6809ea1368ca 2024-02-26T04:57:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11 smithi191 running (2m) 59s ago 2m 41.3M 4270M 19.0.0-1514-g25296a8f a4519cb49bed ee7fffe40575 2024-02-26T04:57:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.280+0000 7ffa48912700 1 -- 172.21.15.60:0/4262999199 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa300a2150 msgr2=0x7ffa300a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.280+0000 7ffa48912700 1 --2- 172.21.15.60:0/4262999199 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa300a2150 0x7ffa300a4610 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffa44198190 tx=0x7ffa2c009190 comp rx=0 tx=0).stop 2024-02-26T04:57:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.280+0000 7ffa48912700 1 -- 172.21.15.60:0/4262999199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa44100b90 msgr2=0x7ffa44196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.280+0000 7ffa48912700 1 --2- 172.21.15.60:0/4262999199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa44100b90 0x7ffa44196c80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400c970 tx=0x7ffa3400cc80 comp rx=0 tx=0).stop 2024-02-26T04:57:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.280+0000 7ffa48912700 1 -- 172.21.15.60:0/4262999199 shutdown_connections 2024-02-26T04:57:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.280+0000 7ffa48912700 1 --2- 172.21.15.60:0/4262999199 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ffa300a2150 0x7ffa300a4610 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.280+0000 7ffa48912700 1 --2- 172.21.15.60:0/4262999199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa44102730 0x7ffa4419e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:27.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.280+0000 7ffa48912700 1 --2- 172.21.15.60:0/4262999199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa44100b90 0x7ffa44196c80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:27.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.280+0000 7ffa48912700 1 --2- 172.21.15.60:0/4262999199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa44101d90 0x7ffa441971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:27.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.280+0000 7ffa48912700 1 -- 172.21.15.60:0/4262999199 >> 172.21.15.60:0/4262999199 conn(0x7ffa440fc130 msgr2=0x7ffa44104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:27.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.280+0000 7ffa48912700 1 -- 172.21.15.60:0/4262999199 shutdown_connections 2024-02-26T04:57:27.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:27.281+0000 7ffa48912700 1 -- 172.21.15.60:0/4262999199 wait complete. 2024-02-26T04:57:27.294 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:27 smithi060 ceph-mon[32382]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-26T04:57:27.294 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:27 smithi060 ceph-mon[32382]: Cluster is now healthy 2024-02-26T04:57:27.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:27 smithi191 ceph-mon[36352]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-26T04:57:27.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:27 smithi191 ceph-mon[36352]: Cluster is now healthy 2024-02-26T04:57:27.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:27 smithi156 ceph-mon[35805]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-26T04:57:27.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:27 smithi156 ceph-mon[35805]: Cluster is now healthy 2024-02-26T04:57:28.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-26T04:57:28.274 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:57:28.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:28 smithi191 ceph-mon[36352]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 7 op/s; 72 B/s, 1 keys/s, 0 objects/s recovering 2024-02-26T04:57:28.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:28 smithi191 ceph-mon[36352]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:28.578 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:28 smithi060 ceph-mon[32382]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 7 op/s; 72 B/s, 1 keys/s, 0 objects/s recovering 2024-02-26T04:57:28.579 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:28 smithi060 ceph-mon[32382]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:28.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:28 smithi156 ceph-mon[35805]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 7 op/s; 72 B/s, 1 keys/s, 0 objects/s recovering 2024-02-26T04:57:28.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:28 smithi156 ceph-mon[35805]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:30.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.050+0000 7f256f4e8700 1 -- 172.21.15.60:0/3373376194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2568100ba0 msgr2=0x7f2568100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:30.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.050+0000 7f256f4e8700 1 --2- 172.21.15.60:0/3373376194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2568100ba0 0x7f2568100fc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f255c0099c0 tx=0x7f255c009cd0 comp rx=0 tx=0).stop 2024-02-26T04:57:30.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.050+0000 7f256f4e8700 1 -- 172.21.15.60:0/3373376194 shutdown_connections 2024-02-26T04:57:30.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.050+0000 7f256f4e8700 1 --2- 172.21.15.60:0/3373376194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2568102740 0x7f2568108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:30.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.050+0000 7f256f4e8700 1 --2- 172.21.15.60:0/3373376194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2568100ba0 0x7f2568100fc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:30.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.050+0000 7f256f4e8700 1 --2- 172.21.15.60:0/3373376194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2568101da0 0x7f2568102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:30.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.050+0000 7f256f4e8700 1 -- 172.21.15.60:0/3373376194 >> 172.21.15.60:0/3373376194 conn(0x7f25680fc140 msgr2=0x7f25680fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:30.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.050+0000 7f256f4e8700 1 -- 172.21.15.60:0/3373376194 shutdown_connections 2024-02-26T04:57:30.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.050+0000 7f256f4e8700 1 -- 172.21.15.60:0/3373376194 wait complete. 2024-02-26T04:57:30.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.051+0000 7f256f4e8700 1 Processor -- start 2024-02-26T04:57:30.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.051+0000 7f256f4e8700 1 -- start start 2024-02-26T04:57:30.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.051+0000 7f256f4e8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2568100ba0 0x7f2568104ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:30.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.051+0000 7f256f4e8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2568101da0 0x7f2568103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:30.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.051+0000 7f256f4e8700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2568102740 0x7f25681036c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:30.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.051+0000 7f256f4e8700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2568103e60 con 0x7f2568102740 2024-02-26T04:57:30.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.051+0000 7f256f4e8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2568103fd0 con 0x7f2568100ba0 2024-02-26T04:57:30.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.051+0000 7f256f4e8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25681a1d90 con 0x7f2568101da0 2024-02-26T04:57:30.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.051+0000 7f256da85700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2568102740 0x7f25681036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:30.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.051+0000 7f256da85700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2568102740 0x7f25681036c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50324/0 (socket says 172.21.15.60:50324) 2024-02-26T04:57:30.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.051+0000 7f256da85700 1 -- 172.21.15.60:0/2214176126 learned_addr learned my addr 172.21.15.60:0/2214176126 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:30.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.051+0000 7f256ca83700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2568101da0 0x7f2568103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:30.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.051+0000 7f256d284700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2568100ba0 0x7f2568104ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.052+0000 7f256da85700 1 -- 172.21.15.60:0/2214176126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2568101da0 msgr2=0x7f2568103180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.052+0000 7f256da85700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2568101da0 0x7f2568103180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.052+0000 7f256da85700 1 -- 172.21.15.60:0/2214176126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2568100ba0 msgr2=0x7f2568104ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.052+0000 7f256da85700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2568100ba0 0x7f2568104ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.052+0000 7f256da85700 1 -- 172.21.15.60:0/2214176126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c0095f0 con 0x7f2568102740 2024-02-26T04:57:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.052+0000 7f256d284700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2568100ba0 0x7f2568104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.052+0000 7f256ca83700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2568101da0 0x7f2568103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.052+0000 7f256da85700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2568102740 0x7f25681036c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2560007ee0 tx=0x7f256000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.052+0000 7f255a7fc700 1 -- 172.21.15.60:0/2214176126 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256000ddd0 con 0x7f2568102740 2024-02-26T04:57:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.052+0000 7f256f4e8700 1 -- 172.21.15.60:0/2214176126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25681a1fc0 con 0x7f2568102740 2024-02-26T04:57:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.052+0000 7f255a7fc700 1 -- 172.21.15.60:0/2214176126 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f256000f040 con 0x7f2568102740 2024-02-26T04:57:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.052+0000 7f255a7fc700 1 -- 172.21.15.60:0/2214176126 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560015400 con 0x7f2568102740 2024-02-26T04:57:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.052+0000 7f256f4e8700 1 -- 172.21.15.60:0/2214176126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25681a2440 con 0x7f2568102740 2024-02-26T04:57:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.053+0000 7f256f4e8700 1 -- 172.21.15.60:0/2214176126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2568068960 con 0x7f2568102740 2024-02-26T04:57:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.054+0000 7f255a7fc700 1 -- 172.21.15.60:0/2214176126 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2560003a00 con 0x7f2568102740 2024-02-26T04:57:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.055+0000 7f255a7fc700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f25540a2150 0x7f25540a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.055+0000 7f256d284700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f25540a2150 0x7f25540a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.055+0000 7f255a7fc700 1 -- 172.21.15.60:0/2214176126 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f25600cd9f0 con 0x7f2568102740 2024-02-26T04:57:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.055+0000 7f256d284700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f25540a2150 0x7f25540a4610 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f255c000c00 tx=0x7f255c003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:30.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.059+0000 7f255a7fc700 1 -- 172.21.15.60:0/2214176126 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f256004fd20 con 0x7f2568102740 2024-02-26T04:57:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.272+0000 7f256f4e8700 1 -- 172.21.15.60:0/2214176126 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f256806a030 con 0x7f25540a2150 2024-02-26T04:57:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.278+0000 7f255a7fc700 1 -- 172.21.15.60:0/2214176126 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f256806a030 con 0x7f25540a2150 2024-02-26T04:57:30.280 INFO:teuthology.orchestra.run.smithi060.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-26T04:57:30.281 INFO:teuthology.orchestra.run.smithi060.stdout:mds.all 12/12 62s ago 2m smithi060=a;smithi060=d;smithi060=g;smithi060=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-02-26T04:57:30.281 INFO:teuthology.orchestra.run.smithi060.stdout:mgr 3/3 62s ago 8m smithi060=x;smithi156=y;smithi191=z;count:3 2024-02-26T04:57:30.281 INFO:teuthology.orchestra.run.smithi060.stdout:mon 3/3 62s ago 8m smithi060:172.21.15.60=a;smithi156:172.21.15.156=b;smithi191:172.21.15.191=c;count:3 2024-02-26T04:57:30.281 INFO:teuthology.orchestra.run.smithi060.stdout:osd 12 62s ago - 2024-02-26T04:57:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.280+0000 7f256f4e8700 1 -- 172.21.15.60:0/2214176126 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f25540a2150 msgr2=0x7f25540a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.280+0000 7f256f4e8700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f25540a2150 0x7f25540a4610 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f255c000c00 tx=0x7f255c003860 comp rx=0 tx=0).stop 2024-02-26T04:57:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.280+0000 7f256f4e8700 1 -- 172.21.15.60:0/2214176126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2568102740 msgr2=0x7f25681036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.280+0000 7f256f4e8700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2568102740 0x7f25681036c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2560007ee0 tx=0x7f256000d3b0 comp rx=0 tx=0).stop 2024-02-26T04:57:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.280+0000 7f256f4e8700 1 -- 172.21.15.60:0/2214176126 shutdown_connections 2024-02-26T04:57:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.280+0000 7f256f4e8700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2568102740 0x7f25681036c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.280+0000 7f256f4e8700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f25540a2150 0x7f25540a4610 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.280+0000 7f256f4e8700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2568100ba0 0x7f2568104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.280+0000 7f256f4e8700 1 --2- 172.21.15.60:0/2214176126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2568101da0 0x7f2568103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.280+0000 7f256f4e8700 1 -- 172.21.15.60:0/2214176126 >> 172.21.15.60:0/2214176126 conn(0x7f25680fc140 msgr2=0x7f2568069920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.280+0000 7f256f4e8700 1 -- 172.21.15.60:0/2214176126 shutdown_connections 2024-02-26T04:57:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:30.280+0000 7f256f4e8700 1 -- 172.21.15.60:0/2214176126 wait complete. 2024-02-26T04:57:30.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:30 smithi191 ceph-mon[36352]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 0 B/s wr, 5 op/s; 71 B/s, 1 keys/s, 0 objects/s recovering 2024-02-26T04:57:30.653 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:30 smithi060 ceph-mon[32382]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 0 B/s wr, 5 op/s; 71 B/s, 1 keys/s, 0 objects/s recovering 2024-02-26T04:57:30.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:30 smithi156 ceph-mon[35805]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 0 B/s wr, 5 op/s; 71 B/s, 1 keys/s, 0 objects/s recovering 2024-02-26T04:57:31.013 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-26T04:57:31.265 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:57:31.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:31 smithi191 ceph-mon[36352]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:31.549 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:31 smithi060 ceph-mon[32382]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:31.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:31 smithi156 ceph-mon[35805]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:32.439 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:32 smithi060 ceph-mon[32382]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 58 B/s, 0 keys/s, 0 objects/s recovering 2024-02-26T04:57:32.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:32 smithi191 ceph-mon[36352]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 58 B/s, 0 keys/s, 0 objects/s recovering 2024-02-26T04:57:32.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:32 smithi156 ceph-mon[35805]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 58 B/s, 0 keys/s, 0 objects/s recovering 2024-02-26T04:57:32.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.979+0000 7f0f533e1700 1 -- 172.21.15.60:0/4010247517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f4c101540 msgr2=0x7f0f4c1019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:32.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.979+0000 7f0f533e1700 1 --2- 172.21.15.60:0/4010247517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f4c101540 0x7f0f4c1019a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00aad0 tx=0x7f0f3c00ade0 comp rx=0 tx=0).stop 2024-02-26T04:57:32.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.979+0000 7f0f533e1700 1 -- 172.21.15.60:0/4010247517 shutdown_connections 2024-02-26T04:57:32.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f533e1700 1 --2- 172.21.15.60:0/4010247517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c101ee0 0x7f0f4c108790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:32.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f533e1700 1 --2- 172.21.15.60:0/4010247517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f4c100340 0x7f0f4c100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f533e1700 1 --2- 172.21.15.60:0/4010247517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f4c101540 0x7f0f4c1019a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f533e1700 1 -- 172.21.15.60:0/4010247517 >> 172.21.15.60:0/4010247517 conn(0x7f0f4c0fb900 msgr2=0x7f0f4c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f533e1700 1 -- 172.21.15.60:0/4010247517 shutdown_connections 2024-02-26T04:57:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f533e1700 1 -- 172.21.15.60:0/4010247517 wait complete. 2024-02-26T04:57:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f533e1700 1 Processor -- start 2024-02-26T04:57:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f533e1700 1 -- start start 2024-02-26T04:57:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f533e1700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f4c100340 0x7f0f4c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f533e1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c101540 0x7f0f4c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f533e1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f4c101ee0 0x7f0f4c19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f533e1700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c19f720 con 0x7f0f4c100340 2024-02-26T04:57:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f533e1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c19f860 con 0x7f0f4c101540 2024-02-26T04:57:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f533e1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c19f9f0 con 0x7f0f4c101ee0 2024-02-26T04:57:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.980+0000 7f0f5117d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f4c100340 0x7f0f4c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f5117d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f4c100340 0x7f0f4c197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50344/0 (socket says 172.21.15.60:50344) 2024-02-26T04:57:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f5117d700 1 -- 172.21.15.60:0/3669970578 learned_addr learned my addr 172.21.15.60:0/3669970578 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f5097c700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c101540 0x7f0f4c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f5197e700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f4c101ee0 0x7f0f4c19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f5117d700 1 -- 172.21.15.60:0/3669970578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f4c101ee0 msgr2=0x7f0f4c19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f5117d700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f4c101ee0 0x7f0f4c19f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f5117d700 1 -- 172.21.15.60:0/3669970578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c101540 msgr2=0x7f0f4c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f5117d700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c101540 0x7f0f4c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f5117d700 1 -- 172.21.15.60:0/3669970578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c00a700 con 0x7f0f4c100340 2024-02-26T04:57:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f5097c700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c101540 0x7f0f4c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f5117d700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f4c100340 0x7f0f4c197b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800c9c0 tx=0x7f0f4800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:32.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f427fc700 1 -- 172.21.15.60:0/3669970578 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f48007ac0 con 0x7f0f4c100340 2024-02-26T04:57:32.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f533e1700 1 -- 172.21.15.60:0/3669970578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4c19fcd0 con 0x7f0f4c100340 2024-02-26T04:57:32.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f427fc700 1 -- 172.21.15.60:0/3669970578 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f48007c20 con 0x7f0f4c100340 2024-02-26T04:57:32.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f427fc700 1 -- 172.21.15.60:0/3669970578 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f48018660 con 0x7f0f4c100340 2024-02-26T04:57:32.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.981+0000 7f0f533e1700 1 -- 172.21.15.60:0/3669970578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4c1a0170 con 0x7f0f4c100340 2024-02-26T04:57:32.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.982+0000 7f0f533e1700 1 -- 172.21.15.60:0/3669970578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f4c107630 con 0x7f0f4c100340 2024-02-26T04:57:32.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.984+0000 7f0f427fc700 1 -- 172.21.15.60:0/3669970578 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0f48004030 con 0x7f0f4c100340 2024-02-26T04:57:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.984+0000 7f0f427fc700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0f380a2150 0x7f0f380a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.985+0000 7f0f427fc700 1 -- 172.21.15.60:0/3669970578 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f0f480ce160 con 0x7f0f4c100340 2024-02-26T04:57:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.985+0000 7f0f5097c700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0f380a2150 0x7f0f380a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.985+0000 7f0f5097c700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0f380a2150 0x7f0f380a4610 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c000c00 tx=0x7f0f3c003cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:32.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:32.988+0000 7f0f427fc700 1 -- 172.21.15.60:0/3669970578 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0f48096d80 con 0x7f0f4c100340 2024-02-26T04:57:33.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:33.195+0000 7f0f533e1700 1 -- 172.21.15.60:0/3669970578 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f4c02da00 con 0x7f0f380a2150 2024-02-26T04:57:33.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:33.197+0000 7f0f427fc700 1 -- 172.21.15.60:0/3669970578 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f0f4c02da00 con 0x7f0f380a2150 2024-02-26T04:57:33.200 INFO:teuthology.orchestra.run.smithi060.stdout:HOST ADDR LABELS STATUS 2024-02-26T04:57:33.200 INFO:teuthology.orchestra.run.smithi060.stdout:smithi060 172.21.15.60 2024-02-26T04:57:33.200 INFO:teuthology.orchestra.run.smithi060.stdout:smithi156 172.21.15.156 2024-02-26T04:57:33.200 INFO:teuthology.orchestra.run.smithi060.stdout:smithi191 172.21.15.191 2024-02-26T04:57:33.200 INFO:teuthology.orchestra.run.smithi060.stdout:3 hosts in cluster 2024-02-26T04:57:33.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:33.200+0000 7f0f533e1700 1 -- 172.21.15.60:0/3669970578 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0f380a2150 msgr2=0x7f0f380a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:33.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:33.200+0000 7f0f533e1700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0f380a2150 0x7f0f380a4610 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c000c00 tx=0x7f0f3c003cc0 comp rx=0 tx=0).stop 2024-02-26T04:57:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:33.200+0000 7f0f533e1700 1 -- 172.21.15.60:0/3669970578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f4c100340 msgr2=0x7f0f4c197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:33.200+0000 7f0f533e1700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f4c100340 0x7f0f4c197b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800c9c0 tx=0x7f0f4800cd80 comp rx=0 tx=0).stop 2024-02-26T04:57:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:33.200+0000 7f0f533e1700 1 -- 172.21.15.60:0/3669970578 shutdown_connections 2024-02-26T04:57:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:33.200+0000 7f0f533e1700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f4c101ee0 0x7f0f4c19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:33.200+0000 7f0f533e1700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f4c100340 0x7f0f4c197b90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:33.200+0000 7f0f533e1700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0f380a2150 0x7f0f380a4610 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:33.200+0000 7f0f533e1700 1 --2- 172.21.15.60:0/3669970578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f4c101540 0x7f0f4c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:33.200+0000 7f0f533e1700 1 -- 172.21.15.60:0/3669970578 >> 172.21.15.60:0/3669970578 conn(0x7f0f4c0fb900 msgr2=0x7f0f4c104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:33.200+0000 7f0f533e1700 1 -- 172.21.15.60:0/3669970578 shutdown_connections 2024-02-26T04:57:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:33.200+0000 7f0f533e1700 1 -- 172.21.15.60:0/3669970578 wait complete. 2024-02-26T04:57:33.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-26T04:57:34.128 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:57:34.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:34 smithi191 ceph-mon[36352]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 52 B/s, 0 keys/s, 0 objects/s recovering 2024-02-26T04:57:34.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:34 smithi191 ceph-mon[36352]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:34.519 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:34 smithi060 ceph-mon[32382]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 52 B/s, 0 keys/s, 0 objects/s recovering 2024-02-26T04:57:34.519 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:34 smithi060 ceph-mon[32382]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:34.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:34 smithi156 ceph-mon[35805]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 52 B/s, 0 keys/s, 0 objects/s recovering 2024-02-26T04:57:34.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:34 smithi156 ceph-mon[35805]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:35.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.841+0000 7fa417a93700 1 -- 172.21.15.60:0/4009159960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa410101da0 msgr2=0x7fa410102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:35.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.841+0000 7fa417a93700 1 --2- 172.21.15.60:0/4009159960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa410101da0 0x7fa410102200 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa4040099c0 tx=0x7fa404009cd0 comp rx=0 tx=0).stop 2024-02-26T04:57:35.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.841+0000 7fa417a93700 1 -- 172.21.15.60:0/4009159960 shutdown_connections 2024-02-26T04:57:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.841+0000 7fa417a93700 1 --2- 172.21.15.60:0/4009159960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa410102740 0x7fa410108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.841+0000 7fa417a93700 1 --2- 172.21.15.60:0/4009159960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa410100ba0 0x7fa410100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.841+0000 7fa417a93700 1 --2- 172.21.15.60:0/4009159960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa410101da0 0x7fa410102200 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.841+0000 7fa417a93700 1 -- 172.21.15.60:0/4009159960 >> 172.21.15.60:0/4009159960 conn(0x7fa4100fc140 msgr2=0x7fa4100fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.841+0000 7fa417a93700 1 -- 172.21.15.60:0/4009159960 shutdown_connections 2024-02-26T04:57:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.841+0000 7fa417a93700 1 -- 172.21.15.60:0/4009159960 wait complete. 2024-02-26T04:57:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa417a93700 1 Processor -- start 2024-02-26T04:57:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa417a93700 1 -- start start 2024-02-26T04:57:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa417a93700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa410100ba0 0x7fa41006ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa417a93700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa410101da0 0x7fa41019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa417a93700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa410102740 0x7fa41019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa417a93700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41019f720 con 0x7fa410101da0 2024-02-26T04:57:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa417a93700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41019f890 con 0x7fa410102740 2024-02-26T04:57:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa417a93700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41019fa50 con 0x7fa410100ba0 2024-02-26T04:57:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa41502e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa410101da0 0x7fa41019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa41582f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa410100ba0 0x7fa41006ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa41502e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa410101da0 0x7fa41019c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50364/0 (socket says 172.21.15.60:50364) 2024-02-26T04:57:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa41502e700 1 -- 172.21.15.60:0/2186115316 learned_addr learned my addr 172.21.15.60:0/2186115316 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa416030700 1 --2- 172.21.15.60:0/2186115316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa410102740 0x7fa41019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa41502e700 1 -- 172.21.15.60:0/2186115316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa410100ba0 msgr2=0x7fa41006ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa41502e700 1 --2- 172.21.15.60:0/2186115316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa410100ba0 0x7fa41006ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa41502e700 1 -- 172.21.15.60:0/2186115316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa410102740 msgr2=0x7fa41019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa41502e700 1 --2- 172.21.15.60:0/2186115316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa410102740 0x7fa41019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.842+0000 7fa41502e700 1 -- 172.21.15.60:0/2186115316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4040095f0 con 0x7fa410101da0 2024-02-26T04:57:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.843+0000 7fa41502e700 1 --2- 172.21.15.60:0/2186115316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa410101da0 0x7fa41019c940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa404003790 tx=0x7fa404005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.843+0000 7fa40effd700 1 -- 172.21.15.60:0/2186115316 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40401d070 con 0x7fa410101da0 2024-02-26T04:57:35.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.843+0000 7fa417a93700 1 -- 172.21.15.60:0/2186115316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41019fcd0 con 0x7fa410101da0 2024-02-26T04:57:35.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.843+0000 7fa40effd700 1 -- 172.21.15.60:0/2186115316 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa404004110 con 0x7fa410101da0 2024-02-26T04:57:35.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.843+0000 7fa40effd700 1 -- 172.21.15.60:0/2186115316 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404022d60 con 0x7fa410101da0 2024-02-26T04:57:35.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.843+0000 7fa417a93700 1 -- 172.21.15.60:0/2186115316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4101a01c0 con 0x7fa410101da0 2024-02-26T04:57:35.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.844+0000 7fa417a93700 1 -- 172.21.15.60:0/2186115316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa410107f80 con 0x7fa410101da0 2024-02-26T04:57:35.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.845+0000 7fa40effd700 1 -- 172.21.15.60:0/2186115316 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fa404022470 con 0x7fa410101da0 2024-02-26T04:57:35.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.846+0000 7fa40effd700 1 --2- 172.21.15.60:0/2186115316 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa3f80a2150 0x7fa3f80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:35.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.846+0000 7fa41582f700 1 --2- 172.21.15.60:0/2186115316 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa3f80a2150 0x7fa3f80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:35.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.846+0000 7fa40effd700 1 -- 172.21.15.60:0/2186115316 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fa4040cf410 con 0x7fa410101da0 2024-02-26T04:57:35.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.846+0000 7fa41582f700 1 --2- 172.21.15.60:0/2186115316 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa3f80a2150 0x7fa3f80a4610 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa400009550 tx=0x7fa4000091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:35.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:35.849+0000 7fa40effd700 1 -- 172.21.15.60:0/2186115316 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa404098030 con 0x7fa410101da0 2024-02-26T04:57:36.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:36.064+0000 7fa417a93700 1 -- 172.21.15.60:0/2186115316 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa41002d8d0 con 0x7fa3f80a2150 2024-02-26T04:57:36.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:36.065+0000 7fa40effd700 1 -- 172.21.15.60:0/2186115316 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fa41002d8d0 con 0x7fa3f80a2150 2024-02-26T04:57:36.068 INFO:teuthology.orchestra.run.smithi060.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-26T04:57:36.068 INFO:teuthology.orchestra.run.smithi060.stdout:smithi060 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476008A480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-02-26T04:57:36.068 INFO:teuthology.orchestra.run.smithi060.stdout:smithi156 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760040480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-02-26T04:57:36.068 INFO:teuthology.orchestra.run.smithi060.stdout:smithi191 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303008M400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-26T04:57:36.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:36.068+0000 7fa417a93700 1 -- 172.21.15.60:0/2186115316 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa3f80a2150 msgr2=0x7fa3f80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:36.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:36.068+0000 7fa417a93700 1 --2- 172.21.15.60:0/2186115316 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa3f80a2150 0x7fa3f80a4610 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa400009550 tx=0x7fa4000091b0 comp rx=0 tx=0).stop 2024-02-26T04:57:36.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:36.068+0000 7fa417a93700 1 -- 172.21.15.60:0/2186115316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa410101da0 msgr2=0x7fa41019c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:36.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:36.068+0000 7fa417a93700 1 --2- 172.21.15.60:0/2186115316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa410101da0 0x7fa41019c940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa404003790 tx=0x7fa404005b00 comp rx=0 tx=0).stop 2024-02-26T04:57:36.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:36.068+0000 7fa417a93700 1 -- 172.21.15.60:0/2186115316 shutdown_connections 2024-02-26T04:57:36.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:36.068+0000 7fa417a93700 1 --2- 172.21.15.60:0/2186115316 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa3f80a2150 0x7fa3f80a4610 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:36.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:36.068+0000 7fa417a93700 1 --2- 172.21.15.60:0/2186115316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa410102740 0x7fa41019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:36.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:36.068+0000 7fa417a93700 1 --2- 172.21.15.60:0/2186115316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa410100ba0 0x7fa41006ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:36.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:36.068+0000 7fa417a93700 1 --2- 172.21.15.60:0/2186115316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa410101da0 0x7fa41019c940 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:36.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:36.068+0000 7fa417a93700 1 -- 172.21.15.60:0/2186115316 >> 172.21.15.60:0/2186115316 conn(0x7fa4100fc140 msgr2=0x7fa4101050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:36.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:36.068+0000 7fa417a93700 1 -- 172.21.15.60:0/2186115316 shutdown_connections 2024-02-26T04:57:36.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:36.068+0000 7fa417a93700 1 -- 172.21.15.60:0/2186115316 wait complete. 2024-02-26T04:57:36.440 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:36 smithi060 ceph-mon[32382]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 48 B/s, 0 keys/s, 0 objects/s recovering 2024-02-26T04:57:36.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:36 smithi191 ceph-mon[36352]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 48 B/s, 0 keys/s, 0 objects/s recovering 2024-02-26T04:57:36.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:36 smithi156 ceph-mon[35805]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 48 B/s, 0 keys/s, 0 objects/s recovering 2024-02-26T04:57:36.848 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-26T04:57:36.857 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T04:57:36.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-26T04:57:37.108 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:57:37.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:37 smithi060 ceph-mon[32382]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:37.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:37 smithi191 ceph-mon[36352]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:37.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:37 smithi156 ceph-mon[35805]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-26T04:57:38.583 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:38 smithi060 ceph-mon[32382]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 48 B/s, 0 objects/s recovering 2024-02-26T04:57:38.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:38 smithi156 ceph-mon[35805]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 48 B/s, 0 objects/s recovering 2024-02-26T04:57:38.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:38 smithi191 ceph-mon[36352]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 48 B/s, 0 objects/s recovering 2024-02-26T04:57:38.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.829+0000 7f7f8cd37700 1 -- 172.21.15.60:0/471202642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88102ea0 msgr2=0x7f7f88103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:38.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.829+0000 7f7f8cd37700 1 --2- 172.21.15.60:0/471202642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88102ea0 0x7f7f88103300 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0099c0 tx=0x7f7f7c009cd0 comp rx=0 tx=0).stop 2024-02-26T04:57:38.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.829+0000 7f7f8cd37700 1 -- 172.21.15.60:0/471202642 shutdown_connections 2024-02-26T04:57:38.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.829+0000 7f7f8cd37700 1 --2- 172.21.15.60:0/471202642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f88103840 0x7f7f8810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:38.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.829+0000 7f7f8cd37700 1 --2- 172.21.15.60:0/471202642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f88101ca0 0x7f7f881020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:38.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.829+0000 7f7f8cd37700 1 --2- 172.21.15.60:0/471202642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88102ea0 0x7f7f88103300 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:38.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.829+0000 7f7f8cd37700 1 -- 172.21.15.60:0/471202642 >> 172.21.15.60:0/471202642 conn(0x7f7f880fd280 msgr2=0x7f7f880ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:38.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.829+0000 7f7f8cd37700 1 -- 172.21.15.60:0/471202642 shutdown_connections 2024-02-26T04:57:38.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.829+0000 7f7f8cd37700 1 -- 172.21.15.60:0/471202642 wait complete. 2024-02-26T04:57:38.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.830+0000 7f7f8cd37700 1 Processor -- start 2024-02-26T04:57:38.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.830+0000 7f7f8cd37700 1 -- start start 2024-02-26T04:57:38.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.830+0000 7f7f8cd37700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f88101ca0 0x7f7f88075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:38.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.830+0000 7f7f8cd37700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f88102ea0 0x7f7f88074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:38.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.830+0000 7f7f8cd37700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88103840 0x7f7f880745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:38.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.830+0000 7f7f8cd37700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f88074d20 con 0x7f7f88103840 2024-02-26T04:57:38.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.830+0000 7f7f8cd37700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f88074e90 con 0x7f7f88102ea0 2024-02-26T04:57:38.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.830+0000 7f7f8cd37700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f881a1c90 con 0x7f7f88101ca0 2024-02-26T04:57:38.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.830+0000 7f7f86d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88103840 0x7f7f880745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:38.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.830+0000 7f7f86d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88103840 0x7f7f880745a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50388/0 (socket says 172.21.15.60:50388) 2024-02-26T04:57:38.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.830+0000 7f7f86d9d700 1 -- 172.21.15.60:0/3374787033 learned_addr learned my addr 172.21.15.60:0/3374787033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:38.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.831+0000 7f7f8659c700 1 --2- 172.21.15.60:0/3374787033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f88101ca0 0x7f7f88075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:38.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.831+0000 7f7f85d9b700 1 --2- 172.21.15.60:0/3374787033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f88102ea0 0x7f7f88074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:38.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.831+0000 7f7f86d9d700 1 -- 172.21.15.60:0/3374787033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f88101ca0 msgr2=0x7f7f88075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:38.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.831+0000 7f7f86d9d700 1 --2- 172.21.15.60:0/3374787033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f88101ca0 0x7f7f88075a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:38.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.831+0000 7f7f86d9d700 1 -- 172.21.15.60:0/3374787033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f88102ea0 msgr2=0x7f7f88074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:38.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.831+0000 7f7f86d9d700 1 --2- 172.21.15.60:0/3374787033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f88102ea0 0x7f7f88074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:38.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.831+0000 7f7f86d9d700 1 -- 172.21.15.60:0/3374787033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c0095f0 con 0x7f7f88103840 2024-02-26T04:57:38.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.831+0000 7f7f86d9d700 1 --2- 172.21.15.60:0/3374787033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88103840 0x7f7f880745a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000ada0 tx=0x7f7f8000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:38.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.831+0000 7f7f777fe700 1 -- 172.21.15.60:0/3374787033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8000f070 con 0x7f7f88103840 2024-02-26T04:57:38.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.831+0000 7f7f777fe700 1 -- 172.21.15.60:0/3374787033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f80007b30 con 0x7f7f88103840 2024-02-26T04:57:38.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.831+0000 7f7f8cd37700 1 -- 172.21.15.60:0/3374787033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f881a1f70 con 0x7f7f88103840 2024-02-26T04:57:38.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.831+0000 7f7f777fe700 1 -- 172.21.15.60:0/3374787033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f80005500 con 0x7f7f88103840 2024-02-26T04:57:38.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.831+0000 7f7f8cd37700 1 -- 172.21.15.60:0/3374787033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f881a2390 con 0x7f7f88103840 2024-02-26T04:57:38.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.832+0000 7f7f8cd37700 1 -- 172.21.15.60:0/3374787033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f88068930 con 0x7f7f88103840 2024-02-26T04:57:38.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.834+0000 7f7f777fe700 1 -- 172.21.15.60:0/3374787033 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f7f80004ad0 con 0x7f7f88103840 2024-02-26T04:57:38.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.834+0000 7f7f777fe700 1 --2- 172.21.15.60:0/3374787033 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7f700a2150 0x7f7f700a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:38.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.835+0000 7f7f777fe700 1 -- 172.21.15.60:0/3374787033 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f7f800cdda0 con 0x7f7f88103840 2024-02-26T04:57:38.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.835+0000 7f7f8659c700 1 --2- 172.21.15.60:0/3374787033 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7f700a2150 0x7f7f700a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:38.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.835+0000 7f7f8659c700 1 --2- 172.21.15.60:0/3374787033 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7f700a2150 0x7f7f700a4610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7f78005950 tx=0x7f7f780058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:38.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:38.838+0000 7f7f777fe700 1 -- 172.21.15.60:0/3374787033 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7f800969c0 con 0x7f7f88103840 2024-02-26T04:57:39.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:39.083+0000 7f7f8cd37700 1 -- 172.21.15.60:0/3374787033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7f881090e0 con 0x7f7f88103840 2024-02-26T04:57:39.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:39.084+0000 7f7f777fe700 1 -- 172.21.15.60:0/3374787033 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+2945 (secure 0 0 0) 0x7f7f8001a310 con 0x7f7f88103840 2024-02-26T04:57:39.086 INFO:teuthology.orchestra.run.smithi060.stdout:e22 2024-02-26T04:57:39.087 INFO:teuthology.orchestra.run.smithi060.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-26T04:57:39.087 INFO:teuthology.orchestra.run.smithi060.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T04:57:39.087 INFO:teuthology.orchestra.run.smithi060.stdout:legacy client fscid: 1 2024-02-26T04:57:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:39.087 INFO:teuthology.orchestra.run.smithi060.stdout:Filesystem 'cephfs' (1) 2024-02-26T04:57:39.087 INFO:teuthology.orchestra.run.smithi060.stdout:fs_name cephfs 2024-02-26T04:57:39.087 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 22 2024-02-26T04:57:39.087 INFO:teuthology.orchestra.run.smithi060.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-26T04:57:39.087 INFO:teuthology.orchestra.run.smithi060.stdout:created 2024-02-26T04:55:27.553789+0000 2024-02-26T04:57:39.087 INFO:teuthology.orchestra.run.smithi060.stdout:modified 2024-02-26T04:55:54.140330+0000 2024-02-26T04:57:39.087 INFO:teuthology.orchestra.run.smithi060.stdout:tableserver 0 2024-02-26T04:57:39.087 INFO:teuthology.orchestra.run.smithi060.stdout:root 0 2024-02-26T04:57:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:session_timeout 300 2024-02-26T04:57:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:session_autoclose 300 2024-02-26T04:57:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:max_file_size 1099511627776 2024-02-26T04:57:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:max_xattr_size 65536 2024-02-26T04:57:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:required_client_features {} 2024-02-26T04:57:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:last_failure 0 2024-02-26T04:57:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:last_failure_osd_epoch 0 2024-02-26T04:57:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-26T04:57:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:max_mds 3 2024-02-26T04:57:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:in 0,1,2 2024-02-26T04:57:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:up {0=24524,1=24476,2=24472} 2024-02-26T04:57:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:failed 2024-02-26T04:57:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:damaged 2024-02-26T04:57:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:stopped 2024-02-26T04:57:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:data_pools [3,4] 2024-02-26T04:57:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:metadata_pool 2 2024-02-26T04:57:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:inline_data disabled 2024-02-26T04:57:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:balancer 2024-02-26T04:57:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:bal_rank_mask -1 2024-02-26T04:57:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:standby_count_wanted 0 2024-02-26T04:57:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b{0:24524} state up:active seq 3 addr [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T04:57:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c{0:24454} state up:standby-replay seq 1 addr [v2:172.21.15.191:6836/3000497710,v1:172.21.15.191:6837/3000497710] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T04:57:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.h{1:24476} state up:active seq 12 addr [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T04:57:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.e{1:24440} state up:standby-replay seq 1 addr [v2:172.21.15.156:6834/2819944290,v1:172.21.15.156:6835/2819944290] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T04:57:39.090 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.i{2:24472} state up:active seq 12 addr [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T04:57:39.090 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.f{2:24436} state up:standby-replay seq 1 addr [v2:172.21.15.191:6834/1642858086,v1:172.21.15.191:6835/1642858086] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T04:57:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:39.090 INFO:teuthology.orchestra.run.smithi060.stdout:Standby daemons: 2024-02-26T04:57:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:39.090 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.60:6834/894258792,v1:172.21.15.60:6835/894258792] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T04:57:39.090 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d{-1:14535} state up:standby seq 1 addr [v2:172.21.15.60:6836/1357036637,v1:172.21.15.60:6837/1357036637] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T04:57:39.090 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.g{-1:14607} state up:standby seq 1 addr [v2:172.21.15.60:6838/2080933038,v1:172.21.15.60:6839/2080933038] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T04:57:39.090 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a{-1:14625} state up:standby seq 1 addr [v2:172.21.15.60:6840/1896690404,v1:172.21.15.60:6841/1896690404] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T04:57:39.090 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.l{-1:24397} state up:standby seq 1 addr [v2:172.21.15.191:6832/1119041220,v1:172.21.15.191:6833/1119041220] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T04:57:39.090 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.k{-1:24407} state up:standby seq 1 addr [v2:172.21.15.156:6832/629272517,v1:172.21.15.156:6833/629272517] compat {c=[1],r=[1],i=[fff]}] 2024-02-26T04:57:39.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:39.087+0000 7f7f8cd37700 1 -- 172.21.15.60:0/3374787033 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7f700a2150 msgr2=0x7f7f700a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:39.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:39.087+0000 7f7f8cd37700 1 --2- 172.21.15.60:0/3374787033 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7f700a2150 0x7f7f700a4610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7f78005950 tx=0x7f7f780058e0 comp rx=0 tx=0).stop 2024-02-26T04:57:39.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:39.087+0000 7f7f8cd37700 1 -- 172.21.15.60:0/3374787033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88103840 msgr2=0x7f7f880745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:39.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:39.087+0000 7f7f8cd37700 1 --2- 172.21.15.60:0/3374787033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88103840 0x7f7f880745a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000ada0 tx=0x7f7f8000c330 comp rx=0 tx=0).stop 2024-02-26T04:57:39.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:39.087+0000 7f7f8cd37700 1 -- 172.21.15.60:0/3374787033 shutdown_connections 2024-02-26T04:57:39.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:39.087+0000 7f7f8cd37700 1 --2- 172.21.15.60:0/3374787033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f88103840 0x7f7f880745a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:39.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:39.087+0000 7f7f8cd37700 1 --2- 172.21.15.60:0/3374787033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f88101ca0 0x7f7f88075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:39.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:39.087+0000 7f7f8cd37700 1 --2- 172.21.15.60:0/3374787033 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7f700a2150 0x7f7f700a4610 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:39.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:39.087+0000 7f7f8cd37700 1 --2- 172.21.15.60:0/3374787033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f88102ea0 0x7f7f88074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:39.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:39.087+0000 7f7f8cd37700 1 -- 172.21.15.60:0/3374787033 >> 172.21.15.60:0/3374787033 conn(0x7f7f880fd280 msgr2=0x7f7f88106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:39.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:39.087+0000 7f7f8cd37700 1 -- 172.21.15.60:0/3374787033 shutdown_connections 2024-02-26T04:57:39.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:39.087+0000 7f7f8cd37700 1 -- 172.21.15.60:0/3374787033 wait complete. 2024-02-26T04:57:39.092 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-02-26T04:57:39.460 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:39 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3374787033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-26T04:57:39.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:39 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3374787033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-26T04:57:39.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:39 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3374787033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-26T04:57:39.826 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- bash -c 'ceph osd dump' 2024-02-26T04:57:40.076 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:57:40.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:40 smithi060 ceph-mon[32382]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:40.578 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:40 smithi156 ceph-mon[35805]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:40.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:40 smithi191 ceph-mon[36352]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.860+0000 7f7926bb7700 1 -- 172.21.15.60:0/2006469670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79200746d0 msgr2=0x7f7920074b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.860+0000 7f7926bb7700 1 --2- 172.21.15.60:0/2006469670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79200746d0 0x7f7920074b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f790c009990 tx=0x7f790c009ca0 comp rx=0 tx=0).stop 2024-02-26T04:57:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.860+0000 7f7926bb7700 1 -- 172.21.15.60:0/2006469670 shutdown_connections 2024-02-26T04:57:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.860+0000 7f7926bb7700 1 --2- 172.21.15.60:0/2006469670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f792010e280 0x7f7920110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.860+0000 7f7926bb7700 1 --2- 172.21.15.60:0/2006469670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7920075ca0 0x7f7920074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.860+0000 7f7926bb7700 1 --2- 172.21.15.60:0/2006469670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79200746d0 0x7f7920074b50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.860+0000 7f7926bb7700 1 -- 172.21.15.60:0/2006469670 >> 172.21.15.60:0/2006469670 conn(0x7f79200fd280 msgr2=0x7f79200ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.860+0000 7f7926bb7700 1 -- 172.21.15.60:0/2006469670 shutdown_connections 2024-02-26T04:57:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.860+0000 7f7926bb7700 1 -- 172.21.15.60:0/2006469670 wait complete. 2024-02-26T04:57:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.860+0000 7f7926bb7700 1 Processor -- start 2024-02-26T04:57:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.860+0000 7f7926bb7700 1 -- start start 2024-02-26T04:57:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7926bb7700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79200746d0 0x7f792019e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7926bb7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7920075ca0 0x7f792019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7926bb7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f792010e280 0x7f79201a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7926bb7700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79201a5db0 con 0x7f79200746d0 2024-02-26T04:57:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7926bb7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79201a5ef0 con 0x7f7920075ca0 2024-02-26T04:57:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7926bb7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79201a6080 con 0x7f792010e280 2024-02-26T04:57:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7924953700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79200746d0 0x7f792019e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7924953700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79200746d0 0x7f792019e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37850/0 (socket says 172.21.15.60:37850) 2024-02-26T04:57:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7924953700 1 -- 172.21.15.60:0/2021383543 learned_addr learned my addr 172.21.15.60:0/2021383543 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f791ffff700 1 --2- 172.21.15.60:0/2021383543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7920075ca0 0x7f792019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7925154700 1 --2- 172.21.15.60:0/2021383543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f792010e280 0x7f79201a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7924953700 1 -- 172.21.15.60:0/2021383543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f792010e280 msgr2=0x7f79201a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7924953700 1 --2- 172.21.15.60:0/2021383543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f792010e280 0x7f79201a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7924953700 1 -- 172.21.15.60:0/2021383543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7920075ca0 msgr2=0x7f792019e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7924953700 1 --2- 172.21.15.60:0/2021383543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7920075ca0 0x7f792019e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7924953700 1 -- 172.21.15.60:0/2021383543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c0095f0 con 0x7f79200746d0 2024-02-26T04:57:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7924953700 1 --2- 172.21.15.60:0/2021383543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79200746d0 0x7f792019e220 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f791400c920 tx=0x7f791400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f791dffb700 1 -- 172.21.15.60:0/2021383543 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914007ac0 con 0x7f79200746d0 2024-02-26T04:57:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f7926bb7700 1 -- 172.21.15.60:0/2021383543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79201a6360 con 0x7f79200746d0 2024-02-26T04:57:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f791dffb700 1 -- 172.21.15.60:0/2021383543 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f791400ce70 con 0x7f79200746d0 2024-02-26T04:57:41.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.861+0000 7f791dffb700 1 -- 172.21.15.60:0/2021383543 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914018660 con 0x7f79200746d0 2024-02-26T04:57:41.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.862+0000 7f7926bb7700 1 -- 172.21.15.60:0/2021383543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79201a68b0 con 0x7f79200746d0 2024-02-26T04:57:41.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.862+0000 7f7926bb7700 1 -- 172.21.15.60:0/2021383543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79201044e0 con 0x7f79200746d0 2024-02-26T04:57:41.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.863+0000 7f791dffb700 1 -- 172.21.15.60:0/2021383543 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f791400f450 con 0x7f79200746d0 2024-02-26T04:57:41.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.864+0000 7f791dffb700 1 --2- 172.21.15.60:0/2021383543 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f79100a2150 0x7f79100a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:41.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.864+0000 7f791ffff700 1 --2- 172.21.15.60:0/2021383543 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f79100a2150 0x7f79100a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:41.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.864+0000 7f791dffb700 1 -- 172.21.15.60:0/2021383543 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f79140cdfc0 con 0x7f79200746d0 2024-02-26T04:57:41.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.865+0000 7f791ffff700 1 --2- 172.21.15.60:0/2021383543 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f79100a2150 0x7f79100a4610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f790c003790 tx=0x7f790c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:41.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:41.869+0000 7f791dffb700 1 -- 172.21.15.60:0/2021383543 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7914096b50 con 0x7f79200746d0 2024-02-26T04:57:42.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.072+0000 7f7926bb7700 1 -- 172.21.15.60:0/2021383543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f7920068930 con 0x7f79200746d0 2024-02-26T04:57:42.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.073+0000 7f791dffb700 1 -- 172.21.15.60:0/2021383543 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) v1 ==== 56+0+5807 (secure 0 0 0) 0x7f791409a210 con 0x7f79200746d0 2024-02-26T04:57:42.075 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 93 2024-02-26T04:57:42.075 INFO:teuthology.orchestra.run.smithi060.stdout:fsid f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T04:57:42.075 INFO:teuthology.orchestra.run.smithi060.stdout:created 2024-02-26T04:47:22.521838+0000 2024-02-26T04:57:42.076 INFO:teuthology.orchestra.run.smithi060.stdout:modified 2024-02-26T04:57:21.168776+0000 2024-02-26T04:57:42.076 INFO:teuthology.orchestra.run.smithi060.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-02-26T04:57:42.076 INFO:teuthology.orchestra.run.smithi060.stdout:crush_version 27 2024-02-26T04:57:42.076 INFO:teuthology.orchestra.run.smithi060.stdout:full_ratio 0.95 2024-02-26T04:57:42.076 INFO:teuthology.orchestra.run.smithi060.stdout:backfillfull_ratio 0.9 2024-02-26T04:57:42.076 INFO:teuthology.orchestra.run.smithi060.stdout:nearfull_ratio 0.85 2024-02-26T04:57:42.076 INFO:teuthology.orchestra.run.smithi060.stdout:require_min_compat_client luminous 2024-02-26T04:57:42.076 INFO:teuthology.orchestra.run.smithi060.stdout:min_compat_client luminous 2024-02-26T04:57:42.076 INFO:teuthology.orchestra.run.smithi060.stdout:require_osd_release squid 2024-02-26T04:57:42.076 INFO:teuthology.orchestra.run.smithi060.stdout:stretch_mode_enabled false 2024-02-26T04:57:42.076 INFO:teuthology.orchestra.run.smithi060.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-02-26T04:57:42.076 INFO:teuthology.orchestra.run.smithi060.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-02-26T04:57:42.076 INFO:teuthology.orchestra.run.smithi060.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-02-26T04:57:42.077 INFO:teuthology.orchestra.run.smithi060.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-02-26T04:57:42.077 INFO:teuthology.orchestra.run.smithi060.stdout:max_osd 12 2024-02-26T04:57:42.077 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6802/108541016,v1:172.21.15.60:6803/108541016] [v2:172.21.15.60:6804/108541016,v1:172.21.15.60:6805/108541016] exists,up cd04e2cc-5293-493c-a8e2-bef04f89c6ce 2024-02-26T04:57:42.077 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6800/1258051364,v1:172.21.15.156:6801/1258051364] [v2:172.21.15.156:6802/1258051364,v1:172.21.15.156:6803/1258051364] exists,up c43bde4b-f31d-4d09-8410-583371866db3 2024-02-26T04:57:42.077 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 up in weight 1 up_from 18 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6800/1607191954,v1:172.21.15.191:6801/1607191954] [v2:172.21.15.191:6802/1607191954,v1:172.21.15.191:6803/1607191954] exists,up 5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e 2024-02-26T04:57:42.077 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6810/3123201146,v1:172.21.15.60:6811/3123201146] [v2:172.21.15.60:6812/3123201146,v1:172.21.15.60:6813/3123201146] exists,up 7e2d7dc0-1002-44e9-9295-2447bf76a03a 2024-02-26T04:57:42.077 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 up in weight 1 up_from 33 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6808/714533576,v1:172.21.15.156:6809/714533576] [v2:172.21.15.156:6810/714533576,v1:172.21.15.156:6811/714533576] exists,up d040e4b1-df8e-4f0c-9f84-767081c41939 2024-02-26T04:57:42.077 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 up in weight 1 up_from 38 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6808/1681496417,v1:172.21.15.191:6809/1681496417] [v2:172.21.15.191:6810/1681496417,v1:172.21.15.191:6811/1681496417] exists,up a7807402-620a-40f2-85b4-71f762a1a73b 2024-02-26T04:57:42.077 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6818/3054430804,v1:172.21.15.60:6819/3054430804] [v2:172.21.15.60:6820/3054430804,v1:172.21.15.60:6821/3054430804] exists,up 43d4456c-935e-4e9c-afe8-9a6c72a8a4ad 2024-02-26T04:57:42.077 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 up in weight 1 up_from 51 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6816/2499710318,v1:172.21.15.156:6817/2499710318] [v2:172.21.15.156:6818/2499710318,v1:172.21.15.156:6819/2499710318] exists,up e158aaaf-5538-4711-ae58-2d9acc28c9f2 2024-02-26T04:57:42.077 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8 up in weight 1 up_from 59 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6816/3417694461,v1:172.21.15.191:6817/3417694461] [v2:172.21.15.191:6818/3417694461,v1:172.21.15.191:6819/3417694461] exists,up 5669eb6b-6750-411a-bd9f-61cf8db220fc 2024-02-26T04:57:42.077 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9 up in weight 1 up_from 67 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6826/1562445343,v1:172.21.15.60:6827/1562445343] [v2:172.21.15.60:6828/1562445343,v1:172.21.15.60:6829/1562445343] exists,up c0802892-114a-4875-9914-aa936d399e4f 2024-02-26T04:57:42.077 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6824/3738247798,v1:172.21.15.156:6825/3738247798] [v2:172.21.15.156:6826/3738247798,v1:172.21.15.156:6827/3738247798] exists,up eda157d9-fe5f-46ad-8a82-0d60cf51c438 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6824/277456354,v1:172.21.15.191:6825/277456354] [v2:172.21.15.191:6826/277456354,v1:172.21.15.191:6827/277456354] exists,up 80ebe4a4-3a19-4074-aa49-bfe2339295b7 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.0 [8,4] 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.a [2,11] 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.1d [2,4] 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 2.1f [7,11] 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.6 [4,7] 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.c [5,6] 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.16 [1,8] 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.17 [4,8] 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.1d [4,7] 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.1e [3,9] 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.27 [3,9] 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.31 [4,8] 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.37 [1,9] 2024-02-26T04:57:42.078 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 3.39 [5,6] 2024-02-26T04:57:42.079 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 4.f [9,8] 2024-02-26T04:57:42.079 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 4.13 [9,10] 2024-02-26T04:57:42.079 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 4.17 [9,8] 2024-02-26T04:57:42.079 INFO:teuthology.orchestra.run.smithi060.stdout:pg_upmap_items 4.1d [5,8] 2024-02-26T04:57:42.079 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/556269024 expires 2024-02-27T04:48:18.223906+0000 2024-02-26T04:57:42.079 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/1221449590 expires 2024-02-27T04:48:18.223906+0000 2024-02-26T04:57:42.079 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6800/114766175 expires 2024-02-27T04:47:50.799728+0000 2024-02-26T04:57:42.079 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/2375965956 expires 2024-02-27T04:47:50.799728+0000 2024-02-26T04:57:42.079 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/1037269853 expires 2024-02-27T04:48:18.223906+0000 2024-02-26T04:57:42.079 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6800/3694881479 expires 2024-02-27T04:48:18.223906+0000 2024-02-26T04:57:42.079 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6801/114766175 expires 2024-02-27T04:47:50.799728+0000 2024-02-26T04:57:42.079 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/558398124 expires 2024-02-27T04:47:50.799728+0000 2024-02-26T04:57:42.079 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6801/3694881479 expires 2024-02-27T04:48:18.223906+0000 2024-02-26T04:57:42.079 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/1162202856 expires 2024-02-27T04:47:50.799728+0000 2024-02-26T04:57:42.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.075+0000 7f7926bb7700 1 -- 172.21.15.60:0/2021383543 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f79100a2150 msgr2=0x7f79100a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:42.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.075+0000 7f7926bb7700 1 --2- 172.21.15.60:0/2021383543 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f79100a2150 0x7f79100a4610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f790c003790 tx=0x7f790c005fd0 comp rx=0 tx=0).stop 2024-02-26T04:57:42.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.075+0000 7f7926bb7700 1 -- 172.21.15.60:0/2021383543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79200746d0 msgr2=0x7f792019e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:42.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.075+0000 7f7926bb7700 1 --2- 172.21.15.60:0/2021383543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79200746d0 0x7f792019e220 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f791400c920 tx=0x7f791400cc30 comp rx=0 tx=0).stop 2024-02-26T04:57:42.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.075+0000 7f7926bb7700 1 -- 172.21.15.60:0/2021383543 shutdown_connections 2024-02-26T04:57:42.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.075+0000 7f7926bb7700 1 --2- 172.21.15.60:0/2021383543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f792010e280 0x7f79201a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:42.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.075+0000 7f7926bb7700 1 --2- 172.21.15.60:0/2021383543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79200746d0 0x7f792019e220 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:42.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.075+0000 7f7926bb7700 1 --2- 172.21.15.60:0/2021383543 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f79100a2150 0x7f79100a4610 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:42.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.075+0000 7f7926bb7700 1 --2- 172.21.15.60:0/2021383543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7920075ca0 0x7f792019e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:42.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.075+0000 7f7926bb7700 1 -- 172.21.15.60:0/2021383543 >> 172.21.15.60:0/2021383543 conn(0x7f79200fd280 msgr2=0x7f7920101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:42.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.075+0000 7f7926bb7700 1 -- 172.21.15.60:0/2021383543 shutdown_connections 2024-02-26T04:57:42.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.075+0000 7f7926bb7700 1 -- 172.21.15.60:0/2021383543 wait complete. 2024-02-26T04:57:42.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:42 smithi156 ceph-mon[35805]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:57:42.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:42 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2021383543' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-26T04:57:42.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:42 smithi060 ceph-mon[32382]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:57:42.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:42 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2021383543' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-26T04:57:42.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:42 smithi191 ceph-mon[36352]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:57:42.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:42 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2021383543' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-26T04:57:42.783 INFO:teuthology.run_tasks:Running task fs.ready... 2024-02-26T04:57:42.797 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:57:42.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.903+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2296792604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a68102e00 msgr2=0x7f6a68109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:42.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.903+0000 7f6a6e1ed640 1 --2- 172.21.15.60:0/2296792604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a68102e00 0x7f6a68109690 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6a60009ca0 tx=0x7f6a6002c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:42.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.903+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2296792604 shutdown_connections 2024-02-26T04:57:42.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.903+0000 7f6a6e1ed640 1 --2- 172.21.15.60:0/2296792604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a68102e00 0x7f6a68109690 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:42.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.903+0000 7f6a6e1ed640 1 --2- 172.21.15.60:0/2296792604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a68102440 0x7f6a681028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.903+0000 7f6a6e1ed640 1 --2- 172.21.15.60:0/2296792604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680ff470 0x7f6a680ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.903+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2296792604 >> 172.21.15.60:0/2296792604 conn(0x7f6a680fada0 msgr2=0x7f6a680fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.903+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2296792604 shutdown_connections 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.903+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2296792604 wait complete. 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.903+0000 7f6a6e1ed640 1 Processor -- start 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.903+0000 7f6a6e1ed640 1 -- start start 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.903+0000 7f6a6e1ed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a680ff470 0x7f6a68197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a6e1ed640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a68102440 0x7f6a681977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a6e1ed640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a68102e00 0x7f6a6819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a6e1ed640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6819ed60 con 0x7f6a680ff470 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a6e1ed640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6819eed0 con 0x7f6a68102e00 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a6e1ed640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6819f090 con 0x7f6a68102440 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a677fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a680ff470 0x7f6a68197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a677fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a680ff470 0x7f6a68197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37864/0 (socket says 172.21.15.60:37864) 2024-02-26T04:57:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a677fe640 1 -- 172.21.15.60:0/2730207865 learned_addr learned my addr 172.21.15.60:0/2730207865 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:42.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a67fff640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a68102e00 0x7f6a6819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:42.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a677fe640 1 -- 172.21.15.60:0/2730207865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a68102440 msgr2=0x7f6a681977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:42.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a66ffd640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a68102440 0x7f6a681977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:42.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a677fe640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a68102440 0x7f6a681977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:42.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a677fe640 1 -- 172.21.15.60:0/2730207865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a68102e00 msgr2=0x7f6a6819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:42.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a677fe640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a68102e00 0x7f6a6819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:42.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a677fe640 1 -- 172.21.15.60:0/2730207865 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a60009900 con 0x7f6a680ff470 2024-02-26T04:57:42.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a67fff640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a68102e00 0x7f6a6819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:42.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a66ffd640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a68102440 0x7f6a681977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:42.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a677fe640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a680ff470 0x7f6a68197270 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800b450 tx=0x7f6a5800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:42.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a64ff9640 1 -- 172.21.15.60:0/2730207865 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a58011890 con 0x7f6a680ff470 2024-02-26T04:57:42.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a64ff9640 1 -- 172.21.15.60:0/2730207865 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a580103e0 con 0x7f6a680ff470 2024-02-26T04:57:42.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2730207865 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6819f370 con 0x7f6a680ff470 2024-02-26T04:57:42.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a64ff9640 1 -- 172.21.15.60:0/2730207865 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800f580 con 0x7f6a680ff470 2024-02-26T04:57:42.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.904+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2730207865 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6819f880 con 0x7f6a680ff470 2024-02-26T04:57:42.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.905+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2730207865 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a68108570 con 0x7f6a680ff470 2024-02-26T04:57:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.906+0000 7f6a64ff9640 1 -- 172.21.15.60:0/2730207865 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6a580108e0 con 0x7f6a680ff470 2024-02-26T04:57:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.907+0000 7f6a64ff9640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6a500a20c0 0x7f6a500a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.907+0000 7f6a64ff9640 1 -- 172.21.15.60:0/2730207865 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f6a580cdf10 con 0x7f6a680ff470 2024-02-26T04:57:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.907+0000 7f6a66ffd640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6a500a20c0 0x7f6a500a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.907+0000 7f6a66ffd640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6a500a20c0 0x7f6a500a4580 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6a68198780 tx=0x7f6a5c005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:42.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:42.911+0000 7f6a64ff9640 1 -- 172.21.15.60:0/2730207865 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6a58096aa0 con 0x7f6a680ff470 2024-02-26T04:57:43.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.093+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2730207865 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a6802ce40 con 0x7f6a680ff470 2024-02-26T04:57:43.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.095+0000 7f6a64ff9640 1 -- 172.21.15.60:0/2730207865 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24972 (secure 0 0 0) 0x7f6a5809a160 con 0x7f6a680ff470 2024-02-26T04:57:43.098 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:43.098 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:57:21.168776+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:57:43.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.097+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2730207865 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6a500a20c0 msgr2=0x7f6a500a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.097+0000 7f6a6e1ed640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6a500a20c0 0x7f6a500a4580 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6a68198780 tx=0x7f6a5c005d30 comp rx=0 tx=0).stop 2024-02-26T04:57:43.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.097+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2730207865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a680ff470 msgr2=0x7f6a68197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.097+0000 7f6a6e1ed640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a680ff470 0x7f6a68197270 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800b450 tx=0x7f6a5800b910 comp rx=0 tx=0).stop 2024-02-26T04:57:43.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.097+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2730207865 shutdown_connections 2024-02-26T04:57:43.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.097+0000 7f6a6e1ed640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f6a500a20c0 0x7f6a500a4580 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.097+0000 7f6a6e1ed640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a68102e00 0x7f6a6819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.097+0000 7f6a6e1ed640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a68102440 0x7f6a681977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.097+0000 7f6a6e1ed640 1 --2- 172.21.15.60:0/2730207865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a680ff470 0x7f6a68197270 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.097+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2730207865 >> 172.21.15.60:0/2730207865 conn(0x7f6a680fada0 msgr2=0x7f6a68105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:43.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.097+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2730207865 shutdown_connections 2024-02-26T04:57:43.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.097+0000 7f6a6e1ed640 1 -- 172.21.15.60:0/2730207865 wait complete. 2024-02-26T04:57:43.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T04:50:44.572921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T04:55:25.530720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T04:55:26.741604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T04:55:30.453569+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T04:57:43.112 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T04:57:43.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.216+0000 7fce06964640 1 -- 172.21.15.60:0/2567450034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce00100080 msgr2=0x7fce001004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.216+0000 7fce06964640 1 --2- 172.21.15.60:0/2567450034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce00100080 0x7fce001004a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009900 tx=0x7fcdf002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:43.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.217+0000 7fce06964640 1 -- 172.21.15.60:0/2567450034 shutdown_connections 2024-02-26T04:57:43.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.217+0000 7fce06964640 1 --2- 172.21.15.60:0/2567450034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce00101c30 0x7fce001084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.217+0000 7fce06964640 1 --2- 172.21.15.60:0/2567450034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce00101270 0x7fce001016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.217+0000 7fce06964640 1 --2- 172.21.15.60:0/2567450034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce00100080 0x7fce001004a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.217+0000 7fce06964640 1 -- 172.21.15.60:0/2567450034 >> 172.21.15.60:0/2567450034 conn(0x7fce000fb610 msgr2=0x7fce000fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:43.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.217+0000 7fce06964640 1 -- 172.21.15.60:0/2567450034 shutdown_connections 2024-02-26T04:57:43.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.217+0000 7fce06964640 1 -- 172.21.15.60:0/2567450034 wait complete. 2024-02-26T04:57:43.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.217+0000 7fce06964640 1 Processor -- start 2024-02-26T04:57:43.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.217+0000 7fce06964640 1 -- start start 2024-02-26T04:57:43.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.217+0000 7fce06964640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce00100080 0x7fce00068eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.217+0000 7fce06964640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce00101270 0x7fce0019bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fce06964640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce00101c30 0x7fce0019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fce06964640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0019ed70 con 0x7fce00101270 2024-02-26T04:57:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fce06964640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0019eee0 con 0x7fce00100080 2024-02-26T04:57:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fce06964640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0019f0a0 con 0x7fce00101c30 2024-02-26T04:57:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fcdff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce00101270 0x7fce0019bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fcdff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce00101270 0x7fce0019bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37870/0 (socket says 172.21.15.60:37870) 2024-02-26T04:57:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fcdff7fe640 1 -- 172.21.15.60:0/1822269747 learned_addr learned my addr 172.21.15.60:0/1822269747 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fce04f01640 1 --2- 172.21.15.60:0/1822269747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce00101c30 0x7fce0019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fcdfffff640 1 --2- 172.21.15.60:0/1822269747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce00100080 0x7fce00068eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fcdff7fe640 1 -- 172.21.15.60:0/1822269747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce00101c30 msgr2=0x7fce0019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fcdff7fe640 1 --2- 172.21.15.60:0/1822269747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce00101c30 0x7fce0019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fcdff7fe640 1 -- 172.21.15.60:0/1822269747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce00100080 msgr2=0x7fce00068eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fcdff7fe640 1 --2- 172.21.15.60:0/1822269747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce00100080 0x7fce00068eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fcdff7fe640 1 -- 172.21.15.60:0/1822269747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf0009560 con 0x7fce00101270 2024-02-26T04:57:43.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fcdff7fe640 1 --2- 172.21.15.60:0/1822269747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce00101270 0x7fce0019bf70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400d950 tx=0x7fcdf400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:43.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fcdfd7fa640 1 -- 172.21.15.60:0/1822269747 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4009c30 con 0x7fce00101270 2024-02-26T04:57:43.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fcdfd7fa640 1 -- 172.21.15.60:0/1822269747 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdf4009d90 con 0x7fce00101270 2024-02-26T04:57:43.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fce06964640 1 -- 172.21.15.60:0/1822269747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0019f380 con 0x7fce00101270 2024-02-26T04:57:43.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fcdfd7fa640 1 -- 172.21.15.60:0/1822269747 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf400f7e0 con 0x7fce00101270 2024-02-26T04:57:43.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.218+0000 7fce06964640 1 -- 172.21.15.60:0/1822269747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0019f890 con 0x7fce00101270 2024-02-26T04:57:43.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.219+0000 7fce06964640 1 -- 172.21.15.60:0/1822269747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce00107440 con 0x7fce00101270 2024-02-26T04:57:43.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.220+0000 7fcdfd7fa640 1 -- 172.21.15.60:0/1822269747 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcdf4010ad0 con 0x7fce00101270 2024-02-26T04:57:43.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.221+0000 7fcdfd7fa640 1 --2- 172.21.15.60:0/1822269747 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcde80a20c0 0x7fcde80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:43.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.221+0000 7fcdfd7fa640 1 -- 172.21.15.60:0/1822269747 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fcdf40cdca0 con 0x7fce00101270 2024-02-26T04:57:43.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.221+0000 7fcdfffff640 1 --2- 172.21.15.60:0/1822269747 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcde80a20c0 0x7fcde80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:43.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.221+0000 7fcdfffff640 1 --2- 172.21.15.60:0/1822269747 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcde80a20c0 0x7fcde80a4580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005ab0 tx=0x7fcdf0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:43.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.225+0000 7fcdfd7fa640 1 -- 172.21.15.60:0/1822269747 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcdf40968e0 con 0x7fce00101270 2024-02-26T04:57:43.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.409+0000 7fce06964640 1 -- 172.21.15.60:0/1822269747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fce0002cda0 con 0x7fce00101270 2024-02-26T04:57:43.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.410+0000 7fcdfd7fa640 1 -- 172.21.15.60:0/1822269747 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcdf4099fa0 con 0x7fce00101270 2024-02-26T04:57:43.412 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T04:57:43.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.412+0000 7fce06964640 1 -- 172.21.15.60:0/1822269747 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcde80a20c0 msgr2=0x7fcde80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.412+0000 7fce06964640 1 --2- 172.21.15.60:0/1822269747 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcde80a20c0 0x7fcde80a4580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005ab0 tx=0x7fcdf0005fd0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.412+0000 7fce06964640 1 -- 172.21.15.60:0/1822269747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce00101270 msgr2=0x7fce0019bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.412+0000 7fce06964640 1 --2- 172.21.15.60:0/1822269747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce00101270 0x7fce0019bf70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400d950 tx=0x7fcdf400de10 comp rx=0 tx=0).stop 2024-02-26T04:57:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.412+0000 7fce06964640 1 -- 172.21.15.60:0/1822269747 shutdown_connections 2024-02-26T04:57:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.412+0000 7fce06964640 1 --2- 172.21.15.60:0/1822269747 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcde80a20c0 0x7fcde80a4580 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.412+0000 7fce06964640 1 --2- 172.21.15.60:0/1822269747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce00101c30 0x7fce0019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.412+0000 7fce06964640 1 --2- 172.21.15.60:0/1822269747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce00101270 0x7fce0019bf70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.412+0000 7fce06964640 1 --2- 172.21.15.60:0/1822269747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce00100080 0x7fce00068eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.412+0000 7fce06964640 1 -- 172.21.15.60:0/1822269747 >> 172.21.15.60:0/1822269747 conn(0x7fce000fb610 msgr2=0x7fce00104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.412+0000 7fce06964640 1 -- 172.21.15.60:0/1822269747 shutdown_connections 2024-02-26T04:57:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.412+0000 7fce06964640 1 -- 172.21.15.60:0/1822269747 wait complete. 2024-02-26T04:57:43.424 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:43 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2730207865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:43.425 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T04:57:43.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.528+0000 7f67104d5640 1 -- 172.21.15.60:0/2441638196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6708101c40 msgr2=0x7f67081084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.528+0000 7f67104d5640 1 --2- 172.21.15.60:0/2441638196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6708101c40 0x7f67081084d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6704009ca0 tx=0x7f670402c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 -- 172.21.15.60:0/2441638196 shutdown_connections 2024-02-26T04:57:43.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 --2- 172.21.15.60:0/2441638196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6708101c40 0x7f67081084d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 --2- 172.21.15.60:0/2441638196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6708101280 0x7f6708101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 --2- 172.21.15.60:0/2441638196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6708100090 0x7f67081004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 -- 172.21.15.60:0/2441638196 >> 172.21.15.60:0/2441638196 conn(0x7f67080fb620 msgr2=0x7f67080fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:43.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 -- 172.21.15.60:0/2441638196 shutdown_connections 2024-02-26T04:57:43.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 -- 172.21.15.60:0/2441638196 wait complete. 2024-02-26T04:57:43.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 Processor -- start 2024-02-26T04:57:43.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 -- start start 2024-02-26T04:57:43.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6708100090 0x7f6708104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:43.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6708101280 0x7f6708102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:43.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6708101c40 0x7f6708102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:43.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6708103300 con 0x7f6708101280 2024-02-26T04:57:43.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6708103470 con 0x7f6708101c40 2024-02-26T04:57:43.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f67104d5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67081a11f0 con 0x7f6708100090 2024-02-26T04:57:43.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.529+0000 7f670da70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6708101280 0x7f6708102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:43.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f670da70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6708101280 0x7f6708102660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37890/0 (socket says 172.21.15.60:37890) 2024-02-26T04:57:43.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f670da70640 1 -- 172.21.15.60:0/3088510676 learned_addr learned my addr 172.21.15.60:0/3088510676 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:43.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f670ea72640 1 --2- 172.21.15.60:0/3088510676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6708101c40 0x7f6708102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:43.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f670e271640 1 --2- 172.21.15.60:0/3088510676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6708100090 0x7f6708104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:43.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f670da70640 1 -- 172.21.15.60:0/3088510676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6708100090 msgr2=0x7f6708104020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f670da70640 1 --2- 172.21.15.60:0/3088510676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6708100090 0x7f6708104020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f670da70640 1 -- 172.21.15.60:0/3088510676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6708101c40 msgr2=0x7f6708102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f670da70640 1 --2- 172.21.15.60:0/3088510676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6708101c40 0x7f6708102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f670da70640 1 -- 172.21.15.60:0/3088510676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6704009900 con 0x7f6708101280 2024-02-26T04:57:43.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f670da70640 1 --2- 172.21.15.60:0/3088510676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6708101280 0x7f6708102660 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f66f800c950 tx=0x7f66f800ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:43.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f66ff7fe640 1 -- 172.21.15.60:0/3088510676 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8007c70 con 0x7f6708101280 2024-02-26T04:57:43.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f66ff7fe640 1 -- 172.21.15.60:0/3088510676 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66f8007dd0 con 0x7f6708101280 2024-02-26T04:57:43.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f67104d5640 1 -- 172.21.15.60:0/3088510676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67081a14a0 con 0x7f6708101280 2024-02-26T04:57:43.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f66ff7fe640 1 -- 172.21.15.60:0/3088510676 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f80188c0 con 0x7f6708101280 2024-02-26T04:57:43.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.530+0000 7f67104d5640 1 -- 172.21.15.60:0/3088510676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67081a1a00 con 0x7f6708101280 2024-02-26T04:57:43.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.531+0000 7f67104d5640 1 -- 172.21.15.60:0/3088510676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670810baa0 con 0x7f6708101280 2024-02-26T04:57:43.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.533+0000 7f66ff7fe640 1 -- 172.21.15.60:0/3088510676 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f66f8018a20 con 0x7f6708101280 2024-02-26T04:57:43.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.533+0000 7f66ff7fe640 1 --2- 172.21.15.60:0/3088510676 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f66f40a2140 0x7f66f40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:43.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.533+0000 7f670e271640 1 --2- 172.21.15.60:0/3088510676 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f66f40a2140 0x7f66f40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:43.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.534+0000 7f66ff7fe640 1 -- 172.21.15.60:0/3088510676 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f66f80ce400 con 0x7f6708101280 2024-02-26T04:57:43.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.534+0000 7f670e271640 1 --2- 172.21.15.60:0/3088510676 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f66f40a2140 0x7f66f40a4600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f67000058c0 tx=0x7f670000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:43.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.537+0000 7f66ff7fe640 1 -- 172.21.15.60:0/3088510676 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f66f80970a0 con 0x7f6708101280 2024-02-26T04:57:43.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:43 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2730207865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:43.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.717+0000 7f67104d5640 1 -- 172.21.15.60:0/3088510676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67081a1c80 con 0x7f6708101280 2024-02-26T04:57:43.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.718+0000 7f66ff7fe640 1 -- 172.21.15.60:0/3088510676 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f66f809a760 con 0x7f6708101280 2024-02-26T04:57:43.720 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:57:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.720+0000 7f67104d5640 1 -- 172.21.15.60:0/3088510676 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f66f40a2140 msgr2=0x7f66f40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.720+0000 7f67104d5640 1 --2- 172.21.15.60:0/3088510676 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f66f40a2140 0x7f66f40a4600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f67000058c0 tx=0x7f670000a2d0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.720+0000 7f67104d5640 1 -- 172.21.15.60:0/3088510676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6708101280 msgr2=0x7f6708102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.720+0000 7f67104d5640 1 --2- 172.21.15.60:0/3088510676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6708101280 0x7f6708102660 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f66f800c950 tx=0x7f66f800ce10 comp rx=0 tx=0).stop 2024-02-26T04:57:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.720+0000 7f67104d5640 1 -- 172.21.15.60:0/3088510676 shutdown_connections 2024-02-26T04:57:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.720+0000 7f67104d5640 1 --2- 172.21.15.60:0/3088510676 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f66f40a2140 0x7f66f40a4600 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.720+0000 7f67104d5640 1 --2- 172.21.15.60:0/3088510676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6708101c40 0x7f6708102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.720+0000 7f67104d5640 1 --2- 172.21.15.60:0/3088510676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6708101280 0x7f6708102660 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.720+0000 7f67104d5640 1 --2- 172.21.15.60:0/3088510676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6708100090 0x7f6708104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.720+0000 7f67104d5640 1 -- 172.21.15.60:0/3088510676 >> 172.21.15.60:0/3088510676 conn(0x7f67080fb620 msgr2=0x7f67080fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.720+0000 7f67104d5640 1 -- 172.21.15.60:0/3088510676 shutdown_connections 2024-02-26T04:57:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.720+0000 7f67104d5640 1 -- 172.21.15.60:0/3088510676 wait complete. 2024-02-26T04:57:43.733 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T04:57:43.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:43 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2730207865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:43.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.836+0000 7fe5ac964640 1 -- 172.21.15.60:0/249153922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a8101270 msgr2=0x7fe5a81016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.836+0000 7fe5ac964640 1 --2- 172.21.15.60:0/249153922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a8101270 0x7fe5a81016f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe590009900 tx=0x7fe59002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:43.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.837+0000 7fe5ac964640 1 -- 172.21.15.60:0/249153922 shutdown_connections 2024-02-26T04:57:43.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.837+0000 7fe5ac964640 1 --2- 172.21.15.60:0/249153922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5a8101c30 0x7fe5a81084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.837+0000 7fe5ac964640 1 --2- 172.21.15.60:0/249153922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a8101270 0x7fe5a81016f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.837+0000 7fe5ac964640 1 --2- 172.21.15.60:0/249153922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5a8100080 0x7fe5a81004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.837+0000 7fe5ac964640 1 -- 172.21.15.60:0/249153922 >> 172.21.15.60:0/249153922 conn(0x7fe5a80fb610 msgr2=0x7fe5a80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:43.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.837+0000 7fe5ac964640 1 -- 172.21.15.60:0/249153922 shutdown_connections 2024-02-26T04:57:43.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.837+0000 7fe5ac964640 1 -- 172.21.15.60:0/249153922 wait complete. 2024-02-26T04:57:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.837+0000 7fe5ac964640 1 Processor -- start 2024-02-26T04:57:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.837+0000 7fe5ac964640 1 -- start start 2024-02-26T04:57:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.837+0000 7fe5ac964640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5a8100080 0x7fe5a81971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.837+0000 7fe5ac964640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a8101270 0x7fe5a8197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5ac964640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5a8101c30 0x7fe5a819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:43.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5ac964640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a819ecc0 con 0x7fe5a8101270 2024-02-26T04:57:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5ac964640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a819ee30 con 0x7fe5a8100080 2024-02-26T04:57:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5ac964640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a819eff0 con 0x7fe5a8101c30 2024-02-26T04:57:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5a5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a8101270 0x7fe5a8197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5a5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a8101270 0x7fe5a8197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37908/0 (socket says 172.21.15.60:37908) 2024-02-26T04:57:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5a5d9b640 1 -- 172.21.15.60:0/1473382078 learned_addr learned my addr 172.21.15.60:0/1473382078 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5a659c640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5a8100080 0x7fe5a81971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5a6d9d640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5a8101c30 0x7fe5a819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5a5d9b640 1 -- 172.21.15.60:0/1473382078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5a8101c30 msgr2=0x7fe5a819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5a5d9b640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5a8101c30 0x7fe5a819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5a5d9b640 1 -- 172.21.15.60:0/1473382078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5a8100080 msgr2=0x7fe5a81971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5a5d9b640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5a8100080 0x7fe5a81971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5a5d9b640 1 -- 172.21.15.60:0/1473382078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe590009560 con 0x7fe5a8101270 2024-02-26T04:57:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5a6d9d640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5a8101c30 0x7fe5a819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5a5d9b640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a8101270 0x7fe5a8197710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe590006010 tx=0x7fe5900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1473382078 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5900044e0 con 0x7fe5a8101270 2024-02-26T04:57:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5ac964640 1 -- 172.21.15.60:0/1473382078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a819f270 con 0x7fe5a8101270 2024-02-26T04:57:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1473382078 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5900054e0 con 0x7fe5a8101270 2024-02-26T04:57:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1473382078 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59003e450 con 0x7fe5a8101270 2024-02-26T04:57:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.838+0000 7fe5ac964640 1 -- 172.21.15.60:0/1473382078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a819f7d0 con 0x7fe5a8101270 2024-02-26T04:57:43.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.839+0000 7fe5ac964640 1 -- 172.21.15.60:0/1473382078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a810ba90 con 0x7fe5a8101270 2024-02-26T04:57:43.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.840+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1473382078 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe5900035f0 con 0x7fe5a8101270 2024-02-26T04:57:43.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.841+0000 7fe59f7fe640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5940a20c0 0x7fe5940a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:43.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.841+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1473382078 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fe5900f0b20 con 0x7fe5a8101270 2024-02-26T04:57:43.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.841+0000 7fe5a659c640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5940a20c0 0x7fe5940a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:43.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.841+0000 7fe5a659c640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5940a20c0 0x7fe5940a4580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00058c0 tx=0x7fe5a0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:43.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:43.844+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1473382078 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe5900b96b0 con 0x7fe5a8101270 2024-02-26T04:57:44.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.026+0000 7fe5ac964640 1 -- 172.21.15.60:0/1473382078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe5a819f400 con 0x7fe5a8101270 2024-02-26T04:57:44.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.026+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1473382078 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe5a819f400 con 0x7fe5a8101270 2024-02-26T04:57:44.029 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T04:57:44.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.028+0000 7fe5ac964640 1 -- 172.21.15.60:0/1473382078 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5940a20c0 msgr2=0x7fe5940a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.028+0000 7fe5ac964640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5940a20c0 0x7fe5940a4580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00058c0 tx=0x7fe5a0005850 comp rx=0 tx=0).stop 2024-02-26T04:57:44.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.028+0000 7fe5ac964640 1 -- 172.21.15.60:0/1473382078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a8101270 msgr2=0x7fe5a8197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.028+0000 7fe5ac964640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a8101270 0x7fe5a8197710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe590006010 tx=0x7fe5900042f0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.028+0000 7fe5ac964640 1 -- 172.21.15.60:0/1473382078 shutdown_connections 2024-02-26T04:57:44.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.028+0000 7fe5ac964640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5940a20c0 0x7fe5940a4580 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.028+0000 7fe5ac964640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5a8101c30 0x7fe5a819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.028+0000 7fe5ac964640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a8101270 0x7fe5a8197710 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.028+0000 7fe5ac964640 1 --2- 172.21.15.60:0/1473382078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5a8100080 0x7fe5a81971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.028+0000 7fe5ac964640 1 -- 172.21.15.60:0/1473382078 >> 172.21.15.60:0/1473382078 conn(0x7fe5a80fb610 msgr2=0x7fe5a80fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:44.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.028+0000 7fe5ac964640 1 -- 172.21.15.60:0/1473382078 shutdown_connections 2024-02-26T04:57:44.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.028+0000 7fe5ac964640 1 -- 172.21.15.60:0/1473382078 wait complete. 2024-02-26T04:57:44.041 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T04:57:44.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.144+0000 7f0af86a2640 1 -- 172.21.15.60:0/4035618433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101290 msgr2=0x7f0af0101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.144+0000 7f0af86a2640 1 --2- 172.21.15.60:0/4035618433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101290 0x7f0af0101710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009900 tx=0x7f0ae002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:44.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.144+0000 7f0af86a2640 1 -- 172.21.15.60:0/4035618433 shutdown_connections 2024-02-26T04:57:44.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.144+0000 7f0af86a2640 1 --2- 172.21.15.60:0/4035618433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af0101c50 0x7f0af01084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.144+0000 7f0af86a2640 1 --2- 172.21.15.60:0/4035618433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101290 0x7f0af0101710 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.144+0000 7f0af86a2640 1 --2- 172.21.15.60:0/4035618433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af01000a0 0x7f0af01004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.144+0000 7f0af86a2640 1 -- 172.21.15.60:0/4035618433 >> 172.21.15.60:0/4035618433 conn(0x7f0af00fb630 msgr2=0x7f0af00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:44.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.144+0000 7f0af86a2640 1 -- 172.21.15.60:0/4035618433 shutdown_connections 2024-02-26T04:57:44.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.144+0000 7f0af86a2640 1 -- 172.21.15.60:0/4035618433 wait complete. 2024-02-26T04:57:44.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af86a2640 1 Processor -- start 2024-02-26T04:57:44.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af86a2640 1 -- start start 2024-02-26T04:57:44.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af86a2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af01000a0 0x7f0af01971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:44.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af86a2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af0101290 0x7f0af01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:44.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af86a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101c50 0x7f0af019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af86a2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af019ec90 con 0x7f0af0101c50 2024-02-26T04:57:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af86a2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af019ee00 con 0x7f0af01000a0 2024-02-26T04:57:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af86a2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af019efc0 con 0x7f0af0101290 2024-02-26T04:57:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af6c3f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101c50 0x7f0af019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af6c3f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101c50 0x7f0af019e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37924/0 (socket says 172.21.15.60:37924) 2024-02-26T04:57:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af6c3f640 1 -- 172.21.15.60:0/2648910260 learned_addr learned my addr 172.21.15.60:0/2648910260 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af5c3d640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af0101290 0x7f0af01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af643e640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af01000a0 0x7f0af01971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af6c3f640 1 -- 172.21.15.60:0/2648910260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af0101290 msgr2=0x7f0af01976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af6c3f640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af0101290 0x7f0af01976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af6c3f640 1 -- 172.21.15.60:0/2648910260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af01000a0 msgr2=0x7f0af01971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af6c3f640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af01000a0 0x7f0af01971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af6c3f640 1 -- 172.21.15.60:0/2648910260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0009560 con 0x7f0af0101c50 2024-02-26T04:57:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.145+0000 7f0af5c3d640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af0101290 0x7f0af01976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.146+0000 7f0af6c3f640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101c50 0x7f0af019e750 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800cad0 tx=0x7f0ae800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.146+0000 7f0ae77fe640 1 -- 172.21.15.60:0/2648910260 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800ae20 con 0x7f0af0101c50 2024-02-26T04:57:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.146+0000 7f0ae77fe640 1 -- 172.21.15.60:0/2648910260 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ae8005020 con 0x7f0af0101c50 2024-02-26T04:57:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.146+0000 7f0ae77fe640 1 -- 172.21.15.60:0/2648910260 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800f6b0 con 0x7f0af0101c50 2024-02-26T04:57:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.146+0000 7f0af86a2640 1 -- 172.21.15.60:0/2648910260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af019f2a0 con 0x7f0af0101c50 2024-02-26T04:57:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.146+0000 7f0af86a2640 1 -- 172.21.15.60:0/2648910260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af019f800 con 0x7f0af0101c50 2024-02-26T04:57:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.147+0000 7f0af86a2640 1 -- 172.21.15.60:0/2648910260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af0107360 con 0x7f0af0101c50 2024-02-26T04:57:44.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.148+0000 7f0ae77fe640 1 -- 172.21.15.60:0/2648910260 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0ae8005520 con 0x7f0af0101c50 2024-02-26T04:57:44.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.148+0000 7f0ae77fe640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0adc0a2140 0x7f0adc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:44.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.148+0000 7f0af643e640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0adc0a2140 0x7f0adc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:44.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.149+0000 7f0ae77fe640 1 -- 172.21.15.60:0/2648910260 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f0ae80ce510 con 0x7f0af0101c50 2024-02-26T04:57:44.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.149+0000 7f0af643e640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0adc0a2140 0x7f0adc0a4600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0058c0 tx=0x7f0aec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:44.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.152+0000 7f0ae77fe640 1 -- 172.21.15.60:0/2648910260 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ae80970c0 con 0x7f0af0101c50 2024-02-26T04:57:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.337+0000 7f0af86a2640 1 -- 172.21.15.60:0/2648910260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0af002ce40 con 0x7f0af0101c50 2024-02-26T04:57:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.338+0000 7f0ae77fe640 1 -- 172.21.15.60:0/2648910260 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0ae801d020 con 0x7f0af0101c50 2024-02-26T04:57:44.340 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:57:44.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.340+0000 7f0af86a2640 1 -- 172.21.15.60:0/2648910260 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0adc0a2140 msgr2=0x7f0adc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.340+0000 7f0af86a2640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0adc0a2140 0x7f0adc0a4600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0058c0 tx=0x7f0aec005850 comp rx=0 tx=0).stop 2024-02-26T04:57:44.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.340+0000 7f0af86a2640 1 -- 172.21.15.60:0/2648910260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101c50 msgr2=0x7f0af019e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.340+0000 7f0af86a2640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101c50 0x7f0af019e750 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800cad0 tx=0x7f0ae800cf90 comp rx=0 tx=0).stop 2024-02-26T04:57:44.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.340+0000 7f0af86a2640 1 -- 172.21.15.60:0/2648910260 shutdown_connections 2024-02-26T04:57:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.340+0000 7f0af86a2640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0adc0a2140 0x7f0adc0a4600 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.340+0000 7f0af86a2640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101c50 0x7f0af019e750 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.340+0000 7f0af86a2640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af0101290 0x7f0af01976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.340+0000 7f0af86a2640 1 --2- 172.21.15.60:0/2648910260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af01000a0 0x7f0af01971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.340+0000 7f0af86a2640 1 -- 172.21.15.60:0/2648910260 >> 172.21.15.60:0/2648910260 conn(0x7f0af00fb630 msgr2=0x7f0af0104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.340+0000 7f0af86a2640 1 -- 172.21.15.60:0/2648910260 shutdown_connections 2024-02-26T04:57:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.340+0000 7f0af86a2640 1 -- 172.21.15.60:0/2648910260 wait complete. 2024-02-26T04:57:44.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:44 smithi060 ceph-mon[32382]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:57:44.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:44 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1822269747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:44.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:44 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3088510676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:44.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:44 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1473382078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:44.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:57:44.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.453+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3620933296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e14106f00 msgr2=0x7f2e14109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.453+0000 7f2e1bb8c640 1 --2- 172.21.15.60:0/3620933296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e14106f00 0x7f2e14109350 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009ca0 tx=0x7f2e0c02c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.453+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3620933296 shutdown_connections 2024-02-26T04:57:44.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.453+0000 7f2e1bb8c640 1 --2- 172.21.15.60:0/3620933296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e14106f00 0x7f2e14109350 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.453+0000 7f2e1bb8c640 1 --2- 172.21.15.60:0/3620933296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e141045d0 0x7f2e141069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.453+0000 7f2e1bb8c640 1 --2- 172.21.15.60:0/3620933296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e14101ca0 0x7f2e14104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.453+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3620933296 >> 172.21.15.60:0/3620933296 conn(0x7f2e140fb6b0 msgr2=0x7f2e140fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:44.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.453+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3620933296 shutdown_connections 2024-02-26T04:57:44.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.453+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3620933296 wait complete. 2024-02-26T04:57:44.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.454+0000 7f2e1bb8c640 1 Processor -- start 2024-02-26T04:57:44.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.454+0000 7f2e1bb8c640 1 -- start start 2024-02-26T04:57:44.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.454+0000 7f2e1bb8c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e14101ca0 0x7f2e14101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:44.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.454+0000 7f2e1bb8c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e141045d0 0x7f2e140ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:44.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.454+0000 7f2e1bb8c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e14106f00 0x7f2e14100190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:44.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.454+0000 7f2e1bb8c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e141008f0 con 0x7f2e14106f00 2024-02-26T04:57:44.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.454+0000 7f2e1bb8c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e14100a60 con 0x7f2e141045d0 2024-02-26T04:57:44.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.454+0000 7f2e1bb8c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e141a11b0 con 0x7f2e14101ca0 2024-02-26T04:57:44.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.454+0000 7f2e1a129640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e14106f00 0x7f2e14100190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:44.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.454+0000 7f2e1a129640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e14106f00 0x7f2e14100190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37944/0 (socket says 172.21.15.60:37944) 2024-02-26T04:57:44.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.454+0000 7f2e1a129640 1 -- 172.21.15.60:0/3181639540 learned_addr learned my addr 172.21.15.60:0/3181639540 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:44.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.454+0000 7f2e19928640 1 --2- 172.21.15.60:0/3181639540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e14101ca0 0x7f2e14101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:44.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.454+0000 7f2e19127640 1 --2- 172.21.15.60:0/3181639540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e141045d0 0x7f2e140ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:44.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.455+0000 7f2e1a129640 1 -- 172.21.15.60:0/3181639540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e14101ca0 msgr2=0x7f2e14101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.455+0000 7f2e1a129640 1 --2- 172.21.15.60:0/3181639540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e14101ca0 0x7f2e14101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.455+0000 7f2e1a129640 1 -- 172.21.15.60:0/3181639540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e141045d0 msgr2=0x7f2e140ffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.455+0000 7f2e1a129640 1 --2- 172.21.15.60:0/3181639540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e141045d0 0x7f2e140ffc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.455+0000 7f2e1a129640 1 -- 172.21.15.60:0/3181639540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c009900 con 0x7f2e14106f00 2024-02-26T04:57:44.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.455+0000 7f2e1a129640 1 --2- 172.21.15.60:0/3181639540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e14106f00 0x7f2e14100190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0098d0 tx=0x7f2e0c034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:44.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.455+0000 7f2e0affd640 1 -- 172.21.15.60:0/3181639540 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c032eb0 con 0x7f2e14106f00 2024-02-26T04:57:44.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.455+0000 7f2e0affd640 1 -- 172.21.15.60:0/3181639540 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e0c003e50 con 0x7f2e14106f00 2024-02-26T04:57:44.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.455+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3181639540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e141a1400 con 0x7f2e14106f00 2024-02-26T04:57:44.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.455+0000 7f2e0affd640 1 -- 172.21.15.60:0/3181639540 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c0406b0 con 0x7f2e14106f00 2024-02-26T04:57:44.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.455+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3181639540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e141a18b0 con 0x7f2e14106f00 2024-02-26T04:57:44.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.456+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3181639540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e140fd1a0 con 0x7f2e14106f00 2024-02-26T04:57:44.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.457+0000 7f2e0affd640 1 -- 172.21.15.60:0/3181639540 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2e0c00a040 con 0x7f2e14106f00 2024-02-26T04:57:44.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.457+0000 7f2e0affd640 1 --2- 172.21.15.60:0/3181639540 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2e000a20c0 0x7f2e000a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:44.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.458+0000 7f2e19928640 1 --2- 172.21.15.60:0/3181639540 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2e000a20c0 0x7f2e000a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:44.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.458+0000 7f2e0affd640 1 -- 172.21.15.60:0/3181639540 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f2e0c0f14b0 con 0x7f2e14106f00 2024-02-26T04:57:44.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.458+0000 7f2e19928640 1 --2- 172.21.15.60:0/3181639540 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2e000a20c0 0x7f2e000a4580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2e10005fd0 tx=0x7f2e1000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:44.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.461+0000 7f2e0affd640 1 -- 172.21.15.60:0/3181639540 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e0c0ba070 con 0x7f2e14106f00 2024-02-26T04:57:44.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:44 smithi156 ceph-mon[35805]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:57:44.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:44 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1822269747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:44.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:44 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3088510676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:44.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:44 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1473382078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:44.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.680+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3181639540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e1410cbc0 con 0x7f2e14106f00 2024-02-26T04:57:44.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.680+0000 7f2e0affd640 1 -- 172.21.15.60:0/3181639540 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11347 (secure 0 0 0) 0x7f2e0c0bd730 con 0x7f2e14106f00 2024-02-26T04:57:44.685 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:44.685 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:55:54.140330+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24476,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T04:57:44.686 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-02-26T04:57:44.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.682+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3181639540 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2e000a20c0 msgr2=0x7f2e000a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.682+0000 7f2e1bb8c640 1 --2- 172.21.15.60:0/3181639540 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2e000a20c0 0x7f2e000a4580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2e10005fd0 tx=0x7f2e1000b390 comp rx=0 tx=0).stop 2024-02-26T04:57:44.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.682+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3181639540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e14106f00 msgr2=0x7f2e14100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.682+0000 7f2e1bb8c640 1 --2- 172.21.15.60:0/3181639540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e14106f00 0x7f2e14100190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0098d0 tx=0x7f2e0c034d40 comp rx=0 tx=0).stop 2024-02-26T04:57:44.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.682+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3181639540 shutdown_connections 2024-02-26T04:57:44.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.682+0000 7f2e1bb8c640 1 --2- 172.21.15.60:0/3181639540 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2e000a20c0 0x7f2e000a4580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.682+0000 7f2e1bb8c640 1 --2- 172.21.15.60:0/3181639540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e14106f00 0x7f2e14100190 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.682+0000 7f2e1bb8c640 1 --2- 172.21.15.60:0/3181639540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e141045d0 0x7f2e140ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.682+0000 7f2e1bb8c640 1 --2- 172.21.15.60:0/3181639540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e14101ca0 0x7f2e14101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.682+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3181639540 >> 172.21.15.60:0/3181639540 conn(0x7f2e140fb6b0 msgr2=0x7f2e141025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:44.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.682+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3181639540 shutdown_connections 2024-02-26T04:57:44.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.682+0000 7f2e1bb8c640 1 -- 172.21.15.60:0/3181639540 wait complete. 2024-02-26T04:57:44.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:57:44.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:44 smithi191 ceph-mon[36352]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:57:44.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:44 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1822269747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:44.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:44 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3088510676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:44.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:44 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1473382078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:44.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.797+0000 7f0af60d5640 1 -- 172.21.15.60:0/4287718999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af00ff850 msgr2=0x7f0af00ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.797+0000 7f0af60d5640 1 --2- 172.21.15.60:0/4287718999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af00ff850 0x7f0af00ffc70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009900 tx=0x7f0ae002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:44.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.797+0000 7f0af60d5640 1 -- 172.21.15.60:0/4287718999 shutdown_connections 2024-02-26T04:57:44.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.797+0000 7f0af60d5640 1 --2- 172.21.15.60:0/4287718999 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af0101400 0x7f0af0107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.797+0000 7f0af60d5640 1 --2- 172.21.15.60:0/4287718999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af0100a40 0x7f0af0100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.797+0000 7f0af60d5640 1 --2- 172.21.15.60:0/4287718999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af00ff850 0x7f0af00ffc70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.797+0000 7f0af60d5640 1 -- 172.21.15.60:0/4287718999 >> 172.21.15.60:0/4287718999 conn(0x7f0af00fadc0 msgr2=0x7f0af00fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.797+0000 7f0af60d5640 1 -- 172.21.15.60:0/4287718999 shutdown_connections 2024-02-26T04:57:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.797+0000 7f0af60d5640 1 -- 172.21.15.60:0/4287718999 wait complete. 2024-02-26T04:57:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.797+0000 7f0af60d5640 1 Processor -- start 2024-02-26T04:57:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0af60d5640 1 -- start start 2024-02-26T04:57:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0af60d5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af00ff850 0x7f0af010afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0af60d5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af0100a40 0x7f0af01095e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0af60d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101400 0x7f0af0109b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0af60d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af010a280 con 0x7f0af0101400 2024-02-26T04:57:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0af60d5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af010a3f0 con 0x7f0af0100a40 2024-02-26T04:57:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0af60d5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af01a12c0 con 0x7f0af00ff850 2024-02-26T04:57:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0aeffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101400 0x7f0af0109b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0aeffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101400 0x7f0af0109b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37966/0 (socket says 172.21.15.60:37966) 2024-02-26T04:57:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0aeffff640 1 -- 172.21.15.60:0/879398441 learned_addr learned my addr 172.21.15.60:0/879398441 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0aef7fe640 1 --2- 172.21.15.60:0/879398441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af00ff850 0x7f0af010afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0aeeffd640 1 --2- 172.21.15.60:0/879398441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af0100a40 0x7f0af01095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0aeffff640 1 -- 172.21.15.60:0/879398441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af00ff850 msgr2=0x7f0af010afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0aeffff640 1 --2- 172.21.15.60:0/879398441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af00ff850 0x7f0af010afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0aeffff640 1 -- 172.21.15.60:0/879398441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af0100a40 msgr2=0x7f0af01095e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0aeffff640 1 --2- 172.21.15.60:0/879398441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af0100a40 0x7f0af01095e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0aeffff640 1 -- 172.21.15.60:0/879398441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0009560 con 0x7f0af0101400 2024-02-26T04:57:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.798+0000 7f0aeffff640 1 --2- 172.21.15.60:0/879398441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101400 0x7f0af0109b20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800ead0 tx=0x7f0ae800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.799+0000 7f0aecff9640 1 -- 172.21.15.60:0/879398441 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800cde0 con 0x7f0af0101400 2024-02-26T04:57:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.799+0000 7f0aecff9640 1 -- 172.21.15.60:0/879398441 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ae8005020 con 0x7f0af0101400 2024-02-26T04:57:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.799+0000 7f0af60d5640 1 -- 172.21.15.60:0/879398441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af01a1570 con 0x7f0af0101400 2024-02-26T04:57:44.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.799+0000 7f0aecff9640 1 -- 172.21.15.60:0/879398441 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae80116b0 con 0x7f0af0101400 2024-02-26T04:57:44.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.799+0000 7f0af60d5640 1 -- 172.21.15.60:0/879398441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af01a1a80 con 0x7f0af0101400 2024-02-26T04:57:44.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.800+0000 7f0af60d5640 1 -- 172.21.15.60:0/879398441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af0106b70 con 0x7f0af0101400 2024-02-26T04:57:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.801+0000 7f0aecff9640 1 -- 172.21.15.60:0/879398441 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0ae8005520 con 0x7f0af0101400 2024-02-26T04:57:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.801+0000 7f0aecff9640 1 --2- 172.21.15.60:0/879398441 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0ad80a20c0 0x7f0ad80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.801+0000 7f0aef7fe640 1 --2- 172.21.15.60:0/879398441 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0ad80a20c0 0x7f0ad80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.801+0000 7f0aecff9640 1 -- 172.21.15.60:0/879398441 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f0ae80ce200 con 0x7f0af0101400 2024-02-26T04:57:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.802+0000 7f0aef7fe640 1 --2- 172.21.15.60:0/879398441 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0ad80a20c0 0x7f0ad80a4580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0ae002c240 tx=0x7f0ae002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:44.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.805+0000 7f0aecff9640 1 -- 172.21.15.60:0/879398441 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ae8096e40 con 0x7f0af0101400 2024-02-26T04:57:44.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.988+0000 7f0af60d5640 1 -- 172.21.15.60:0/879398441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0af002ce40 con 0x7f0af0101400 2024-02-26T04:57:44.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.989+0000 7f0aecff9640 1 -- 172.21.15.60:0/879398441 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24972 (secure 0 0 0) 0x7f0ae801f090 con 0x7f0af0101400 2024-02-26T04:57:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:44.992 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:57:21.168776+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:57:44.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.991+0000 7f0af60d5640 1 -- 172.21.15.60:0/879398441 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0ad80a20c0 msgr2=0x7f0ad80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.991+0000 7f0af60d5640 1 --2- 172.21.15.60:0/879398441 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0ad80a20c0 0x7f0ad80a4580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0ae002c240 tx=0x7f0ae002c7e0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.991+0000 7f0af60d5640 1 -- 172.21.15.60:0/879398441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101400 msgr2=0x7f0af0109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:44.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.991+0000 7f0af60d5640 1 --2- 172.21.15.60:0/879398441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101400 0x7f0af0109b20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800ead0 tx=0x7f0ae800ef90 comp rx=0 tx=0).stop 2024-02-26T04:57:44.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.991+0000 7f0af60d5640 1 -- 172.21.15.60:0/879398441 shutdown_connections 2024-02-26T04:57:44.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.991+0000 7f0af60d5640 1 --2- 172.21.15.60:0/879398441 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0ad80a20c0 0x7f0ad80a4580 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.991+0000 7f0af60d5640 1 --2- 172.21.15.60:0/879398441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0101400 0x7f0af0109b20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.991+0000 7f0af60d5640 1 --2- 172.21.15.60:0/879398441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af0100a40 0x7f0af01095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.991+0000 7f0af60d5640 1 --2- 172.21.15.60:0/879398441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af00ff850 0x7f0af010afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:44.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.991+0000 7f0af60d5640 1 -- 172.21.15.60:0/879398441 >> 172.21.15.60:0/879398441 conn(0x7f0af00fadc0 msgr2=0x7f0af0103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:44.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.992+0000 7f0af60d5640 1 -- 172.21.15.60:0/879398441 shutdown_connections 2024-02-26T04:57:44.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:44.992+0000 7f0af60d5640 1 -- 172.21.15.60:0/879398441 wait complete. 2024-02-26T04:57:45.005 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T04:50:44.572921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T04:55:25.530720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T04:55:26.741604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T04:55:30.453569+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T04:57:45.006 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T04:57:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.110+0000 7fcf65e2d640 1 -- 172.21.15.60:0/306435429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf601000a0 msgr2=0x7fcf601004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.110+0000 7fcf65e2d640 1 --2- 172.21.15.60:0/306435429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf601000a0 0x7fcf601004c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009ca0 tx=0x7fcf5002c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.110+0000 7fcf65e2d640 1 -- 172.21.15.60:0/306435429 shutdown_connections 2024-02-26T04:57:45.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.110+0000 7fcf65e2d640 1 --2- 172.21.15.60:0/306435429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf60101c50 0x7fcf601084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.110+0000 7fcf65e2d640 1 --2- 172.21.15.60:0/306435429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf60101290 0x7fcf60101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.110+0000 7fcf65e2d640 1 --2- 172.21.15.60:0/306435429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf601000a0 0x7fcf601004c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.110+0000 7fcf65e2d640 1 -- 172.21.15.60:0/306435429 >> 172.21.15.60:0/306435429 conn(0x7fcf600fb630 msgr2=0x7fcf600fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:45.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.110+0000 7fcf65e2d640 1 -- 172.21.15.60:0/306435429 shutdown_connections 2024-02-26T04:57:45.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.110+0000 7fcf65e2d640 1 -- 172.21.15.60:0/306435429 wait complete. 2024-02-26T04:57:45.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.110+0000 7fcf65e2d640 1 Processor -- start 2024-02-26T04:57:45.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf65e2d640 1 -- start start 2024-02-26T04:57:45.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf65e2d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf601000a0 0x7fcf6010a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf65e2d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf60101290 0x7fcf60108fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf65e2d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf60101c50 0x7fcf601094e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf65e2d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf60109c40 con 0x7fcf601000a0 2024-02-26T04:57:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf65e2d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf60109db0 con 0x7fcf60101290 2024-02-26T04:57:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf65e2d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf601a12b0 con 0x7fcf60101c50 2024-02-26T04:57:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf5f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf601000a0 0x7fcf6010a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf5f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf601000a0 0x7fcf6010a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37980/0 (socket says 172.21.15.60:37980) 2024-02-26T04:57:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf5f7fe640 1 -- 172.21.15.60:0/903608918 learned_addr learned my addr 172.21.15.60:0/903608918 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf5effd640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf60101290 0x7fcf60108fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf5ffff640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf60101c50 0x7fcf601094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf5f7fe640 1 -- 172.21.15.60:0/903608918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf60101c50 msgr2=0x7fcf601094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf5f7fe640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf60101c50 0x7fcf601094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf5f7fe640 1 -- 172.21.15.60:0/903608918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf60101290 msgr2=0x7fcf60108fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf5f7fe640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf60101290 0x7fcf60108fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf5f7fe640 1 -- 172.21.15.60:0/903608918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf50009900 con 0x7fcf601000a0 2024-02-26T04:57:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf5effd640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf60101290 0x7fcf60108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf5f7fe640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf601000a0 0x7fcf6010a960 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcf500348f0 tx=0x7fcf50034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.111+0000 7fcf5cff9640 1 -- 172.21.15.60:0/903608918 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50032eb0 con 0x7fcf601000a0 2024-02-26T04:57:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.112+0000 7fcf65e2d640 1 -- 172.21.15.60:0/903608918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf601a1500 con 0x7fcf601000a0 2024-02-26T04:57:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.112+0000 7fcf5cff9640 1 -- 172.21.15.60:0/903608918 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf50003e50 con 0x7fcf601000a0 2024-02-26T04:57:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.112+0000 7fcf5cff9640 1 -- 172.21.15.60:0/903608918 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf500406b0 con 0x7fcf601000a0 2024-02-26T04:57:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.112+0000 7fcf65e2d640 1 -- 172.21.15.60:0/903608918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf601a19b0 con 0x7fcf601000a0 2024-02-26T04:57:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.113+0000 7fcf65e2d640 1 -- 172.21.15.60:0/903608918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf60107440 con 0x7fcf601000a0 2024-02-26T04:57:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.113+0000 7fcf5cff9640 1 -- 172.21.15.60:0/903608918 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcf5000a040 con 0x7fcf601000a0 2024-02-26T04:57:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.114+0000 7fcf5cff9640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcf480a20c0 0x7fcf480a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.114+0000 7fcf5effd640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcf480a20c0 0x7fcf480a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.114+0000 7fcf5cff9640 1 -- 172.21.15.60:0/903608918 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fcf500f0fb0 con 0x7fcf601000a0 2024-02-26T04:57:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.115+0000 7fcf5effd640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcf480a20c0 0x7fcf480a4580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcf54005d00 tx=0x7fcf54005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:45.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.118+0000 7fcf5cff9640 1 -- 172.21.15.60:0/903608918 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcf500b9b40 con 0x7fcf601000a0 2024-02-26T04:57:45.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.300+0000 7fcf65e2d640 1 -- 172.21.15.60:0/903608918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf6010cbb0 con 0x7fcf601000a0 2024-02-26T04:57:45.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.300+0000 7fcf5cff9640 1 -- 172.21.15.60:0/903608918 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf6010cbb0 con 0x7fcf601000a0 2024-02-26T04:57:45.303 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T04:57:45.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.302+0000 7fcf65e2d640 1 -- 172.21.15.60:0/903608918 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcf480a20c0 msgr2=0x7fcf480a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.302+0000 7fcf65e2d640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcf480a20c0 0x7fcf480a4580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcf54005d00 tx=0x7fcf54005c70 comp rx=0 tx=0).stop 2024-02-26T04:57:45.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.302+0000 7fcf65e2d640 1 -- 172.21.15.60:0/903608918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf601000a0 msgr2=0x7fcf6010a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.302+0000 7fcf65e2d640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf601000a0 0x7fcf6010a960 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcf500348f0 tx=0x7fcf50034d40 comp rx=0 tx=0).stop 2024-02-26T04:57:45.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.302+0000 7fcf65e2d640 1 -- 172.21.15.60:0/903608918 shutdown_connections 2024-02-26T04:57:45.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.302+0000 7fcf65e2d640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcf480a20c0 0x7fcf480a4580 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.302+0000 7fcf65e2d640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf60101c50 0x7fcf601094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.302+0000 7fcf65e2d640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf60101290 0x7fcf60108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.302+0000 7fcf65e2d640 1 --2- 172.21.15.60:0/903608918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf601000a0 0x7fcf6010a960 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.302+0000 7fcf65e2d640 1 -- 172.21.15.60:0/903608918 >> 172.21.15.60:0/903608918 conn(0x7fcf600fb630 msgr2=0x7fcf60104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:45.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.302+0000 7fcf65e2d640 1 -- 172.21.15.60:0/903608918 shutdown_connections 2024-02-26T04:57:45.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.302+0000 7fcf65e2d640 1 -- 172.21.15.60:0/903608918 wait complete. 2024-02-26T04:57:45.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T04:57:45.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.418+0000 7fcdde31e640 1 -- 172.21.15.60:0/3652606781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd81013d0 msgr2=0x7fcdd8107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.418+0000 7fcdde31e640 1 --2- 172.21.15.60:0/3652606781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd81013d0 0x7fcdd8107c60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0009ca0 tx=0x7fcdd002c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.418+0000 7fcdde31e640 1 -- 172.21.15.60:0/3652606781 shutdown_connections 2024-02-26T04:57:45.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.418+0000 7fcdde31e640 1 --2- 172.21.15.60:0/3652606781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd81013d0 0x7fcdd8107c60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.418+0000 7fcdde31e640 1 --2- 172.21.15.60:0/3652606781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdd8100a10 0x7fcdd8100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.418+0000 7fcdde31e640 1 --2- 172.21.15.60:0/3652606781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdd80ff820 0x7fcdd80ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.418+0000 7fcdde31e640 1 -- 172.21.15.60:0/3652606781 >> 172.21.15.60:0/3652606781 conn(0x7fcdd80fad90 msgr2=0x7fcdd80fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.418+0000 7fcdde31e640 1 -- 172.21.15.60:0/3652606781 shutdown_connections 2024-02-26T04:57:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.418+0000 7fcdde31e640 1 -- 172.21.15.60:0/3652606781 wait complete. 2024-02-26T04:57:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.419+0000 7fcdde31e640 1 Processor -- start 2024-02-26T04:57:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.419+0000 7fcdde31e640 1 -- start start 2024-02-26T04:57:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.419+0000 7fcdde31e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd80ff820 0x7fcdd81971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.419+0000 7fcdde31e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdd8100a10 0x7fcdd8197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.419+0000 7fcdde31e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdd81013d0 0x7fcdd819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.419+0000 7fcdde31e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd819ece0 con 0x7fcdd80ff820 2024-02-26T04:57:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.419+0000 7fcdde31e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd819ee50 con 0x7fcdd81013d0 2024-02-26T04:57:45.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.419+0000 7fcdde31e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd819f010 con 0x7fcdd8100a10 2024-02-26T04:57:45.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.419+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd80ff820 0x7fcdd81971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:45.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.419+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd80ff820 0x7fcdd81971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38002/0 (socket says 172.21.15.60:38002) 2024-02-26T04:57:45.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.419+0000 7fcdd7fff640 1 -- 172.21.15.60:0/948888673 learned_addr learned my addr 172.21.15.60:0/948888673 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:45.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcdd77fe640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdd8100a10 0x7fcdd8197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:45.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcddc8bb640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdd81013d0 0x7fcdd819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:45.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcdd7fff640 1 -- 172.21.15.60:0/948888673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdd8100a10 msgr2=0x7fcdd8197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcdd7fff640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdd8100a10 0x7fcdd8197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcdd7fff640 1 -- 172.21.15.60:0/948888673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdd81013d0 msgr2=0x7fcdd819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcdd7fff640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdd81013d0 0x7fcdd819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcdd7fff640 1 -- 172.21.15.60:0/948888673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd0009900 con 0x7fcdd80ff820 2024-02-26T04:57:45.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcdd77fe640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdd8100a10 0x7fcdd8197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:45.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcddc8bb640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdd81013d0 0x7fcdd819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:45.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcdd7fff640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd80ff820 0x7fcdd81971f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00d950 tx=0x7fcdcc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:45.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcdd57fa640 1 -- 172.21.15.60:0/948888673 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc009c30 con 0x7fcdd80ff820 2024-02-26T04:57:45.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcdde31e640 1 -- 172.21.15.60:0/948888673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd819f2f0 con 0x7fcdd80ff820 2024-02-26T04:57:45.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcdd57fa640 1 -- 172.21.15.60:0/948888673 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdcc009d90 con 0x7fcdd80ff820 2024-02-26T04:57:45.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcdd57fa640 1 -- 172.21.15.60:0/948888673 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00f790 con 0x7fcdd80ff820 2024-02-26T04:57:45.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.420+0000 7fcdde31e640 1 -- 172.21.15.60:0/948888673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd819f850 con 0x7fcdd80ff820 2024-02-26T04:57:45.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.421+0000 7fcdde31e640 1 -- 172.21.15.60:0/948888673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd810a970 con 0x7fcdd80ff820 2024-02-26T04:57:45.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.422+0000 7fcdd57fa640 1 -- 172.21.15.60:0/948888673 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcdcc010ad0 con 0x7fcdd80ff820 2024-02-26T04:57:45.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.423+0000 7fcdd57fa640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcdc00a20c0 0x7fcdc00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:45.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.423+0000 7fcdd77fe640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcdc00a20c0 0x7fcdc00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:45.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.423+0000 7fcdd57fa640 1 -- 172.21.15.60:0/948888673 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fcdcc0ce450 con 0x7fcdd80ff820 2024-02-26T04:57:45.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.423+0000 7fcdd77fe640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcdc00a20c0 0x7fcdc00a4580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8198700 tx=0x7fcdc8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:45.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.427+0000 7fcdd57fa640 1 -- 172.21.15.60:0/948888673 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcdcc096f80 con 0x7fcdd80ff820 2024-02-26T04:57:45.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.607+0000 7fcdde31e640 1 -- 172.21.15.60:0/948888673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcdd810ab50 con 0x7fcdd80ff820 2024-02-26T04:57:45.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.608+0000 7fcdd57fa640 1 -- 172.21.15.60:0/948888673 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcdcc09a640 con 0x7fcdd80ff820 2024-02-26T04:57:45.610 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:57:45.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.609+0000 7fcdde31e640 1 -- 172.21.15.60:0/948888673 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcdc00a20c0 msgr2=0x7fcdc00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.609+0000 7fcdde31e640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcdc00a20c0 0x7fcdc00a4580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8198700 tx=0x7fcdc8005f90 comp rx=0 tx=0).stop 2024-02-26T04:57:45.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.609+0000 7fcdde31e640 1 -- 172.21.15.60:0/948888673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd80ff820 msgr2=0x7fcdd81971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.609+0000 7fcdde31e640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd80ff820 0x7fcdd81971f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00d950 tx=0x7fcdcc00de10 comp rx=0 tx=0).stop 2024-02-26T04:57:45.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.610+0000 7fcdde31e640 1 -- 172.21.15.60:0/948888673 shutdown_connections 2024-02-26T04:57:45.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.610+0000 7fcdde31e640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcdc00a20c0 0x7fcdc00a4580 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.610+0000 7fcdde31e640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdd81013d0 0x7fcdd819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.610+0000 7fcdde31e640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdd8100a10 0x7fcdd8197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.610+0000 7fcdde31e640 1 --2- 172.21.15.60:0/948888673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd80ff820 0x7fcdd81971f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.610+0000 7fcdde31e640 1 -- 172.21.15.60:0/948888673 >> 172.21.15.60:0/948888673 conn(0x7fcdd80fad90 msgr2=0x7fcdd8105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.610+0000 7fcdde31e640 1 -- 172.21.15.60:0/948888673 shutdown_connections 2024-02-26T04:57:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.610+0000 7fcdde31e640 1 -- 172.21.15.60:0/948888673 wait complete. 2024-02-26T04:57:45.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:45 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2648910260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:45.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:45 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3181639540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:45.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:45 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/879398441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:45.623 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T04:57:45.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:45 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2648910260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:45.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:45 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3181639540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:45.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:45 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/879398441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:45.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.726+0000 7f3782b1e640 1 -- 172.21.15.60:0/2964869930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377c101c30 msgr2=0x7f377c1084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.726+0000 7f3782b1e640 1 --2- 172.21.15.60:0/2964869930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377c101c30 0x7f377c1084c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3774009ca0 tx=0x7f377402c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.726+0000 7f3782b1e640 1 -- 172.21.15.60:0/2964869930 shutdown_connections 2024-02-26T04:57:45.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.726+0000 7f3782b1e640 1 --2- 172.21.15.60:0/2964869930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377c101c30 0x7f377c1084c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.726+0000 7f3782b1e640 1 --2- 172.21.15.60:0/2964869930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c101270 0x7f377c1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.726+0000 7f3782b1e640 1 --2- 172.21.15.60:0/2964869930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f377c100080 0x7f377c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.726+0000 7f3782b1e640 1 -- 172.21.15.60:0/2964869930 >> 172.21.15.60:0/2964869930 conn(0x7f377c0fb630 msgr2=0x7f377c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:45.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.726+0000 7f3782b1e640 1 -- 172.21.15.60:0/2964869930 shutdown_connections 2024-02-26T04:57:45.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.726+0000 7f3782b1e640 1 -- 172.21.15.60:0/2964869930 wait complete. 2024-02-26T04:57:45.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f3782b1e640 1 Processor -- start 2024-02-26T04:57:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f3782b1e640 1 -- start start 2024-02-26T04:57:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f3782b1e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c100080 0x7f377c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f3782b1e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f377c101270 0x7f377c19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f3782b1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377c101c30 0x7f377c1a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f3782b1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c1a0f80 con 0x7f377c101c30 2024-02-26T04:57:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f3782b1e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c1a10f0 con 0x7f377c100080 2024-02-26T04:57:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f3782b1e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c1a12b0 con 0x7f377c101270 2024-02-26T04:57:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f37810bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377c101c30 0x7f377c1a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f37810bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377c101c30 0x7f377c1a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38020/0 (socket says 172.21.15.60:38020) 2024-02-26T04:57:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f37810bb640 1 -- 172.21.15.60:0/2508180133 learned_addr learned my addr 172.21.15.60:0/2508180133 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f377bfff640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f377c101270 0x7f377c19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f37808ba640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c100080 0x7f377c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f37810bb640 1 -- 172.21.15.60:0/2508180133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f377c101270 msgr2=0x7f377c19e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f37810bb640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f377c101270 0x7f377c19e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f37810bb640 1 -- 172.21.15.60:0/2508180133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c100080 msgr2=0x7f377c06a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f37810bb640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c100080 0x7f377c06a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f37810bb640 1 -- 172.21.15.60:0/2508180133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3774009900 con 0x7f377c101c30 2024-02-26T04:57:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.727+0000 7f37808ba640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c100080 0x7f377c06a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.728+0000 7f37810bb640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377c101c30 0x7f377c1a08c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3774032650 tx=0x7f3774034b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.728+0000 7f3779ffb640 1 -- 172.21.15.60:0/2508180133 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37740329b0 con 0x7f377c101c30 2024-02-26T04:57:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.728+0000 7f3782b1e640 1 -- 172.21.15.60:0/2508180133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377c1a14d0 con 0x7f377c101c30 2024-02-26T04:57:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.728+0000 7f3779ffb640 1 -- 172.21.15.60:0/2508180133 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3774003e50 con 0x7f377c101c30 2024-02-26T04:57:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.728+0000 7f3779ffb640 1 -- 172.21.15.60:0/2508180133 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774040450 con 0x7f377c101c30 2024-02-26T04:57:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.728+0000 7f3782b1e640 1 -- 172.21.15.60:0/2508180133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377c1a1950 con 0x7f377c101c30 2024-02-26T04:57:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.729+0000 7f3782b1e640 1 -- 172.21.15.60:0/2508180133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377c107340 con 0x7f377c101c30 2024-02-26T04:57:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.730+0000 7f3779ffb640 1 -- 172.21.15.60:0/2508180133 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f377400a040 con 0x7f377c101c30 2024-02-26T04:57:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.731+0000 7f3779ffb640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f376c0a20f0 0x7f376c0a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.731+0000 7f3779ffb640 1 -- 172.21.15.60:0/2508180133 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f37740f1380 con 0x7f377c101c30 2024-02-26T04:57:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.731+0000 7f37808ba640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f376c0a20f0 0x7f376c0a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.731+0000 7f37808ba640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f376c0a20f0 0x7f376c0a45b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f377c1010d0 tx=0x7f3770005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:45.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.734+0000 7f3779ffb640 1 -- 172.21.15.60:0/2508180133 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f37740b9f10 con 0x7f377c101c30 2024-02-26T04:57:45.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:45 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2648910260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:45.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:45 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3181639540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:45.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:45 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/879398441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:45.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.916+0000 7f3782b1e640 1 -- 172.21.15.60:0/2508180133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f377c10bc90 con 0x7f377c101c30 2024-02-26T04:57:45.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.916+0000 7f3779ffb640 1 -- 172.21.15.60:0/2508180133 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f37740bd5d0 con 0x7f377c101c30 2024-02-26T04:57:45.919 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T04:57:45.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.918+0000 7f3782b1e640 1 -- 172.21.15.60:0/2508180133 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f376c0a20f0 msgr2=0x7f376c0a45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.918+0000 7f3782b1e640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f376c0a20f0 0x7f376c0a45b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f377c1010d0 tx=0x7f3770005f90 comp rx=0 tx=0).stop 2024-02-26T04:57:45.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.918+0000 7f3782b1e640 1 -- 172.21.15.60:0/2508180133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377c101c30 msgr2=0x7f377c1a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:45.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.918+0000 7f3782b1e640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377c101c30 0x7f377c1a08c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3774032650 tx=0x7f3774034b10 comp rx=0 tx=0).stop 2024-02-26T04:57:45.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.918+0000 7f3782b1e640 1 -- 172.21.15.60:0/2508180133 shutdown_connections 2024-02-26T04:57:45.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.918+0000 7f3782b1e640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f376c0a20f0 0x7f376c0a45b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.918+0000 7f3782b1e640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377c101c30 0x7f377c1a08c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.918+0000 7f3782b1e640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f377c101270 0x7f377c19e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.918+0000 7f3782b1e640 1 --2- 172.21.15.60:0/2508180133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c100080 0x7f377c06a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:45.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.918+0000 7f3782b1e640 1 -- 172.21.15.60:0/2508180133 >> 172.21.15.60:0/2508180133 conn(0x7f377c0fb630 msgr2=0x7f377c104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:45.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.918+0000 7f3782b1e640 1 -- 172.21.15.60:0/2508180133 shutdown_connections 2024-02-26T04:57:45.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:45.918+0000 7f3782b1e640 1 -- 172.21.15.60:0/2508180133 wait complete. 2024-02-26T04:57:45.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T04:57:46.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.035+0000 7f2282bf0640 1 -- 172.21.15.60:0/2793548144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f227c100360 msgr2=0x7f227c1007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.035+0000 7f2282bf0640 1 --2- 172.21.15.60:0/2793548144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f227c100360 0x7f227c1007e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2268009930 tx=0x7f226802bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:46.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.035+0000 7f2282bf0640 1 -- 172.21.15.60:0/2793548144 shutdown_connections 2024-02-26T04:57:46.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.035+0000 7f2282bf0640 1 --2- 172.21.15.60:0/2793548144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f227c100d20 0x7f227c1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.035+0000 7f2282bf0640 1 --2- 172.21.15.60:0/2793548144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f227c100360 0x7f227c1007e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.035+0000 7f2282bf0640 1 --2- 172.21.15.60:0/2793548144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f227c101630 0x7f227c101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.035+0000 7f2282bf0640 1 -- 172.21.15.60:0/2793548144 >> 172.21.15.60:0/2793548144 conn(0x7f227c0fad70 msgr2=0x7f227c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:46.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.035+0000 7f2282bf0640 1 -- 172.21.15.60:0/2793548144 shutdown_connections 2024-02-26T04:57:46.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.035+0000 7f2282bf0640 1 -- 172.21.15.60:0/2793548144 wait complete. 2024-02-26T04:57:46.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.035+0000 7f2282bf0640 1 Processor -- start 2024-02-26T04:57:46.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.035+0000 7f2282bf0640 1 -- start start 2024-02-26T04:57:46.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f2282bf0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f227c100360 0x7f227c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:46.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f2282bf0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f227c100d20 0x7f227c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:46.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f2282bf0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f227c101630 0x7f227c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:46.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f2282bf0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c19edb0 con 0x7f227c101630 2024-02-26T04:57:46.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f2282bf0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c19ef20 con 0x7f227c100d20 2024-02-26T04:57:46.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f2282bf0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c19f0e0 con 0x7f227c100360 2024-02-26T04:57:46.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f228118d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f227c101630 0x7f227c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:46.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f228118d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f227c101630 0x7f227c19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38032/0 (socket says 172.21.15.60:38032) 2024-02-26T04:57:46.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f228118d640 1 -- 172.21.15.60:0/2416391170 learned_addr learned my addr 172.21.15.60:0/2416391170 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:46.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f228098c640 1 --2- 172.21.15.60:0/2416391170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f227c100360 0x7f227c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:46.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f227bfff640 1 --2- 172.21.15.60:0/2416391170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f227c100d20 0x7f227c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:46.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f228118d640 1 -- 172.21.15.60:0/2416391170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f227c100360 msgr2=0x7f227c1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f228118d640 1 --2- 172.21.15.60:0/2416391170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f227c100360 0x7f227c1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f228118d640 1 -- 172.21.15.60:0/2416391170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f227c100d20 msgr2=0x7f227c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f228118d640 1 --2- 172.21.15.60:0/2416391170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f227c100d20 0x7f227c197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f228118d640 1 -- 172.21.15.60:0/2416391170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2268009560 con 0x7f227c101630 2024-02-26T04:57:46.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f228118d640 1 --2- 172.21.15.60:0/2416391170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f227c101630 0x7f227c19e870 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f227400eb10 tx=0x7f227400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:46.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f2279ffb640 1 -- 172.21.15.60:0/2416391170 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274004b20 con 0x7f227c101630 2024-02-26T04:57:46.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f2279ffb640 1 -- 172.21.15.60:0/2416391170 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2274004c80 con 0x7f227c101630 2024-02-26T04:57:46.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f2282bf0640 1 -- 172.21.15.60:0/2416391170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227c19f3c0 con 0x7f227c101630 2024-02-26T04:57:46.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.036+0000 7f2279ffb640 1 -- 172.21.15.60:0/2416391170 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274010650 con 0x7f227c101630 2024-02-26T04:57:46.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.037+0000 7f2282bf0640 1 -- 172.21.15.60:0/2416391170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227c19f850 con 0x7f227c101630 2024-02-26T04:57:46.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.038+0000 7f2282bf0640 1 -- 172.21.15.60:0/2416391170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227c1074f0 con 0x7f227c101630 2024-02-26T04:57:46.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.039+0000 7f2279ffb640 1 -- 172.21.15.60:0/2416391170 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2274005020 con 0x7f227c101630 2024-02-26T04:57:46.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.039+0000 7f2279ffb640 1 --2- 172.21.15.60:0/2416391170 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f226c0a20c0 0x7f226c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:46.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.039+0000 7f228098c640 1 --2- 172.21.15.60:0/2416391170 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f226c0a20c0 0x7f226c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:46.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.039+0000 7f2279ffb640 1 -- 172.21.15.60:0/2416391170 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f2274014070 con 0x7f227c101630 2024-02-26T04:57:46.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.040+0000 7f228098c640 1 --2- 172.21.15.60:0/2416391170 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f226c0a20c0 0x7f226c0a4580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2270005fd0 tx=0x7f227000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:46.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.044+0000 7f2279ffb640 1 -- 172.21.15.60:0/2416391170 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2274097210 con 0x7f227c101630 2024-02-26T04:57:46.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.227+0000 7f2282bf0640 1 -- 172.21.15.60:0/2416391170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f227c02ce40 con 0x7f227c101630 2024-02-26T04:57:46.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.228+0000 7f2279ffb640 1 -- 172.21.15.60:0/2416391170 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f227409a8d0 con 0x7f227c101630 2024-02-26T04:57:46.230 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:57:46.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.230+0000 7f2282bf0640 1 -- 172.21.15.60:0/2416391170 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f226c0a20c0 msgr2=0x7f226c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.230+0000 7f2282bf0640 1 --2- 172.21.15.60:0/2416391170 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f226c0a20c0 0x7f226c0a4580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2270005fd0 tx=0x7f227000b390 comp rx=0 tx=0).stop 2024-02-26T04:57:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.230+0000 7f2282bf0640 1 -- 172.21.15.60:0/2416391170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f227c101630 msgr2=0x7f227c19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.230+0000 7f2282bf0640 1 --2- 172.21.15.60:0/2416391170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f227c101630 0x7f227c19e870 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f227400eb10 tx=0x7f227400c2d0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.230+0000 7f2282bf0640 1 -- 172.21.15.60:0/2416391170 shutdown_connections 2024-02-26T04:57:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.230+0000 7f2282bf0640 1 --2- 172.21.15.60:0/2416391170 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f226c0a20c0 0x7f226c0a4580 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.230+0000 7f2282bf0640 1 --2- 172.21.15.60:0/2416391170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f227c101630 0x7f227c19e870 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.230+0000 7f2282bf0640 1 --2- 172.21.15.60:0/2416391170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f227c100d20 0x7f227c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.230+0000 7f2282bf0640 1 --2- 172.21.15.60:0/2416391170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f227c100360 0x7f227c1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.230+0000 7f2282bf0640 1 -- 172.21.15.60:0/2416391170 >> 172.21.15.60:0/2416391170 conn(0x7f227c0fad70 msgr2=0x7f227c1046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.230+0000 7f2282bf0640 1 -- 172.21.15.60:0/2416391170 shutdown_connections 2024-02-26T04:57:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.230+0000 7f2282bf0640 1 -- 172.21.15.60:0/2416391170 wait complete. 2024-02-26T04:57:46.244 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:57:46.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.344+0000 7f0768515640 1 -- 172.21.15.60:0/3412827588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0760102390 msgr2=0x7f0760102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.344+0000 7f0768515640 1 --2- 172.21.15.60:0/3412827588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0760102390 0x7f0760102810 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0750009900 tx=0x7f075002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:46.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.344+0000 7f0768515640 1 -- 172.21.15.60:0/3412827588 shutdown_connections 2024-02-26T04:57:46.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.344+0000 7f0768515640 1 --2- 172.21.15.60:0/3412827588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0760102d50 0x7f07601095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.344+0000 7f0768515640 1 --2- 172.21.15.60:0/3412827588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0760102390 0x7f0760102810 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.344+0000 7f0768515640 1 --2- 172.21.15.60:0/3412827588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07601011a0 0x7f07601015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.344+0000 7f0768515640 1 -- 172.21.15.60:0/3412827588 >> 172.21.15.60:0/3412827588 conn(0x7f07600fc730 msgr2=0x7f07600feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.344+0000 7f0768515640 1 -- 172.21.15.60:0/3412827588 shutdown_connections 2024-02-26T04:57:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.344+0000 7f0768515640 1 -- 172.21.15.60:0/3412827588 wait complete. 2024-02-26T04:57:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.345+0000 7f0768515640 1 Processor -- start 2024-02-26T04:57:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.345+0000 7f0768515640 1 -- start start 2024-02-26T04:57:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.345+0000 7f0768515640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07601011a0 0x7f0760074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.345+0000 7f0768515640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0760102390 0x7f0760073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.345+0000 7f0768515640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0760102d50 0x7f0760073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.345+0000 7f0768515640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0760074210 con 0x7f07601011a0 2024-02-26T04:57:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.345+0000 7f0768515640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0760074380 con 0x7f0760102d50 2024-02-26T04:57:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.345+0000 7f0768515640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07601a1310 con 0x7f0760102390 2024-02-26T04:57:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.345+0000 7f07662b1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07601011a0 0x7f0760074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.345+0000 7f07662b1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07601011a0 0x7f0760074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38048/0 (socket says 172.21.15.60:38048) 2024-02-26T04:57:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.345+0000 7f07662b1640 1 -- 172.21.15.60:0/2586465443 learned_addr learned my addr 172.21.15.60:0/2586465443 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f0766ab2640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0760102d50 0x7f0760073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f0765ab0640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0760102390 0x7f0760073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f07662b1640 1 -- 172.21.15.60:0/2586465443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0760102390 msgr2=0x7f0760073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f07662b1640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0760102390 0x7f0760073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f07662b1640 1 -- 172.21.15.60:0/2586465443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0760102d50 msgr2=0x7f0760073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f07662b1640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0760102d50 0x7f0760073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f07662b1640 1 -- 172.21.15.60:0/2586465443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0750009560 con 0x7f07601011a0 2024-02-26T04:57:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f0766ab2640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0760102d50 0x7f0760073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f0765ab0640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0760102390 0x7f0760073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f07662b1640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07601011a0 0x7f0760074f30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f075c00c9a0 tx=0x7f075c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f07577fe640 1 -- 172.21.15.60:0/2586465443 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c004220 con 0x7f07601011a0 2024-02-26T04:57:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f07577fe640 1 -- 172.21.15.60:0/2586465443 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f075c007e90 con 0x7f07601011a0 2024-02-26T04:57:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f07577fe640 1 -- 172.21.15.60:0/2586465443 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c004a00 con 0x7f07601011a0 2024-02-26T04:57:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f0768515640 1 -- 172.21.15.60:0/2586465443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07601a15c0 con 0x7f07601011a0 2024-02-26T04:57:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.346+0000 7f0768515640 1 -- 172.21.15.60:0/2586465443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07601a1b00 con 0x7f07601011a0 2024-02-26T04:57:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.347+0000 7f0768515640 1 -- 172.21.15.60:0/2586465443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07601084c0 con 0x7f07601011a0 2024-02-26T04:57:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.348+0000 7f07577fe640 1 -- 172.21.15.60:0/2586465443 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f075c004380 con 0x7f07601011a0 2024-02-26T04:57:46.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.349+0000 7f07577fe640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f074c0a20c0 0x7f074c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:46.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.349+0000 7f0765ab0640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f074c0a20c0 0x7f074c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:46.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.349+0000 7f07577fe640 1 -- 172.21.15.60:0/2586465443 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f075c0ce280 con 0x7f07601011a0 2024-02-26T04:57:46.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.349+0000 7f0765ab0640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f074c0a20c0 0x7f074c0a4580 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0760074ca0 tx=0x7f075002c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:46.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.353+0000 7f07577fe640 1 -- 172.21.15.60:0/2586465443 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f075c096db0 con 0x7f07601011a0 2024-02-26T04:57:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.573+0000 7f0768515640 1 -- 172.21.15.60:0/2586465443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f076010cc10 con 0x7f07601011a0 2024-02-26T04:57:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.574+0000 7f07577fe640 1 -- 172.21.15.60:0/2586465443 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11347 (secure 0 0 0) 0x7f075c09a470 con 0x7f07601011a0 2024-02-26T04:57:46.576 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:46.577 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:55:54.140330+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24476,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T04:57:46.577 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-02-26T04:57:46.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.576+0000 7f0768515640 1 -- 172.21.15.60:0/2586465443 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f074c0a20c0 msgr2=0x7f074c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.576+0000 7f0768515640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f074c0a20c0 0x7f074c0a4580 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0760074ca0 tx=0x7f075002c800 comp rx=0 tx=0).stop 2024-02-26T04:57:46.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.576+0000 7f0768515640 1 -- 172.21.15.60:0/2586465443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07601011a0 msgr2=0x7f0760074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.576+0000 7f0768515640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07601011a0 0x7f0760074f30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f075c00c9a0 tx=0x7f075c00ce60 comp rx=0 tx=0).stop 2024-02-26T04:57:46.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.576+0000 7f0768515640 1 -- 172.21.15.60:0/2586465443 shutdown_connections 2024-02-26T04:57:46.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.576+0000 7f0768515640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f074c0a20c0 0x7f074c0a4580 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.576+0000 7f0768515640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0760102d50 0x7f0760073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.576+0000 7f0768515640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0760102390 0x7f0760073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.576+0000 7f0768515640 1 --2- 172.21.15.60:0/2586465443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07601011a0 0x7f0760074f30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.576+0000 7f0768515640 1 -- 172.21.15.60:0/2586465443 >> 172.21.15.60:0/2586465443 conn(0x7f07600fc730 msgr2=0x7f0760105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:46.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.576+0000 7f0768515640 1 -- 172.21.15.60:0/2586465443 shutdown_connections 2024-02-26T04:57:46.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.576+0000 7f0768515640 1 -- 172.21.15.60:0/2586465443 wait complete. 2024-02-26T04:57:46.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:46 smithi060 ceph-mon[32382]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:46.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:46 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/903608918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:46.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:46 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/948888673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:46.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:46 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2508180133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:46.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:46 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2416391170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:46.589 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:57:46.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:46 smithi156 ceph-mon[35805]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:46.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:46 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/903608918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:46.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:46 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/948888673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:46.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:46 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2508180133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:46.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:46 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2416391170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.689+0000 7f48ee436640 1 -- 172.21.15.60:0/517411985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e8102390 msgr2=0x7f48e8102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.689+0000 7f48ee436640 1 --2- 172.21.15.60:0/517411985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e8102390 0x7f48e8102810 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009900 tx=0x7f48dc02bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.689+0000 7f48ee436640 1 -- 172.21.15.60:0/517411985 shutdown_connections 2024-02-26T04:57:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.689+0000 7f48ee436640 1 --2- 172.21.15.60:0/517411985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48e8102d50 0x7f48e81095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.689+0000 7f48ee436640 1 --2- 172.21.15.60:0/517411985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e8102390 0x7f48e8102810 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.689+0000 7f48ee436640 1 --2- 172.21.15.60:0/517411985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e81011a0 0x7f48e81015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.689+0000 7f48ee436640 1 -- 172.21.15.60:0/517411985 >> 172.21.15.60:0/517411985 conn(0x7f48e80fc730 msgr2=0x7f48e80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.689+0000 7f48ee436640 1 -- 172.21.15.60:0/517411985 shutdown_connections 2024-02-26T04:57:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.689+0000 7f48ee436640 1 -- 172.21.15.60:0/517411985 wait complete. 2024-02-26T04:57:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48ee436640 1 Processor -- start 2024-02-26T04:57:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48ee436640 1 -- start start 2024-02-26T04:57:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48ee436640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e81011a0 0x7f48e81971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48ee436640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8102390 0x7f48e81976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48ee436640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48e8102d50 0x7f48e819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48ee436640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e819ec90 con 0x7f48e81011a0 2024-02-26T04:57:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48ee436640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e819ee00 con 0x7f48e8102390 2024-02-26T04:57:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48ee436640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e819efc0 con 0x7f48e8102d50 2024-02-26T04:57:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48e7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e81011a0 0x7f48e81971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48e7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e81011a0 0x7f48e81971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38054/0 (socket says 172.21.15.60:38054) 2024-02-26T04:57:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48e7fff640 1 -- 172.21.15.60:0/2313792251 learned_addr learned my addr 172.21.15.60:0/2313792251 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48e77fe640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8102390 0x7f48e81976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48e7fff640 1 -- 172.21.15.60:0/2313792251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48e8102d50 msgr2=0x7f48e819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48ec9d3640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48e8102d50 0x7f48e819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48e7fff640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48e8102d50 0x7f48e819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48e7fff640 1 -- 172.21.15.60:0/2313792251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8102390 msgr2=0x7f48e81976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48e7fff640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8102390 0x7f48e81976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48e7fff640 1 -- 172.21.15.60:0/2313792251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc009560 con 0x7f48e81011a0 2024-02-26T04:57:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48ec9d3640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48e8102d50 0x7f48e819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48e77fe640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8102390 0x7f48e81976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.690+0000 7f48e7fff640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e81011a0 0x7f48e81971a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f48d800d770 tx=0x7f48d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.691+0000 7f48e57fa640 1 -- 172.21.15.60:0/2313792251 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8009c30 con 0x7f48e81011a0 2024-02-26T04:57:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.691+0000 7f48ee436640 1 -- 172.21.15.60:0/2313792251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e819f2a0 con 0x7f48e81011a0 2024-02-26T04:57:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.691+0000 7f48e57fa640 1 -- 172.21.15.60:0/2313792251 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48d800de80 con 0x7f48e81011a0 2024-02-26T04:57:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.691+0000 7f48e57fa640 1 -- 172.21.15.60:0/2313792251 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8010460 con 0x7f48e81011a0 2024-02-26T04:57:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.691+0000 7f48ee436640 1 -- 172.21.15.60:0/2313792251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e819f7b0 con 0x7f48e81011a0 2024-02-26T04:57:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.692+0000 7f48ee436640 1 -- 172.21.15.60:0/2313792251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e81084c0 con 0x7f48e81011a0 2024-02-26T04:57:46.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.693+0000 7f48e57fa640 1 -- 172.21.15.60:0/2313792251 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f48d8003f60 con 0x7f48e81011a0 2024-02-26T04:57:46.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.693+0000 7f48e57fa640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48d00a20c0 0x7f48d00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:46.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.693+0000 7f48e57fa640 1 -- 172.21.15.60:0/2313792251 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f48d80cdec0 con 0x7f48e81011a0 2024-02-26T04:57:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.693+0000 7f48e77fe640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48d00a20c0 0x7f48d00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.694+0000 7f48e77fe640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48d00a20c0 0x7f48d00a4580 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f48e81986b0 tx=0x7f48dc005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.697+0000 7f48e57fa640 1 -- 172.21.15.60:0/2313792251 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f48d8096a50 con 0x7f48e81011a0 2024-02-26T04:57:46.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:46 smithi191 ceph-mon[36352]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:46.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:46 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/903608918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:46.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:46 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/948888673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:46.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:46 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2508180133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:46.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:46 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2416391170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:46.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.879+0000 7f48ee436640 1 -- 172.21.15.60:0/2313792251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48e810ccd0 con 0x7f48e81011a0 2024-02-26T04:57:46.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.881+0000 7f48e57fa640 1 -- 172.21.15.60:0/2313792251 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24972 (secure 0 0 0) 0x7f48d8017090 con 0x7f48e81011a0 2024-02-26T04:57:46.883 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:46.883 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:57:21.168776+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:57:46.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.883+0000 7f48ee436640 1 -- 172.21.15.60:0/2313792251 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48d00a20c0 msgr2=0x7f48d00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.883+0000 7f48ee436640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48d00a20c0 0x7f48d00a4580 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f48e81986b0 tx=0x7f48dc005be0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.883+0000 7f48ee436640 1 -- 172.21.15.60:0/2313792251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e81011a0 msgr2=0x7f48e81971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:46.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.883+0000 7f48ee436640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e81011a0 0x7f48e81971a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f48d800d770 tx=0x7f48d800dc30 comp rx=0 tx=0).stop 2024-02-26T04:57:46.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.883+0000 7f48ee436640 1 -- 172.21.15.60:0/2313792251 shutdown_connections 2024-02-26T04:57:46.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.883+0000 7f48ee436640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48d00a20c0 0x7f48d00a4580 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.883+0000 7f48ee436640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48e8102d50 0x7f48e819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.883+0000 7f48ee436640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48e8102390 0x7f48e81976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.883+0000 7f48ee436640 1 --2- 172.21.15.60:0/2313792251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e81011a0 0x7f48e81971a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:46.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.883+0000 7f48ee436640 1 -- 172.21.15.60:0/2313792251 >> 172.21.15.60:0/2313792251 conn(0x7f48e80fc730 msgr2=0x7f48e8105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:46.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.883+0000 7f48ee436640 1 -- 172.21.15.60:0/2313792251 shutdown_connections 2024-02-26T04:57:46.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.883+0000 7f48ee436640 1 -- 172.21.15.60:0/2313792251 wait complete. 2024-02-26T04:57:46.897 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-26T04:55:27.553789+0000', 'modified': '2024-02-26T04:55:54.140330+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24524, 'mds_1': 24476, 'mds_2': 24472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6835/1642858086', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 1642858086}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 1642858086}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/2819944290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 2819944290}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 2819944290}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6837/3000497710', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 3000497710}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 3000497710}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.191:6839/640686517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6838', 'nonce': 640686517}, {'type': 'v1', 'addr': '172.21.15.191:6839', 'nonce': 640686517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.156:6837/707575792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 707575792}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 707575792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.156:6839/1768629491', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 1768629491}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 1768629491}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-26T04:57:46.897 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-02-26T04:57:46.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-26T04:57:47.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.999+0000 7f242c6c8640 1 -- 172.21.15.60:0/364138827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2424101270 msgr2=0x7f24241016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.999+0000 7f242c6c8640 1 --2- 172.21.15.60:0/364138827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2424101270 0x7f24241016f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2414009900 tx=0x7f241402bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:47.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.999+0000 7f242c6c8640 1 -- 172.21.15.60:0/364138827 shutdown_connections 2024-02-26T04:57:47.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.999+0000 7f242c6c8640 1 --2- 172.21.15.60:0/364138827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2424101c30 0x7f24241084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.999+0000 7f242c6c8640 1 --2- 172.21.15.60:0/364138827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2424101270 0x7f24241016f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.999+0000 7f242c6c8640 1 --2- 172.21.15.60:0/364138827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2424100080 0x7f24241004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.999+0000 7f242c6c8640 1 -- 172.21.15.60:0/364138827 >> 172.21.15.60:0/364138827 conn(0x7f24240fb610 msgr2=0x7f24240fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.999+0000 7f242c6c8640 1 -- 172.21.15.60:0/364138827 shutdown_connections 2024-02-26T04:57:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:46.999+0000 7f242c6c8640 1 -- 172.21.15.60:0/364138827 wait complete. 2024-02-26T04:57:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242c6c8640 1 Processor -- start 2024-02-26T04:57:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242c6c8640 1 -- start start 2024-02-26T04:57:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242c6c8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2424100080 0x7f2424197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242c6c8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2424101270 0x7f24241977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242c6c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2424101c30 0x7f242419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242c6c8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242419ed80 con 0x7f2424101c30 2024-02-26T04:57:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242c6c8640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242419eef0 con 0x7f2424100080 2024-02-26T04:57:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242c6c8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242419f0b0 con 0x7f2424101270 2024-02-26T04:57:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242ac65640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2424101c30 0x7f242419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242ac65640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2424101c30 0x7f242419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38070/0 (socket says 172.21.15.60:38070) 2024-02-26T04:57:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242ac65640 1 -- 172.21.15.60:0/3084981588 learned_addr learned my addr 172.21.15.60:0/3084981588 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242a464640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2424100080 0x7f2424197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f2429c63640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2424101270 0x7f24241977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242ac65640 1 -- 172.21.15.60:0/3084981588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2424101270 msgr2=0x7f24241977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242ac65640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2424101270 0x7f24241977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242ac65640 1 -- 172.21.15.60:0/3084981588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2424100080 msgr2=0x7f2424197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242ac65640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2424100080 0x7f2424197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242ac65640 1 -- 172.21.15.60:0/3084981588 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2414009560 con 0x7f2424101c30 2024-02-26T04:57:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f2429c63640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2424101270 0x7f24241977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.000+0000 7f242ac65640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2424101c30 0x7f242419e840 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f241c00b970 tx=0x7f241c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.001+0000 7f241b7fe640 1 -- 172.21.15.60:0/3084981588 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00cb40 con 0x7f2424101c30 2024-02-26T04:57:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.001+0000 7f242c6c8640 1 -- 172.21.15.60:0/3084981588 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242419f390 con 0x7f2424101c30 2024-02-26T04:57:47.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.001+0000 7f241b7fe640 1 -- 172.21.15.60:0/3084981588 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f241c00cca0 con 0x7f2424101c30 2024-02-26T04:57:47.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.001+0000 7f241b7fe640 1 -- 172.21.15.60:0/3084981588 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c0077d0 con 0x7f2424101c30 2024-02-26T04:57:47.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.001+0000 7f242c6c8640 1 -- 172.21.15.60:0/3084981588 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242419f8d0 con 0x7f2424101c30 2024-02-26T04:57:47.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.003+0000 7f241b7fe640 1 -- 172.21.15.60:0/3084981588 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f241c014440 con 0x7f2424101c30 2024-02-26T04:57:47.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.003+0000 7f242f574740 1 Processor -- start 2024-02-26T04:57:47.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.003+0000 7f242f574740 1 -- start start 2024-02-26T04:57:47.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242f574740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171068f50 0x55b171069370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f241b7fe640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f24100a20c0 0x7f24100a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242f574740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171069940 0x55b17106e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242f574740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b171070f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242f574740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b171024ca0 con 0x55b17106eab0 2024-02-26T04:57:47.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242f574740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b170fd7ef0 con 0x55b171068f50 2024-02-26T04:57:47.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242f574740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b170fd5420 con 0x55b171069940 2024-02-26T04:57:47.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242a464640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f24100a20c0 0x7f24100a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242ac65640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b171070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242ac65640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b171070f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38072/0 (socket says 172.21.15.60:38072) 2024-02-26T04:57:47.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242ac65640 1 -- 172.21.15.60:0/466645026 learned_addr learned my addr 172.21.15.60:0/466645026 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f2429c63640 1 --2- 172.21.15.60:0/466645026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171068f50 0x55b171069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f241b7fe640 1 -- 172.21.15.60:0/3084981588 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f241c0cda40 con 0x7f2424101c30 2024-02-26T04:57:47.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f2429c63640 1 --2- 172.21.15.60:0/466645026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171069940 0x55b17106e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242ac65640 1 -- 172.21.15.60:0/466645026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171069940 msgr2=0x55b17106e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242ac65640 1 --2- 172.21.15.60:0/466645026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171069940 0x55b17106e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242ac65640 1 -- 172.21.15.60:0/466645026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171068f50 msgr2=0x55b171069370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242ac65640 1 --2- 172.21.15.60:0/466645026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171068f50 0x55b171069370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242ac65640 1 -- 172.21.15.60:0/466645026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b170f85ef0 con 0x55b17106eab0 2024-02-26T04:57:47.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242a464640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f24100a20c0 0x7f24100a4580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2420005fd0 tx=0x7f2420005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f242ac65640 1 --2- 172.21.15.60:0/466645026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b171070f40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f241c09a030 tx=0x7f241c014ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351d014db20e0454 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.004+0000 7f2418ff9640 1 -- 172.21.15.60:0/466645026 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c021120 con 0x55b17106eab0 2024-02-26T04:57:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.005+0000 7f2418ff9640 1 -- 172.21.15.60:0/466645026 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f241c09aa30 con 0x55b17106eab0 2024-02-26T04:57:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.005+0000 7f2418ff9640 1 -- 172.21.15.60:0/466645026 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c0a2ad0 con 0x55b17106eab0 2024-02-26T04:57:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.005+0000 7f242f574740 1 -- 172.21.15.60:0/466645026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 msgr2=0x55b171070f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.005+0000 7f242f574740 1 --2- 172.21.15.60:0/466645026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b171070f40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f241c09a030 tx=0x7f241c014ce0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.005+0000 7f242f574740 1 -- 172.21.15.60:0/466645026 shutdown_connections 2024-02-26T04:57:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.005+0000 7f242f574740 1 --2- 172.21.15.60:0/466645026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b171070f40 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.005+0000 7f242f574740 1 --2- 172.21.15.60:0/466645026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171069940 0x55b17106e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.005+0000 7f242f574740 1 --2- 172.21.15.60:0/466645026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171068f50 0x55b171069370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.005+0000 7f242f574740 1 -- 172.21.15.60:0/466645026 >> 172.21.15.60:0/466645026 conn(0x55b1710644c0 msgr2=0x55b1710648c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.005+0000 7f242f574740 1 -- 172.21.15.60:0/466645026 shutdown_connections 2024-02-26T04:57:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.005+0000 7f242f574740 1 -- 172.21.15.60:0/466645026 wait complete. 2024-02-26T04:57:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242f574740 1 -- start start 2024-02-26T04:57:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242f574740 1 Processor -- start 2024-02-26T04:57:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242f574740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171068f50 0x55b171073100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242f574740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171069940 0x55b171073640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242f574740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b17107a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242f574740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b170fd0c00 con 0x55b17106eab0 2024-02-26T04:57:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242f574740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b170f75d00 con 0x55b171069940 2024-02-26T04:57:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242f574740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b170ca4310 con 0x55b171068f50 2024-02-26T04:57:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242f574740 10 client.0 _ll_register_callbacks cb 0x55b170f7dd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242ac65640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b17107a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242ac65640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b17107a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38086/0 (socket says 172.21.15.60:38086) 2024-02-26T04:57:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242ac65640 1 -- 172.21.15.60:0/1000371627 learned_addr learned my addr 172.21.15.60:0/1000371627 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f2429c63640 1 --2- 172.21.15.60:0/1000371627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171069940 0x55b171073640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242ac65640 1 -- 172.21.15.60:0/1000371627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171068f50 msgr2=0x55b171073100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242ac65640 1 --2- 172.21.15.60:0/1000371627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171068f50 0x55b171073100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242ac65640 1 -- 172.21.15.60:0/1000371627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171069940 msgr2=0x55b171073640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242ac65640 1 --2- 172.21.15.60:0/1000371627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171069940 0x55b171073640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242ac65640 1 -- 172.21.15.60:0/1000371627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242419f8d0 con 0x55b17106eab0 2024-02-26T04:57:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f2429c63640 1 --2- 172.21.15.60:0/1000371627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171069940 0x55b171073640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f242ac65640 1 --2- 172.21.15.60:0/1000371627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b17107a680 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f241c096d00 tx=0x7f241c014e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69417c226f753daf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.006+0000 7f240dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f240dffb640 1 -- 172.21.15.60:0/1000371627 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c094ea0 con 0x55b17106eab0 2024-02-26T04:57:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f240dffb640 1 -- 172.21.15.60:0/1000371627 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f241c0a2ad0 con 0x55b17106eab0 2024-02-26T04:57:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f242f574740 1 -- 172.21.15.60:0/1000371627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b170cd06c0 con 0x55b17106eab0 2024-02-26T04:57:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f240dffb640 1 -- 172.21.15.60:0/1000371627 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f241c09e130 con 0x55b17106eab0 2024-02-26T04:57:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f242f574740 10 client.15384 fetch_fsmap learned FSMap version 22 2024-02-26T04:57:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f242f574740 1 -- 172.21.15.60:0/1000371627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55b170fcf280 con 0x55b17106eab0 2024-02-26T04:57:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f240dffb640 1 -- 172.21.15.60:0/1000371627 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f241c0a6020 con 0x55b17106eab0 2024-02-26T04:57:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f240dffb640 1 -- 172.21.15.60:0/1000371627 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f241c0a2d20 con 0x55b17106eab0 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f242f574740 10 client.15384 fetch_fsmap finished waiting for FSMap version 22 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f242f574740 10 client.15384 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f242f574740 20 client.15384 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f242f574740 1 --2- 172.21.15.60:0/1000371627 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55b17107eb70 0x55b171083bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f242f574740 4 client.15384 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f242f574740 1 -- 172.21.15.60:0/1000371627 --> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b170d21270 con 0x55b17107eb70 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.007+0000 7f242a464640 1 --2- 172.21.15.60:0/1000371627 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55b17107eb70 0x55b171083bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.008+0000 7f242a464640 1 --2- 172.21.15.60:0/1000371627 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55b17107eb70 0x55b171083bd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=288a397b735f7d15 server_cookie=e8f327b843a06445 in_seq=0 out_seq=0 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.008+0000 7f240dffb640 10 client.15384 ms_handle_connect on v2:172.21.15.191:6838/640686517 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f240dffb640 1 -- 172.21.15.60:0/1000371627 <== mds.2 v2:172.21.15.191:6838/640686517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b170d21270 con 0x55b17107eb70 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f240dffb640 10 client.15384 handle_command_reply: tid=0 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f242f574740 1 client.15384 shutdown 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f242f574740 1 -- 172.21.15.60:0/1000371627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 msgr2=0x55b17107a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f242f574740 1 --2- 172.21.15.60:0/1000371627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b17107a680 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f241c096d00 tx=0x7f241c014e80 comp rx=0 tx=0).stop 2024-02-26T04:57:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f242f574740 1 -- 172.21.15.60:0/1000371627 shutdown_connections 2024-02-26T04:57:47.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f242f574740 1 --2- 172.21.15.60:0/1000371627 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55b17107eb70 0x55b171083bd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f242f574740 1 --2- 172.21.15.60:0/1000371627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b17107a680 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f242f574740 1 --2- 172.21.15.60:0/1000371627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171069940 0x55b171073640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f240dffb640 0 client.15384 ms_handle_reset on v2:172.21.15.191:6838/640686517 2024-02-26T04:57:47.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f242f574740 1 --2- 172.21.15.60:0/1000371627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171068f50 0x55b171073100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f242f574740 1 -- 172.21.15.60:0/1000371627 >> 172.21.15.60:0/1000371627 conn(0x55b1710644c0 msgr2=0x55b1710637a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f242f574740 1 -- 172.21.15.60:0/1000371627 shutdown_connections 2024-02-26T04:57:47.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f242f574740 1 -- 172.21.15.60:0/1000371627 wait complete. 2024-02-26T04:57:47.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.009+0000 7f242f574740 20 client.15384 trim_cache size 0 max 16384 2024-02-26T04:57:47.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242f574740 1 Processor -- start 2024-02-26T04:57:47.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242f574740 1 -- start start 2024-02-26T04:57:47.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242f574740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171068f50 0x55b171088300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242f574740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171069940 0x55b17107a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242f574740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b171062f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242f574740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b170f56460 con 0x55b17106eab0 2024-02-26T04:57:47.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242f574740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b170fc9aa0 con 0x55b171068f50 2024-02-26T04:57:47.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242f574740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b170f76980 con 0x55b171069940 2024-02-26T04:57:47.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242ac65640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b171062f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242ac65640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b171062f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38102/0 (socket says 172.21.15.60:38102) 2024-02-26T04:57:47.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242ac65640 1 -- 172.21.15.60:0/2100349353 learned_addr learned my addr 172.21.15.60:0/2100349353 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242a464640 1 --2- 172.21.15.60:0/2100349353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171068f50 0x55b171088300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242ac65640 1 -- 172.21.15.60:0/2100349353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171069940 msgr2=0x55b17107a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242ac65640 1 --2- 172.21.15.60:0/2100349353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171069940 0x55b17107a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242ac65640 1 -- 172.21.15.60:0/2100349353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171068f50 msgr2=0x55b171088300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242ac65640 1 --2- 172.21.15.60:0/2100349353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171068f50 0x55b171088300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242ac65640 1 -- 172.21.15.60:0/2100349353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b170f6b9f0 con 0x55b17106eab0 2024-02-26T04:57:47.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f242ac65640 1 --2- 172.21.15.60:0/2100349353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b171062f10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f241c09a2c0 tx=0x7f241c014ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aca7507b869fdfb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f240ffff640 1 -- 172.21.15.60:0/2100349353 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c094ea0 con 0x55b17106eab0 2024-02-26T04:57:47.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f240ffff640 1 -- 172.21.15.60:0/2100349353 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f241c0a6030 con 0x55b17106eab0 2024-02-26T04:57:47.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.025+0000 7f240ffff640 1 -- 172.21.15.60:0/2100349353 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c09e130 con 0x55b17106eab0 2024-02-26T04:57:47.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 -- 172.21.15.60:0/2100349353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 msgr2=0x55b171062f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 --2- 172.21.15.60:0/2100349353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b171062f10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f241c09a2c0 tx=0x7f241c014ce0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 -- 172.21.15.60:0/2100349353 shutdown_connections 2024-02-26T04:57:47.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 --2- 172.21.15.60:0/2100349353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b17106eab0 0x55b171062f10 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 --2- 172.21.15.60:0/2100349353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171069940 0x55b17107a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 --2- 172.21.15.60:0/2100349353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b171068f50 0x55b171088300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 -- 172.21.15.60:0/2100349353 >> 172.21.15.60:0/2100349353 conn(0x55b1710644c0 msgr2=0x55b171087bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 -- 172.21.15.60:0/2100349353 shutdown_connections 2024-02-26T04:57:47.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 -- 172.21.15.60:0/2100349353 wait complete. 2024-02-26T04:57:47.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 -- start start 2024-02-26T04:57:47.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 Processor -- start 2024-02-26T04:57:47.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171068f50 0x55b1710790a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b171069940 0x55b1710795e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b17106eab0 0x55b171079b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1710d59b0 con 0x55b171069940 2024-02-26T04:57:47.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1710d5b20 con 0x55b17106eab0 2024-02-26T04:57:47.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.026+0000 7f242f574740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1710d5ce0 con 0x55b171068f50 2024-02-26T04:57:47.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f242f574740 10 client.0 _ll_register_callbacks cb 0x7f241c095f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:47.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f2429c63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b171069940 0x55b1710795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f2429c63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b171069940 0x55b1710795e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38110/0 (socket says 172.21.15.60:38110) 2024-02-26T04:57:47.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f2429c63640 1 -- 172.21.15.60:0/609962744 learned_addr learned my addr 172.21.15.60:0/609962744 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f242ac65640 1 --2- 172.21.15.60:0/609962744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b17106eab0 0x55b171079b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f242a464640 1 --2- 172.21.15.60:0/609962744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171068f50 0x55b1710790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f2429c63640 1 -- 172.21.15.60:0/609962744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171068f50 msgr2=0x55b1710790a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f2429c63640 1 --2- 172.21.15.60:0/609962744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171068f50 0x55b1710790a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f2429c63640 1 -- 172.21.15.60:0/609962744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b17106eab0 msgr2=0x55b171079b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f2429c63640 1 --2- 172.21.15.60:0/609962744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b17106eab0 0x55b171079b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f2429c63640 1 -- 172.21.15.60:0/609962744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b170fcf280 con 0x55b171069940 2024-02-26T04:57:47.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f242a464640 1 --2- 172.21.15.60:0/609962744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171068f50 0x55b1710790a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:47.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f2429c63640 1 --2- 172.21.15.60:0/609962744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b171069940 0x55b1710795e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2414009900 tx=0x7f2414004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1254c17fab8481cf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f240dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:47.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f240dffb640 1 -- 172.21.15.60:0/609962744 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414004250 con 0x55b171069940 2024-02-26T04:57:47.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f240dffb640 1 -- 172.21.15.60:0/609962744 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f24140043b0 con 0x55b171069940 2024-02-26T04:57:47.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f242f574740 1 -- 172.21.15.60:0/609962744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b17107fac0 con 0x55b171069940 2024-02-26T04:57:47.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f240dffb640 1 -- 172.21.15.60:0/609962744 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24140035f0 con 0x55b171069940 2024-02-26T04:57:47.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f242f574740 10 client.15390 fetch_fsmap learned FSMap version 22 2024-02-26T04:57:47.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f242f574740 1 -- 172.21.15.60:0/609962744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55b170f91750 con 0x55b171069940 2024-02-26T04:57:47.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.027+0000 7f240dffb640 1 -- 172.21.15.60:0/609962744 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b17107fac0 con 0x55b171069940 2024-02-26T04:57:47.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.028+0000 7f240dffb640 1 -- 172.21.15.60:0/609962744 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2414005250 con 0x55b171069940 2024-02-26T04:57:47.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.028+0000 7f242f574740 10 client.15390 fetch_fsmap finished waiting for FSMap version 22 2024-02-26T04:57:47.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.028+0000 7f242f574740 10 client.15390 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-26T04:57:47.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.028+0000 7f242f574740 20 client.15390 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:47.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.028+0000 7f242f574740 1 --2- 172.21.15.60:0/609962744 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55b1710db0a0 0x55b1710d7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.028+0000 7f242f574740 4 client.15390 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T04:57:47.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.028+0000 7f242f574740 1 -- 172.21.15.60:0/609962744 --> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f241c09aa30 con 0x55b1710db0a0 2024-02-26T04:57:47.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.028+0000 7f242a464640 1 --2- 172.21.15.60:0/609962744 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55b1710db0a0 0x55b1710d7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.028+0000 7f240dffb640 10 client.15390 ms_handle_connect on v2:172.21.15.191:6838/640686517 2024-02-26T04:57:47.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.028+0000 7f242a464640 1 --2- 172.21.15.60:0/609962744 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55b1710db0a0 0x55b1710d7540 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=626bb7ff924c9b8e server_cookie=7d4b25a1abfc8f98 in_seq=0 out_seq=0 2024-02-26T04:57:47.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f240dffb640 1 -- 172.21.15.60:0/609962744 <== mds.2 v2:172.21.15.191:6838/640686517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f241c09aa30 con 0x55b1710db0a0 2024-02-26T04:57:47.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f240dffb640 10 client.15390 handle_command_reply: tid=0 2024-02-26T04:57:47.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242f574740 1 client.15390 shutdown 2024-02-26T04:57:47.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242f574740 1 -- 172.21.15.60:0/609962744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b171069940 msgr2=0x55b1710795e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242f574740 1 --2- 172.21.15.60:0/609962744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b171069940 0x55b1710795e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2414009900 tx=0x7f2414004c00 comp rx=0 tx=0).stop 2024-02-26T04:57:47.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242f574740 1 -- 172.21.15.60:0/609962744 shutdown_connections 2024-02-26T04:57:47.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242f574740 1 --2- 172.21.15.60:0/609962744 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55b1710db0a0 0x55b1710d7540 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242f574740 1 --2- 172.21.15.60:0/609962744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b17106eab0 0x55b171079b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f240dffb640 0 client.15390 ms_handle_reset on v2:172.21.15.191:6838/640686517 2024-02-26T04:57:47.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242f574740 1 --2- 172.21.15.60:0/609962744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b171069940 0x55b1710795e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242f574740 1 --2- 172.21.15.60:0/609962744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b171068f50 0x55b1710790a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242f574740 1 -- 172.21.15.60:0/609962744 >> 172.21.15.60:0/609962744 conn(0x55b1710644c0 msgr2=0x55b171088480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242f574740 1 -- 172.21.15.60:0/609962744 shutdown_connections 2024-02-26T04:57:47.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242f574740 1 -- 172.21.15.60:0/609962744 wait complete. 2024-02-26T04:57:47.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242f574740 20 client.15390 trim_cache size 0 max 16384 2024-02-26T04:57:47.039 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:57:47.040 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "f12e2206-d461-11ee-95c3-87774f69a715", 2024-02-26T04:57:47.040 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 2, 2024-02-26T04:57:47.040 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24472, 2024-02-26T04:57:47.040 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-02-26T04:57:47.040 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-02-26T04:57:47.040 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-02-26T04:57:47.040 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 113.489457896, 2024-02-26T04:57:47.040 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 22, 2024-02-26T04:57:47.040 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 93, 2024-02-26T04:57:47.040 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 89, 2024-02-26T04:57:47.040 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 153.55648257600001 2024-02-26T04:57:47.040 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:57:47.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242c6c8640 1 -- 172.21.15.60:0/3084981588 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f24100a20c0 msgr2=0x7f24100a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242c6c8640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f24100a20c0 0x7f24100a4580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2420005fd0 tx=0x7f2420005850 comp rx=0 tx=0).stop 2024-02-26T04:57:47.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242c6c8640 1 -- 172.21.15.60:0/3084981588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2424101c30 msgr2=0x7f242419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242c6c8640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2424101c30 0x7f242419e840 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f241c00b970 tx=0x7f241c00be30 comp rx=0 tx=0).stop 2024-02-26T04:57:47.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242c6c8640 1 -- 172.21.15.60:0/3084981588 shutdown_connections 2024-02-26T04:57:47.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242c6c8640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f24100a20c0 0x7f24100a4580 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242c6c8640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2424101c30 0x7f242419e840 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242c6c8640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2424101270 0x7f24241977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242c6c8640 1 --2- 172.21.15.60:0/3084981588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2424100080 0x7f2424197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242c6c8640 1 -- 172.21.15.60:0/3084981588 >> 172.21.15.60:0/3084981588 conn(0x7f24240fb610 msgr2=0x7f24240fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242c6c8640 1 -- 172.21.15.60:0/3084981588 shutdown_connections 2024-02-26T04:57:47.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.029+0000 7f242c6c8640 1 -- 172.21.15.60:0/3084981588 wait complete. 2024-02-26T04:57:47.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-02-26T04:57:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.144+0000 7feb44f8c640 1 -- 172.21.15.60:0/1953426873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40101c40 msgr2=0x7feb401084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.144+0000 7feb44f8c640 1 --2- 172.21.15.60:0/1953426873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40101c40 0x7feb401084d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7feb38009ca0 tx=0x7feb3802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.144+0000 7feb44f8c640 1 -- 172.21.15.60:0/1953426873 shutdown_connections 2024-02-26T04:57:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.144+0000 7feb44f8c640 1 --2- 172.21.15.60:0/1953426873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40101c40 0x7feb401084d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.144+0000 7feb44f8c640 1 --2- 172.21.15.60:0/1953426873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb40101280 0x7feb40101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.144+0000 7feb44f8c640 1 --2- 172.21.15.60:0/1953426873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb40100090 0x7feb401004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.144+0000 7feb44f8c640 1 -- 172.21.15.60:0/1953426873 >> 172.21.15.60:0/1953426873 conn(0x7feb400fb620 msgr2=0x7feb400fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.144+0000 7feb44f8c640 1 -- 172.21.15.60:0/1953426873 shutdown_connections 2024-02-26T04:57:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.144+0000 7feb44f8c640 1 -- 172.21.15.60:0/1953426873 wait complete. 2024-02-26T04:57:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb44f8c640 1 Processor -- start 2024-02-26T04:57:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb44f8c640 1 -- start start 2024-02-26T04:57:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb44f8c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb40100090 0x7feb40104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb44f8c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40101280 0x7feb40102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb44f8c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb40101c40 0x7feb40102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb44f8c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40103300 con 0x7feb40101280 2024-02-26T04:57:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb44f8c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40103470 con 0x7feb40101c40 2024-02-26T04:57:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb44f8c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb401a11b0 con 0x7feb40100090 2024-02-26T04:57:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb37fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40101280 0x7feb40102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb37fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40101280 0x7feb40102660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38128/0 (socket says 172.21.15.60:38128) 2024-02-26T04:57:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb37fff640 1 -- 172.21.15.60:0/616821644 learned_addr learned my addr 172.21.15.60:0/616821644 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb3e59c640 1 --2- 172.21.15.60:0/616821644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb40100090 0x7feb40104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb3ed9d640 1 --2- 172.21.15.60:0/616821644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb40101c40 0x7feb40102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb37fff640 1 -- 172.21.15.60:0/616821644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb40100090 msgr2=0x7feb40104020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb37fff640 1 --2- 172.21.15.60:0/616821644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb40100090 0x7feb40104020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb37fff640 1 -- 172.21.15.60:0/616821644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb40101c40 msgr2=0x7feb40102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb37fff640 1 --2- 172.21.15.60:0/616821644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb40101c40 0x7feb40102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.145+0000 7feb37fff640 1 -- 172.21.15.60:0/616821644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb38009900 con 0x7feb40101280 2024-02-26T04:57:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.146+0000 7feb37fff640 1 --2- 172.21.15.60:0/616821644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40101280 0x7feb40102660 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7feb2800c950 tx=0x7feb2800ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.146+0000 7feb377fe640 1 -- 172.21.15.60:0/616821644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28007c70 con 0x7feb40101280 2024-02-26T04:57:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.146+0000 7feb44f8c640 1 -- 172.21.15.60:0/616821644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb401a1460 con 0x7feb40101280 2024-02-26T04:57:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.146+0000 7feb377fe640 1 -- 172.21.15.60:0/616821644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb28007dd0 con 0x7feb40101280 2024-02-26T04:57:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.146+0000 7feb377fe640 1 -- 172.21.15.60:0/616821644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb280188c0 con 0x7feb40101280 2024-02-26T04:57:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.146+0000 7feb44f8c640 1 -- 172.21.15.60:0/616821644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb401a1970 con 0x7feb40101280 2024-02-26T04:57:47.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.148+0000 7feb377fe640 1 -- 172.21.15.60:0/616821644 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7feb28018a20 con 0x7feb40101280 2024-02-26T04:57:47.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.148+0000 7feb47e35740 1 Processor -- start 2024-02-26T04:57:47.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.148+0000 7feb47e35740 1 -- start start 2024-02-26T04:57:47.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb377fe640 1 --2- 172.21.15.60:0/616821644 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7feb2c0a20c0 0x7feb2c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb47e35740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197ddf50 0x5569197de370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb47e35740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197de940 0x5569197e3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb47e35740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197e3ab0 0x5569197e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb47e35740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556919799ca0 con 0x5569197de940 2024-02-26T04:57:47.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb47e35740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691974cef0 con 0x5569197ddf50 2024-02-26T04:57:47.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb47e35740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691974a420 con 0x5569197e3ab0 2024-02-26T04:57:47.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb37fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197de940 0x5569197e3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb3ed9d640 1 --2- 172.21.15.60:0/616821644 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7feb2c0a20c0 0x7feb2c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb37fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197de940 0x5569197e3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38144/0 (socket says 172.21.15.60:38144) 2024-02-26T04:57:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb3e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197ddf50 0x5569197de370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb37fff640 1 -- 172.21.15.60:0/2814866069 learned_addr learned my addr 172.21.15.60:0/2814866069 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb377fe640 1 -- 172.21.15.60:0/616821644 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7feb280ce2c0 con 0x7feb40101280 2024-02-26T04:57:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb37fff640 1 -- 172.21.15.60:0/2814866069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197e3ab0 msgr2=0x5569197e5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb37fff640 1 --2- 172.21.15.60:0/2814866069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197e3ab0 0x5569197e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb37fff640 1 -- 172.21.15.60:0/2814866069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197ddf50 msgr2=0x5569197de370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb37fff640 1 --2- 172.21.15.60:0/2814866069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197ddf50 0x5569197de370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb37fff640 1 -- 172.21.15.60:0/2814866069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569196faef0 con 0x5569197de940 2024-02-26T04:57:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb3e59c640 1 --2- 172.21.15.60:0/2814866069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197ddf50 0x5569197de370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:57:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb3ed9d640 1 --2- 172.21.15.60:0/616821644 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7feb2c0a20c0 0x7feb2c0a4580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7feb38006010 tx=0x7feb38009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb37fff640 1 --2- 172.21.15.60:0/2814866069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197de940 0x5569197e3440 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7feb2800fdf0 tx=0x7feb2800fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f91fff01658282 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb34ff9640 1 -- 172.21.15.60:0/2814866069 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2809a4b0 con 0x5569197de940 2024-02-26T04:57:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb34ff9640 1 -- 172.21.15.60:0/2814866069 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb2800fb50 con 0x5569197de940 2024-02-26T04:57:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.149+0000 7feb34ff9640 1 -- 172.21.15.60:0/2814866069 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb280baa40 con 0x5569197de940 2024-02-26T04:57:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.150+0000 7feb47e35740 1 -- 172.21.15.60:0/2814866069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197de940 msgr2=0x5569197e3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.150+0000 7feb47e35740 1 --2- 172.21.15.60:0/2814866069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197de940 0x5569197e3440 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7feb2800fdf0 tx=0x7feb2800fe50 comp rx=0 tx=0).stop 2024-02-26T04:57:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.150+0000 7feb47e35740 1 -- 172.21.15.60:0/2814866069 shutdown_connections 2024-02-26T04:57:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.150+0000 7feb47e35740 1 --2- 172.21.15.60:0/2814866069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197e3ab0 0x5569197e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.150+0000 7feb47e35740 1 --2- 172.21.15.60:0/2814866069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197de940 0x5569197e3440 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.150+0000 7feb47e35740 1 --2- 172.21.15.60:0/2814866069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197ddf50 0x5569197de370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.150+0000 7feb47e35740 1 -- 172.21.15.60:0/2814866069 >> 172.21.15.60:0/2814866069 conn(0x5569197d94c0 msgr2=0x5569197d98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.150+0000 7feb47e35740 1 -- 172.21.15.60:0/2814866069 shutdown_connections 2024-02-26T04:57:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.150+0000 7feb47e35740 1 -- 172.21.15.60:0/2814866069 wait complete. 2024-02-26T04:57:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.150+0000 7feb47e35740 1 -- start start 2024-02-26T04:57:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb47e35740 1 Processor -- start 2024-02-26T04:57:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb47e35740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197ddf50 0x5569197e7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb47e35740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197de940 0x5569197e8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb47e35740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197e3ab0 0x5569197ef560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb47e35740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556919745c00 con 0x5569197e3ab0 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb47e35740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569196ead00 con 0x5569197ddf50 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb47e35740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556919419310 con 0x5569197de940 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb47e35740 10 client.0 _ll_register_callbacks cb 0x5569196f2d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb3ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197e3ab0 0x5569197ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb3ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197e3ab0 0x5569197ef560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38152/0 (socket says 172.21.15.60:38152) 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb3ed9d640 1 -- 172.21.15.60:0/3058311180 learned_addr learned my addr 172.21.15.60:0/3058311180 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb37fff640 1 --2- 172.21.15.60:0/3058311180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197de940 0x5569197e8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb3e59c640 1 --2- 172.21.15.60:0/3058311180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197ddf50 0x5569197e7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb3ed9d640 1 -- 172.21.15.60:0/3058311180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197de940 msgr2=0x5569197e8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb3ed9d640 1 --2- 172.21.15.60:0/3058311180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197de940 0x5569197e8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb3ed9d640 1 -- 172.21.15.60:0/3058311180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197ddf50 msgr2=0x5569197e7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb3ed9d640 1 --2- 172.21.15.60:0/3058311180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197ddf50 0x5569197e7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.151+0000 7feb3ed9d640 1 -- 172.21.15.60:0/3058311180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb401a1970 con 0x5569197e3ab0 2024-02-26T04:57:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb3ed9d640 1 --2- 172.21.15.60:0/3058311180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197e3ab0 0x5569197ef560 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7feb38009ca0 tx=0x7feb380095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75e7c35dae3597c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb25ffb640 1 -- 172.21.15.60:0/3058311180 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38003b80 con 0x5569197e3ab0 2024-02-26T04:57:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb47e35740 1 -- 172.21.15.60:0/3058311180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569194456c0 con 0x5569197e3ab0 2024-02-26T04:57:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb25ffb640 1 -- 172.21.15.60:0/3058311180 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7feb38003ce0 con 0x5569197e3ab0 2024-02-26T04:57:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb25ffb640 1 -- 172.21.15.60:0/3058311180 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb38003ea0 con 0x5569197e3ab0 2024-02-26T04:57:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb47e35740 10 client.15402 fetch_fsmap learned FSMap version 22 2024-02-26T04:57:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb47e35740 1 -- 172.21.15.60:0/3058311180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x556919744280 con 0x5569197e3ab0 2024-02-26T04:57:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb25ffb640 1 -- 172.21.15.60:0/3058311180 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5569194456c0 con 0x5569197e3ab0 2024-02-26T04:57:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb25ffb640 1 -- 172.21.15.60:0/3058311180 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feb3803e040 con 0x5569197e3ab0 2024-02-26T04:57:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb47e35740 10 client.15402 fetch_fsmap finished waiting for FSMap version 22 2024-02-26T04:57:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb47e35740 10 client.15402 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-26T04:57:47.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb47e35740 20 client.15402 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:47.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb47e35740 1 --2- 172.21.15.60:0/3058311180 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5569197f49d0 0x5569197f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb47e35740 4 client.15402 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:57:47.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.152+0000 7feb47e35740 1 -- 172.21.15.60:0/3058311180 --> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556919496270 con 0x5569197f49d0 2024-02-26T04:57:47.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.153+0000 7feb3e59c640 1 --2- 172.21.15.60:0/3058311180 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5569197f49d0 0x5569197f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.153+0000 7feb25ffb640 10 client.15402 ms_handle_connect on v2:172.21.15.156:6836/707575792 2024-02-26T04:57:47.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.153+0000 7feb3e59c640 1 --2- 172.21.15.60:0/3058311180 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5569197f49d0 0x5569197f3c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce413cff011582c4 server_cookie=359967ff7e9088a2 in_seq=0 out_seq=0 2024-02-26T04:57:47.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb25ffb640 1 -- 172.21.15.60:0/3058311180 <== mds.1 v2:172.21.15.156:6836/707575792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556919496270 con 0x5569197f49d0 2024-02-26T04:57:47.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb25ffb640 10 client.15402 handle_command_reply: tid=0 2024-02-26T04:57:47.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb47e35740 1 client.15402 shutdown 2024-02-26T04:57:47.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb47e35740 1 -- 172.21.15.60:0/3058311180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197e3ab0 msgr2=0x5569197ef560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb47e35740 1 --2- 172.21.15.60:0/3058311180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197e3ab0 0x5569197ef560 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7feb38009ca0 tx=0x7feb380095c0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb47e35740 1 -- 172.21.15.60:0/3058311180 shutdown_connections 2024-02-26T04:57:47.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb47e35740 1 --2- 172.21.15.60:0/3058311180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197e3ab0 0x5569197ef560 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb47e35740 1 --2- 172.21.15.60:0/3058311180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197de940 0x5569197e8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb47e35740 1 --2- 172.21.15.60:0/3058311180 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5569197f49d0 0x5569197f3c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb47e35740 1 --2- 172.21.15.60:0/3058311180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197ddf50 0x5569197e7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb25ffb640 0 client.15402 ms_handle_reset on v2:172.21.15.156:6836/707575792 2024-02-26T04:57:47.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb47e35740 1 -- 172.21.15.60:0/3058311180 >> 172.21.15.60:0/3058311180 conn(0x5569197d94c0 msgr2=0x5569197d87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb47e35740 1 -- 172.21.15.60:0/3058311180 shutdown_connections 2024-02-26T04:57:47.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb47e35740 1 -- 172.21.15.60:0/3058311180 wait complete. 2024-02-26T04:57:47.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.154+0000 7feb47e35740 20 client.15402 trim_cache size 0 max 16384 2024-02-26T04:57:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb47e35740 1 Processor -- start 2024-02-26T04:57:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb47e35740 1 -- start start 2024-02-26T04:57:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb47e35740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197ddf50 0x5569197dfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb47e35740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197de940 0x5569197e0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb47e35740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197e3ab0 0x5569197ef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb47e35740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569196cb460 con 0x5569197e3ab0 2024-02-26T04:57:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb47e35740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569194456c0 con 0x5569197de940 2024-02-26T04:57:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb47e35740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569197da430 con 0x5569197ddf50 2024-02-26T04:57:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb3ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197e3ab0 0x5569197ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb3ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197e3ab0 0x5569197ef480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38160/0 (socket says 172.21.15.60:38160) 2024-02-26T04:57:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb3ed9d640 1 -- 172.21.15.60:0/54463659 learned_addr learned my addr 172.21.15.60:0/54463659 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb37fff640 1 --2- 172.21.15.60:0/54463659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197de940 0x5569197e0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb3ed9d640 1 -- 172.21.15.60:0/54463659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197ddf50 msgr2=0x5569197dfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb3e59c640 1 --2- 172.21.15.60:0/54463659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197ddf50 0x5569197dfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb3ed9d640 1 --2- 172.21.15.60:0/54463659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197ddf50 0x5569197dfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb3ed9d640 1 -- 172.21.15.60:0/54463659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197de940 msgr2=0x5569197e0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb3ed9d640 1 --2- 172.21.15.60:0/54463659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197de940 0x5569197e0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb3ed9d640 1 -- 172.21.15.60:0/54463659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569196e09f0 con 0x5569197e3ab0 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.170+0000 7feb37fff640 1 --2- 172.21.15.60:0/54463659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197de940 0x5569197e0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.171+0000 7feb3ed9d640 1 --2- 172.21.15.60:0/54463659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197e3ab0 0x5569197ef480 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7feb3802c630 tx=0x7feb38032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7311cbf35eb6905 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.171+0000 7feb27fff640 1 -- 172.21.15.60:0/54463659 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38040430 con 0x5569197e3ab0 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.171+0000 7feb27fff640 1 -- 172.21.15.60:0/54463659 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb38005cf0 con 0x5569197e3ab0 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.171+0000 7feb27fff640 1 -- 172.21.15.60:0/54463659 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38056cb0 con 0x5569197e3ab0 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.171+0000 7feb47e35740 1 -- 172.21.15.60:0/54463659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197e3ab0 msgr2=0x5569197ef480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.171+0000 7feb47e35740 1 --2- 172.21.15.60:0/54463659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197e3ab0 0x5569197ef480 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7feb3802c630 tx=0x7feb38032c40 comp rx=0 tx=0).stop 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.171+0000 7feb47e35740 1 -- 172.21.15.60:0/54463659 shutdown_connections 2024-02-26T04:57:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.171+0000 7feb47e35740 1 --2- 172.21.15.60:0/54463659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197e3ab0 0x5569197ef480 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.171+0000 7feb47e35740 1 --2- 172.21.15.60:0/54463659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197de940 0x5569197e0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.171+0000 7feb47e35740 1 --2- 172.21.15.60:0/54463659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197ddf50 0x5569197dfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.171+0000 7feb47e35740 1 -- 172.21.15.60:0/54463659 >> 172.21.15.60:0/54463659 conn(0x5569197d94c0 msgr2=0x5569197fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.171+0000 7feb47e35740 1 -- 172.21.15.60:0/54463659 shutdown_connections 2024-02-26T04:57:47.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.171+0000 7feb47e35740 1 -- 172.21.15.60:0/54463659 wait complete. 2024-02-26T04:57:47.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.171+0000 7feb47e35740 1 -- start start 2024-02-26T04:57:47.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb47e35740 1 Processor -- start 2024-02-26T04:57:47.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb47e35740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197ddf50 0x5569197eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb47e35740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197de940 0x5569197ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb47e35740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197e3ab0 0x5569197edfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb47e35740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569197ee500 con 0x5569197de940 2024-02-26T04:57:47.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb47e35740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569197ee670 con 0x5569197e3ab0 2024-02-26T04:57:47.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb47e35740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569197ee830 con 0x5569197ddf50 2024-02-26T04:57:47.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb47e35740 10 client.0 _ll_register_callbacks cb 0x7feb2800fd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb37fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197de940 0x5569197ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb37fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197de940 0x5569197ecfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38168/0 (socket says 172.21.15.60:38168) 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb37fff640 1 -- 172.21.15.60:0/156730861 learned_addr learned my addr 172.21.15.60:0/156730861 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb3e59c640 1 --2- 172.21.15.60:0/156730861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197ddf50 0x5569197eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb3ed9d640 1 --2- 172.21.15.60:0/156730861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197e3ab0 0x5569197edfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb37fff640 1 -- 172.21.15.60:0/156730861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197ddf50 msgr2=0x5569197eca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb37fff640 1 --2- 172.21.15.60:0/156730861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197ddf50 0x5569197eca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb37fff640 1 -- 172.21.15.60:0/156730861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197e3ab0 msgr2=0x5569197edfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb37fff640 1 --2- 172.21.15.60:0/156730861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197e3ab0 0x5569197edfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb37fff640 1 -- 172.21.15.60:0/156730861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556919744280 con 0x5569197de940 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb37fff640 1 --2- 172.21.15.60:0/156730861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197de940 0x5569197ecfb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feb280ce230 tx=0x7feb2809a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62e3caad26cd916 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.172+0000 7feb25ffb640 1 -- 172.21.15.60:0/156730861 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2809b250 con 0x5569197de940 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.173+0000 7feb25ffb640 1 -- 172.21.15.60:0/156730861 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7feb280baa40 con 0x5569197de940 2024-02-26T04:57:47.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.173+0000 7feb47e35740 1 -- 172.21.15.60:0/156730861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569197f46e0 con 0x5569197de940 2024-02-26T04:57:47.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.173+0000 7feb25ffb640 1 -- 172.21.15.60:0/156730861 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb2809ab50 con 0x5569197de940 2024-02-26T04:57:47.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.173+0000 7feb47e35740 10 client.15408 fetch_fsmap learned FSMap version 22 2024-02-26T04:57:47.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.173+0000 7feb47e35740 1 -- 172.21.15.60:0/156730861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x556919706750 con 0x5569197de940 2024-02-26T04:57:47.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.173+0000 7feb25ffb640 1 -- 172.21.15.60:0/156730861 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5569197f46e0 con 0x5569197de940 2024-02-26T04:57:47.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.173+0000 7feb25ffb640 1 -- 172.21.15.60:0/156730861 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feb28016020 con 0x5569197de940 2024-02-26T04:57:47.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.173+0000 7feb47e35740 10 client.15408 fetch_fsmap finished waiting for FSMap version 22 2024-02-26T04:57:47.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.173+0000 7feb47e35740 10 client.15408 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-26T04:57:47.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.173+0000 7feb47e35740 20 client.15408 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:47.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.173+0000 7feb47e35740 1 --2- 172.21.15.60:0/156730861 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x556919850140 0x55691984c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.173+0000 7feb47e35740 4 client.15408 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T04:57:47.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.173+0000 7feb47e35740 1 -- 172.21.15.60:0/156730861 --> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7feb2800fb50 con 0x556919850140 2024-02-26T04:57:47.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.173+0000 7feb3e59c640 1 --2- 172.21.15.60:0/156730861 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x556919850140 0x55691984c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb25ffb640 10 client.15408 ms_handle_connect on v2:172.21.15.156:6836/707575792 2024-02-26T04:57:47.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb3e59c640 1 --2- 172.21.15.60:0/156730861 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x556919850140 0x55691984c630 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c2e640fa29ddda8 server_cookie=1ea80c19dac52e9d in_seq=0 out_seq=0 2024-02-26T04:57:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb25ffb640 1 -- 172.21.15.60:0/156730861 <== mds.1 v2:172.21.15.156:6836/707575792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7feb2800fb50 con 0x556919850140 2024-02-26T04:57:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb25ffb640 10 client.15408 handle_command_reply: tid=0 2024-02-26T04:57:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb47e35740 1 client.15408 shutdown 2024-02-26T04:57:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb47e35740 1 -- 172.21.15.60:0/156730861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197de940 msgr2=0x5569197ecfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb47e35740 1 --2- 172.21.15.60:0/156730861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197de940 0x5569197ecfb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feb280ce230 tx=0x7feb2809a520 comp rx=0 tx=0).stop 2024-02-26T04:57:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb47e35740 1 -- 172.21.15.60:0/156730861 shutdown_connections 2024-02-26T04:57:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb47e35740 1 --2- 172.21.15.60:0/156730861 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x556919850140 0x55691984c630 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb47e35740 1 --2- 172.21.15.60:0/156730861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5569197e3ab0 0x5569197edfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb25ffb640 0 client.15408 ms_handle_reset on v2:172.21.15.156:6836/707575792 2024-02-26T04:57:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb47e35740 1 --2- 172.21.15.60:0/156730861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569197de940 0x5569197ecfb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb47e35740 1 --2- 172.21.15.60:0/156730861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569197ddf50 0x5569197eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb47e35740 1 -- 172.21.15.60:0/156730861 >> 172.21.15.60:0/156730861 conn(0x5569197d94c0 msgr2=0x5569197dfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb47e35740 1 -- 172.21.15.60:0/156730861 shutdown_connections 2024-02-26T04:57:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb47e35740 1 -- 172.21.15.60:0/156730861 wait complete. 2024-02-26T04:57:47.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.174+0000 7feb47e35740 20 client.15408 trim_cache size 0 max 16384 2024-02-26T04:57:47.184 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:57:47.184 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "f12e2206-d461-11ee-95c3-87774f69a715", 2024-02-26T04:57:47.184 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 1, 2024-02-26T04:57:47.184 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24476, 2024-02-26T04:57:47.184 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-02-26T04:57:47.184 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-02-26T04:57:47.184 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-02-26T04:57:47.184 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 114.64140949599999, 2024-02-26T04:57:47.184 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 22, 2024-02-26T04:57:47.185 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 93, 2024-02-26T04:57:47.185 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 89, 2024-02-26T04:57:47.185 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 156.48107304300001 2024-02-26T04:57:47.185 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:57:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.175+0000 7feb44f8c640 1 -- 172.21.15.60:0/616821644 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7feb2c0a20c0 msgr2=0x7feb2c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.175+0000 7feb44f8c640 1 --2- 172.21.15.60:0/616821644 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7feb2c0a20c0 0x7feb2c0a4580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7feb38006010 tx=0x7feb38009800 comp rx=0 tx=0).stop 2024-02-26T04:57:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.175+0000 7feb44f8c640 1 -- 172.21.15.60:0/616821644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40101280 msgr2=0x7feb40102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.175+0000 7feb44f8c640 1 --2- 172.21.15.60:0/616821644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40101280 0x7feb40102660 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7feb2800c950 tx=0x7feb2800ce10 comp rx=0 tx=0).stop 2024-02-26T04:57:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.175+0000 7feb44f8c640 1 -- 172.21.15.60:0/616821644 shutdown_connections 2024-02-26T04:57:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.175+0000 7feb44f8c640 1 --2- 172.21.15.60:0/616821644 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7feb2c0a20c0 0x7feb2c0a4580 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.175+0000 7feb44f8c640 1 --2- 172.21.15.60:0/616821644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb40101c40 0x7feb40102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.175+0000 7feb44f8c640 1 --2- 172.21.15.60:0/616821644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40101280 0x7feb40102660 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.175+0000 7feb44f8c640 1 --2- 172.21.15.60:0/616821644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb40100090 0x7feb40104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.175+0000 7feb44f8c640 1 -- 172.21.15.60:0/616821644 >> 172.21.15.60:0/616821644 conn(0x7feb400fb620 msgr2=0x7feb400fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.175+0000 7feb44f8c640 1 -- 172.21.15.60:0/616821644 shutdown_connections 2024-02-26T04:57:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.175+0000 7feb44f8c640 1 -- 172.21.15.60:0/616821644 wait complete. 2024-02-26T04:57:47.188 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-26T04:57:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.289+0000 7f1b68fa3640 1 -- 172.21.15.60:0/2837522184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64101c00 msgr2=0x7f1b64103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.289+0000 7f1b68fa3640 1 --2- 172.21.15.60:0/2837522184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64101c00 0x7f1b64103ff0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009900 tx=0x7f1b5402bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.289+0000 7f1b68fa3640 1 -- 172.21.15.60:0/2837522184 shutdown_connections 2024-02-26T04:57:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.289+0000 7f1b68fa3640 1 --2- 172.21.15.60:0/2837522184 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b64106e60 0x7f1b641092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.289+0000 7f1b68fa3640 1 --2- 172.21.15.60:0/2837522184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b64104530 0x7f1b64106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.289+0000 7f1b68fa3640 1 --2- 172.21.15.60:0/2837522184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64101c00 0x7f1b64103ff0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.289+0000 7f1b68fa3640 1 -- 172.21.15.60:0/2837522184 >> 172.21.15.60:0/2837522184 conn(0x7f1b640fb610 msgr2=0x7f1b640fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.289+0000 7f1b68fa3640 1 -- 172.21.15.60:0/2837522184 shutdown_connections 2024-02-26T04:57:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.289+0000 7f1b68fa3640 1 -- 172.21.15.60:0/2837522184 wait complete. 2024-02-26T04:57:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.290+0000 7f1b68fa3640 1 Processor -- start 2024-02-26T04:57:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.290+0000 7f1b68fa3640 1 -- start start 2024-02-26T04:57:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.290+0000 7f1b68fa3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64101c00 0x7f1b64197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.290+0000 7f1b68fa3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b64104530 0x7f1b641977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.290+0000 7f1b68fa3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b64106e60 0x7f1b6419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.290+0000 7f1b68fa3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6419ed50 con 0x7f1b64101c00 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.290+0000 7f1b68fa3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6419eec0 con 0x7f1b64106e60 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.290+0000 7f1b68fa3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6419f080 con 0x7f1b64104530 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.290+0000 7f1b6259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64101c00 0x7f1b64197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.290+0000 7f1b6259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64101c00 0x7f1b64197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38192/0 (socket says 172.21.15.60:38192) 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.290+0000 7f1b6259c640 1 -- 172.21.15.60:0/1739587161 learned_addr learned my addr 172.21.15.60:0/1739587161 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.290+0000 7f1b59d9b640 1 --2- 172.21.15.60:0/1739587161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b64104530 0x7f1b641977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.290+0000 7f1b62d9d640 1 --2- 172.21.15.60:0/1739587161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b64106e60 0x7f1b6419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.291+0000 7f1b6259c640 1 -- 172.21.15.60:0/1739587161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b64104530 msgr2=0x7f1b641977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.291+0000 7f1b6259c640 1 --2- 172.21.15.60:0/1739587161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b64104530 0x7f1b641977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.291+0000 7f1b6259c640 1 -- 172.21.15.60:0/1739587161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b64106e60 msgr2=0x7f1b6419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.291+0000 7f1b6259c640 1 --2- 172.21.15.60:0/1739587161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b64106e60 0x7f1b6419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.291+0000 7f1b6259c640 1 -- 172.21.15.60:0/1739587161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b54009560 con 0x7f1b64101c00 2024-02-26T04:57:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.291+0000 7f1b6259c640 1 --2- 172.21.15.60:0/1739587161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64101c00 0x7f1b64197260 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1b54006010 tx=0x7f1b54005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.291+0000 7f1b5bfff640 1 -- 172.21.15.60:0/1739587161 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b540043d0 con 0x7f1b64101c00 2024-02-26T04:57:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.291+0000 7f1b5bfff640 1 -- 172.21.15.60:0/1739587161 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b540052d0 con 0x7f1b64101c00 2024-02-26T04:57:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.291+0000 7f1b5bfff640 1 -- 172.21.15.60:0/1739587161 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54003cc0 con 0x7f1b64101c00 2024-02-26T04:57:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.291+0000 7f1b68fa3640 1 -- 172.21.15.60:0/1739587161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6419f300 con 0x7f1b64101c00 2024-02-26T04:57:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.291+0000 7f1b68fa3640 1 -- 172.21.15.60:0/1739587161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6419f860 con 0x7f1b64101c00 2024-02-26T04:57:47.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.293+0000 7f1b5bfff640 1 -- 172.21.15.60:0/1739587161 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1b54005440 con 0x7f1b64101c00 2024-02-26T04:57:47.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.293+0000 7f1b6bd77740 1 Processor -- start 2024-02-26T04:57:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b6bd77740 1 -- start start 2024-02-26T04:57:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b6bd77740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330360 0x561ad7330780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b6bd77740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7330d50 0x561ad7335850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b6bd77740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7335ec0 0x561ad7338350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b6bd77740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad72e7ca0 con 0x561ad7330360 2024-02-26T04:57:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b6bd77740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad729aef0 con 0x561ad7330d50 2024-02-26T04:57:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b6bd77740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad7298420 con 0x561ad7335ec0 2024-02-26T04:57:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b62d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330360 0x561ad7330780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b62d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330360 0x561ad7330780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38204/0 (socket says 172.21.15.60:38204) 2024-02-26T04:57:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b62d9d640 1 -- 172.21.15.60:0/2757217487 learned_addr learned my addr 172.21.15.60:0/2757217487 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b5bfff640 1 --2- 172.21.15.60:0/1739587161 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1b500a2140 0x7f1b500a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b59d9b640 1 --2- 172.21.15.60:0/2757217487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7335ec0 0x561ad7338350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b6259c640 1 --2- 172.21.15.60:0/2757217487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7330d50 0x561ad7335850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b62d9d640 1 --2- 172.21.15.60:0/1739587161 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1b500a2140 0x7f1b500a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b62d9d640 1 -- 172.21.15.60:0/2757217487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7335ec0 msgr2=0x561ad7338350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b62d9d640 1 --2- 172.21.15.60:0/2757217487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7335ec0 0x561ad7338350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b5bfff640 1 -- 172.21.15.60:0/1739587161 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f1b540f0300 con 0x7f1b64101c00 2024-02-26T04:57:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b62d9d640 1 -- 172.21.15.60:0/2757217487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7330d50 msgr2=0x561ad7335850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b62d9d640 1 --2- 172.21.15.60:0/2757217487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7330d50 0x561ad7335850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.294+0000 7f1b62d9d640 1 -- 172.21.15.60:0/2757217487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ad7248ef0 con 0x561ad7330360 2024-02-26T04:57:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.295+0000 7f1b62d9d640 1 --2- 172.21.15.60:0/2757217487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330360 0x561ad7330780 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c012a70 tx=0x7f1b5c012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e857d67a1de8b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.295+0000 7f1b58d99640 1 -- 172.21.15.60:0/2757217487 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c004230 con 0x561ad7330360 2024-02-26T04:57:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.295+0000 7f1b58d99640 1 -- 172.21.15.60:0/2757217487 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b5c004870 con 0x561ad7330360 2024-02-26T04:57:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.295+0000 7f1b58d99640 1 -- 172.21.15.60:0/2757217487 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c01fb50 con 0x561ad7330360 2024-02-26T04:57:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.295+0000 7f1b62d9d640 1 --2- 172.21.15.60:0/1739587161 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1b500a2140 0x7f1b500a4600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c022000 tx=0x7f1b5c020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2757217487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330360 msgr2=0x561ad7330780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2757217487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330360 0x561ad7330780 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c012a70 tx=0x7f1b5c012f30 comp rx=0 tx=0).stop 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2757217487 shutdown_connections 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2757217487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7335ec0 0x561ad7338350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2757217487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7330d50 0x561ad7335850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2757217487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330360 0x561ad7330780 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2757217487 >> 172.21.15.60:0/2757217487 conn(0x561ad73274c0 msgr2=0x561ad73278c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2757217487 shutdown_connections 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2757217487 wait complete. 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 -- start start 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 Processor -- start 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330360 0x561ad733a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330d50 0x561ad733aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7335ec0 0x561ad7341ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad7293c00 con 0x561ad7330d50 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad7238d00 con 0x561ad7335ec0 2024-02-26T04:57:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad6f67310 con 0x561ad7330360 2024-02-26T04:57:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b6bd77740 10 client.0 _ll_register_callbacks cb 0x561ad7240d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b59d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330d50 0x561ad733aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b59d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330d50 0x561ad733aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38218/0 (socket says 172.21.15.60:38218) 2024-02-26T04:57:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.296+0000 7f1b59d9b640 1 -- 172.21.15.60:0/2919716351 learned_addr learned my addr 172.21.15.60:0/2919716351 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b62d9d640 1 --2- 172.21.15.60:0/2919716351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7335ec0 0x561ad7341ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b6259c640 1 --2- 172.21.15.60:0/2919716351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330360 0x561ad733a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b59d9b640 1 -- 172.21.15.60:0/2919716351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330360 msgr2=0x561ad733a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b59d9b640 1 --2- 172.21.15.60:0/2919716351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330360 0x561ad733a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b59d9b640 1 -- 172.21.15.60:0/2919716351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7335ec0 msgr2=0x561ad7341ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b59d9b640 1 --2- 172.21.15.60:0/2919716351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7335ec0 0x561ad7341ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b59d9b640 1 -- 172.21.15.60:0/2919716351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c012650 con 0x561ad7330d50 2024-02-26T04:57:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b62d9d640 1 --2- 172.21.15.60:0/2919716351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7335ec0 0x561ad7341ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b59d9b640 1 --2- 172.21.15.60:0/2919716351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330d50 0x561ad733aa90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0126e0 tx=0x7f1b4c012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f37847aaed9074b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b49ffb640 1 -- 172.21.15.60:0/2919716351 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c017070 con 0x561ad7330d50 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b49ffb640 1 -- 172.21.15.60:0/2919716351 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f1b4c012190 con 0x561ad7330d50 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2919716351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ad6f936c0 con 0x561ad7330d50 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b49ffb640 1 -- 172.21.15.60:0/2919716351 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b4c00f8c0 con 0x561ad7330d50 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b6bd77740 10 client.15420 fetch_fsmap learned FSMap version 22 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2919716351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x561ad7292280 con 0x561ad7330d50 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b49ffb640 1 -- 172.21.15.60:0/2919716351 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561ad6f936c0 con 0x561ad7330d50 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.297+0000 7f1b49ffb640 1 -- 172.21.15.60:0/2919716351 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1b4c010aa0 con 0x561ad7330d50 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.298+0000 7f1b6bd77740 10 client.15420 fetch_fsmap finished waiting for FSMap version 22 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.298+0000 7f1b6bd77740 10 client.15420 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.298+0000 7f1b6bd77740 20 client.15420 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.298+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2919716351 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x561ad7346fd0 0x561ad7346200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.298+0000 7f1b6bd77740 4 client.15420 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:57:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.298+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2919716351 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ad6fe4270 con 0x561ad7346fd0 2024-02-26T04:57:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.298+0000 7f1b6259c640 1 --2- 172.21.15.60:0/2919716351 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x561ad7346fd0 0x561ad7346200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.298+0000 7f1b6259c640 1 --2- 172.21.15.60:0/2919716351 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x561ad7346fd0 0x561ad7346200 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e390ff807aac5835 server_cookie=da43cb6e48b5c9d1 in_seq=0 out_seq=0 2024-02-26T04:57:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.298+0000 7f1b49ffb640 10 client.15420 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b49ffb640 1 -- 172.21.15.60:0/2919716351 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ad6fe4270 con 0x561ad7346fd0 2024-02-26T04:57:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b49ffb640 10 client.15420 handle_command_reply: tid=0 2024-02-26T04:57:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b6bd77740 1 client.15420 shutdown 2024-02-26T04:57:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2919716351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330d50 msgr2=0x561ad733aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2919716351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330d50 0x561ad733aa90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0126e0 tx=0x7f1b4c012ba0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2919716351 shutdown_connections 2024-02-26T04:57:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2919716351 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x561ad7346fd0 0x561ad7346200 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2919716351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7335ec0 0x561ad7341ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b49ffb640 0 client.15420 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2919716351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330d50 0x561ad733aa90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2919716351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330360 0x561ad733a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2919716351 >> 172.21.15.60:0/2919716351 conn(0x561ad73274c0 msgr2=0x561ad73267a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2919716351 shutdown_connections 2024-02-26T04:57:47.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2919716351 wait complete. 2024-02-26T04:57:47.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.299+0000 7f1b6bd77740 20 client.15420 trim_cache size 0 max 16384 2024-02-26T04:57:47.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b6bd77740 1 Processor -- start 2024-02-26T04:57:47.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b6bd77740 1 -- start start 2024-02-26T04:57:47.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b6bd77740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7330360 0x561ad7332210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b6bd77740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330d50 0x561ad7332750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b6bd77740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7335ec0 0x561ad7341a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b6bd77740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad7219460 con 0x561ad7335ec0 2024-02-26T04:57:47.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b6bd77740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad7342980 con 0x561ad7330360 2024-02-26T04:57:47.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b6bd77740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad7327f10 con 0x561ad7330d50 2024-02-26T04:57:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b62d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7335ec0 0x561ad7341a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b62d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7335ec0 0x561ad7341a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38228/0 (socket says 172.21.15.60:38228) 2024-02-26T04:57:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b62d9d640 1 -- 172.21.15.60:0/3921787191 learned_addr learned my addr 172.21.15.60:0/3921787191 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b62d9d640 1 -- 172.21.15.60:0/3921787191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330d50 msgr2=0x561ad7332750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b62d9d640 1 --2- 172.21.15.60:0/3921787191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330d50 0x561ad7332750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b62d9d640 1 -- 172.21.15.60:0/3921787191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7330360 msgr2=0x561ad7332210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b6259c640 1 --2- 172.21.15.60:0/3921787191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7330360 0x561ad7332210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b62d9d640 1 --2- 172.21.15.60:0/3921787191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7330360 0x561ad7332210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b62d9d640 1 -- 172.21.15.60:0/3921787191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ad722e9f0 con 0x561ad7335ec0 2024-02-26T04:57:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.315+0000 7f1b62d9d640 1 --2- 172.21.15.60:0/3921787191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7335ec0 0x561ad7341a90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c000f80 tx=0x7f1b5c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3a8f24ea30c31c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.316+0000 7f1b4bfff640 1 -- 172.21.15.60:0/3921787191 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c01fb30 con 0x561ad7335ec0 2024-02-26T04:57:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.316+0000 7f1b4bfff640 1 -- 172.21.15.60:0/3921787191 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b5c02e040 con 0x561ad7335ec0 2024-02-26T04:57:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.316+0000 7f1b4bfff640 1 -- 172.21.15.60:0/3921787191 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c003ec0 con 0x561ad7335ec0 2024-02-26T04:57:47.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.316+0000 7f1b6bd77740 1 -- 172.21.15.60:0/3921787191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7335ec0 msgr2=0x561ad7341a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.316+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/3921787191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7335ec0 0x561ad7341a90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c000f80 tx=0x7f1b5c009b60 comp rx=0 tx=0).stop 2024-02-26T04:57:47.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.316+0000 7f1b6bd77740 1 -- 172.21.15.60:0/3921787191 shutdown_connections 2024-02-26T04:57:47.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.316+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/3921787191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7335ec0 0x561ad7341a90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.316+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/3921787191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330d50 0x561ad7332750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.316+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/3921787191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7330360 0x561ad7332210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.316+0000 7f1b6bd77740 1 -- 172.21.15.60:0/3921787191 >> 172.21.15.60:0/3921787191 conn(0x561ad73274c0 msgr2=0x561ad733a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.316+0000 7f1b6bd77740 1 -- 172.21.15.60:0/3921787191 shutdown_connections 2024-02-26T04:57:47.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.316+0000 7f1b6bd77740 1 -- 172.21.15.60:0/3921787191 wait complete. 2024-02-26T04:57:47.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.316+0000 7f1b6bd77740 1 -- start start 2024-02-26T04:57:47.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6bd77740 1 Processor -- start 2024-02-26T04:57:47.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6bd77740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330360 0x561ad733ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6bd77740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330d50 0x561ad733f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6bd77740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7335ec0 0x561ad7340460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6bd77740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad73409a0 con 0x561ad7330360 2024-02-26T04:57:47.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6bd77740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad7340b10 con 0x561ad7335ec0 2024-02-26T04:57:47.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6bd77740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad7340cd0 con 0x561ad7330d50 2024-02-26T04:57:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6bd77740 10 client.0 _ll_register_callbacks cb 0x7f1b5c009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330360 0x561ad733ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330360 0x561ad733ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38242/0 (socket says 172.21.15.60:38242) 2024-02-26T04:57:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6259c640 1 -- 172.21.15.60:0/2128299625 learned_addr learned my addr 172.21.15.60:0/2128299625 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b59d9b640 1 --2- 172.21.15.60:0/2128299625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330d50 0x561ad733f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6259c640 1 -- 172.21.15.60:0/2128299625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330d50 msgr2=0x561ad733f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6259c640 1 --2- 172.21.15.60:0/2128299625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330d50 0x561ad733f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6259c640 1 -- 172.21.15.60:0/2128299625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7335ec0 msgr2=0x561ad7340460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6259c640 1 --2- 172.21.15.60:0/2128299625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7335ec0 0x561ad7340460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6259c640 1 -- 172.21.15.60:0/2128299625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ad7248ef0 con 0x561ad7330360 2024-02-26T04:57:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b59d9b640 1 --2- 172.21.15.60:0/2128299625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330d50 0x561ad733f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b6259c640 1 --2- 172.21.15.60:0/2128299625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330360 0x561ad733ef10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1b54004650 tx=0x7f1b54047fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d7a451516ea400 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.317+0000 7f1b49ffb640 1 -- 172.21.15.60:0/2128299625 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b540bd2f0 con 0x561ad7330360 2024-02-26T04:57:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.318+0000 7f1b49ffb640 1 -- 172.21.15.60:0/2128299625 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f1b54003e20 con 0x561ad7330360 2024-02-26T04:57:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.318+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2128299625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ad7348000 con 0x561ad7330360 2024-02-26T04:57:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.318+0000 7f1b49ffb640 1 -- 172.21.15.60:0/2128299625 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b540bcbf0 con 0x561ad7330360 2024-02-26T04:57:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.318+0000 7f1b6bd77740 10 client.15426 fetch_fsmap learned FSMap version 22 2024-02-26T04:57:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.318+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2128299625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x561ad7254750 con 0x561ad7330360 2024-02-26T04:57:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.318+0000 7f1b49ffb640 1 -- 172.21.15.60:0/2128299625 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561ad7348000 con 0x561ad7330360 2024-02-26T04:57:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.318+0000 7f1b49ffb640 1 -- 172.21.15.60:0/2128299625 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1b5403c070 con 0x561ad7330360 2024-02-26T04:57:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.318+0000 7f1b6bd77740 10 client.15426 fetch_fsmap finished waiting for FSMap version 22 2024-02-26T04:57:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.318+0000 7f1b6bd77740 10 client.15426 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T04:57:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.318+0000 7f1b6bd77740 20 client.15426 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.318+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2128299625 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x561ad73a2a50 0x561ad739ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.318+0000 7f1b6bd77740 4 client.15426 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T04:57:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.318+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2128299625 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f1b5c004870 con 0x561ad73a2a50 2024-02-26T04:57:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.318+0000 7f1b59d9b640 1 --2- 172.21.15.60:0/2128299625 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x561ad73a2a50 0x561ad739ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b59d9b640 1 --2- 172.21.15.60:0/2128299625 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x561ad73a2a50 0x561ad739ef00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea50e97b232acb server_cookie=a0412c16ac68f393 in_seq=0 out_seq=0 2024-02-26T04:57:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b49ffb640 10 client.15426 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b49ffb640 1 -- 172.21.15.60:0/2128299625 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f1b5c004870 con 0x561ad73a2a50 2024-02-26T04:57:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b49ffb640 10 client.15426 handle_command_reply: tid=0 2024-02-26T04:57:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b6bd77740 1 client.15426 shutdown 2024-02-26T04:57:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2128299625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330360 msgr2=0x561ad733ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2128299625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330360 0x561ad733ef10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1b54004650 tx=0x7f1b54047fc0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2128299625 shutdown_connections 2024-02-26T04:57:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2128299625 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x561ad73a2a50 0x561ad739ef00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2128299625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ad7335ec0 0x561ad7340460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b49ffb640 0 client.15426 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2128299625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ad7330d50 0x561ad733f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b6bd77740 1 --2- 172.21.15.60:0/2128299625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561ad7330360 0x561ad733ef10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2128299625 >> 172.21.15.60:0/2128299625 conn(0x561ad73274c0 msgr2=0x561ad733a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2128299625 shutdown_connections 2024-02-26T04:57:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b6bd77740 1 -- 172.21.15.60:0/2128299625 wait complete. 2024-02-26T04:57:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.319+0000 7f1b6bd77740 20 client.15426 trim_cache size 0 max 16384 2024-02-26T04:57:47.329 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:57:47.329 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "f12e2206-d461-11ee-95c3-87774f69a715", 2024-02-26T04:57:47.329 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-02-26T04:57:47.329 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 24524, 2024-02-26T04:57:47.329 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-02-26T04:57:47.329 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-02-26T04:57:47.329 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-02-26T04:57:47.330 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 139.134792543, 2024-02-26T04:57:47.330 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 22, 2024-02-26T04:57:47.330 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 93, 2024-02-26T04:57:47.330 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 89, 2024-02-26T04:57:47.330 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 145.37031210699999 2024-02-26T04:57:47.330 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:57:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.320+0000 7f1b68fa3640 1 -- 172.21.15.60:0/1739587161 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1b500a2140 msgr2=0x7f1b500a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.320+0000 7f1b68fa3640 1 --2- 172.21.15.60:0/1739587161 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1b500a2140 0x7f1b500a4600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c022000 tx=0x7f1b5c020000 comp rx=0 tx=0).stop 2024-02-26T04:57:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.320+0000 7f1b68fa3640 1 -- 172.21.15.60:0/1739587161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64101c00 msgr2=0x7f1b64197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.320+0000 7f1b68fa3640 1 --2- 172.21.15.60:0/1739587161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64101c00 0x7f1b64197260 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1b54006010 tx=0x7f1b54005e80 comp rx=0 tx=0).stop 2024-02-26T04:57:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.320+0000 7f1b68fa3640 1 -- 172.21.15.60:0/1739587161 shutdown_connections 2024-02-26T04:57:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.320+0000 7f1b68fa3640 1 --2- 172.21.15.60:0/1739587161 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1b500a2140 0x7f1b500a4600 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.320+0000 7f1b68fa3640 1 --2- 172.21.15.60:0/1739587161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b64106e60 0x7f1b6419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.320+0000 7f1b68fa3640 1 --2- 172.21.15.60:0/1739587161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b64104530 0x7f1b641977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.320+0000 7f1b68fa3640 1 --2- 172.21.15.60:0/1739587161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64101c00 0x7f1b64197260 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.320+0000 7f1b68fa3640 1 -- 172.21.15.60:0/1739587161 >> 172.21.15.60:0/1739587161 conn(0x7f1b640fb610 msgr2=0x7f1b641026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:47.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.320+0000 7f1b68fa3640 1 -- 172.21.15.60:0/1739587161 shutdown_connections 2024-02-26T04:57:47.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:47.320+0000 7f1b68fa3640 1 -- 172.21.15.60:0/1739587161 wait complete. 2024-02-26T04:57:47.336 INFO:teuthology.run_tasks:Running task kclient... 2024-02-26T04:57:47.350 INFO:tasks.kclient:Mounting kernel clients... 2024-02-26T04:57:47.350 INFO:tasks.kclient:config is {'client.0': {}} 2024-02-26T04:57:47.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:47.350 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-02-26T04:57:47.365 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-02-26T04:57:47.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:47.366 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-26T04:57:47.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:47.433 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-02-26T04:57:47.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:47 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2586465443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:47.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:47 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2313792251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:47.508 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-02-26T04:57:47.508 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-26T04:57:47.508 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T04:57:47.508 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph fs ls 2024-02-26T04:57:47.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:47 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2586465443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:47.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:47 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2313792251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:47.754 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:57:47.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:47 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2586465443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:47.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:47 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2313792251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:48.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:48 smithi060 ceph-mon[32382]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:57:48.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:48 smithi156 ceph-mon[35805]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:57:48.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:48 smithi191 ceph-mon[36352]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:57:49.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.533+0000 7f93594e7700 1 -- 172.21.15.60:0/2226455698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93541070b0 msgr2=0x7f935410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:49.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.533+0000 7f93594e7700 1 --2- 172.21.15.60:0/2226455698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93541070b0 0x7f935410ec30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f934c00b0d0 tx=0x7f934c00b3e0 comp rx=0 tx=0).stop 2024-02-26T04:57:49.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.534+0000 7f93594e7700 1 -- 172.21.15.60:0/2226455698 shutdown_connections 2024-02-26T04:57:49.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.534+0000 7f93594e7700 1 --2- 172.21.15.60:0/2226455698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93541070b0 0x7f935410ec30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.534+0000 7f93594e7700 1 --2- 172.21.15.60:0/2226455698 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f935406a610 0x7f935406a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.534+0000 7f93594e7700 1 --2- 172.21.15.60:0/2226455698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f935406afc0 0x7f9354106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.534+0000 7f93594e7700 1 -- 172.21.15.60:0/2226455698 >> 172.21.15.60:0/2226455698 conn(0x7f93540779d0 msgr2=0x7f9354077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.534+0000 7f93594e7700 1 -- 172.21.15.60:0/2226455698 shutdown_connections 2024-02-26T04:57:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.534+0000 7f93594e7700 1 -- 172.21.15.60:0/2226455698 wait complete. 2024-02-26T04:57:49.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.534+0000 7f93594e7700 1 Processor -- start 2024-02-26T04:57:49.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f93594e7700 1 -- start start 2024-02-26T04:57:49.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f93594e7700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f935406a610 0x7f93541982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:49.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f93594e7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f935406afc0 0x7f9354198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:49.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f93594e7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93541070b0 0x7f935419cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:49.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f93594e7700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935419d160 con 0x7f935406a610 2024-02-26T04:57:49.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f93594e7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935419d2d0 con 0x7f93541070b0 2024-02-26T04:57:49.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f93594e7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935419d490 con 0x7f935406afc0 2024-02-26T04:57:49.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f9352ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f935406a610 0x7f93541982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:49.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f9352ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f935406a610 0x7f93541982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43956/0 (socket says 172.21.15.60:43956) 2024-02-26T04:57:49.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f9352ffd700 1 -- 172.21.15.60:0/1612803909 learned_addr learned my addr 172.21.15.60:0/1612803909 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:49.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f93537fe700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93541070b0 0x7f935419cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f93527fc700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f935406afc0 0x7f9354198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f9352ffd700 1 -- 172.21.15.60:0/1612803909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f935406afc0 msgr2=0x7f9354198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f9352ffd700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f935406afc0 0x7f9354198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f9352ffd700 1 -- 172.21.15.60:0/1612803909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93541070b0 msgr2=0x7f935419cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f9352ffd700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93541070b0 0x7f935419cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f9352ffd700 1 -- 172.21.15.60:0/1612803909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c009d00 con 0x7f935406a610 2024-02-26T04:57:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.535+0000 7f93527fc700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f935406afc0 0x7f9354198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.536+0000 7f9352ffd700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f935406a610 0x7f93541982d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f934400d980 tx=0x7f934400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.536+0000 7f933bfff700 1 -- 172.21.15.60:0/1612803909 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344004220 con 0x7f935406a610 2024-02-26T04:57:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.536+0000 7f93594e7700 1 -- 172.21.15.60:0/1612803909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935419d770 con 0x7f935406a610 2024-02-26T04:57:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.536+0000 7f933bfff700 1 -- 172.21.15.60:0/1612803909 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9344007640 con 0x7f935406a610 2024-02-26T04:57:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.536+0000 7f933bfff700 1 -- 172.21.15.60:0/1612803909 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344005720 con 0x7f935406a610 2024-02-26T04:57:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.536+0000 7f93594e7700 1 -- 172.21.15.60:0/1612803909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93541a4fd0 con 0x7f935406a610 2024-02-26T04:57:49.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.537+0000 7f93594e7700 1 -- 172.21.15.60:0/1612803909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935410a310 con 0x7f935406a610 2024-02-26T04:57:49.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.538+0000 7f933bfff700 1 -- 172.21.15.60:0/1612803909 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f934400b750 con 0x7f935406a610 2024-02-26T04:57:49.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.538+0000 7f933bfff700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f933c0a2150 0x7f933c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:49.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.538+0000 7f93527fc700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f933c0a2150 0x7f933c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:49.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.539+0000 7f933bfff700 1 -- 172.21.15.60:0/1612803909 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f93440cdf70 con 0x7f935406a610 2024-02-26T04:57:49.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.539+0000 7f93527fc700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f933c0a2150 0x7f933c0a4610 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9348005e10 tx=0x7f9348005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:49.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.543+0000 7f933bfff700 1 -- 172.21.15.60:0/1612803909 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9344096aa0 con 0x7f935406a610 2024-02-26T04:57:49.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.790+0000 7f93594e7700 1 -- 172.21.15.60:0/1612803909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9354199610 con 0x7f935406a610 2024-02-26T04:57:49.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.790+0000 7f933bfff700 1 -- 172.21.15.60:0/1612803909 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9344016070 con 0x7f935406a610 2024-02-26T04:57:49.793 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-26T04:57:49.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.792+0000 7f93594e7700 1 -- 172.21.15.60:0/1612803909 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f933c0a2150 msgr2=0x7f933c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:49.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.792+0000 7f93594e7700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f933c0a2150 0x7f933c0a4610 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9348005e10 tx=0x7f9348005d60 comp rx=0 tx=0).stop 2024-02-26T04:57:49.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.792+0000 7f93594e7700 1 -- 172.21.15.60:0/1612803909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f935406a610 msgr2=0x7f93541982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:49.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.792+0000 7f93594e7700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f935406a610 0x7f93541982d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f934400d980 tx=0x7f934400dc90 comp rx=0 tx=0).stop 2024-02-26T04:57:49.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.793+0000 7f93594e7700 1 -- 172.21.15.60:0/1612803909 shutdown_connections 2024-02-26T04:57:49.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.793+0000 7f93594e7700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f933c0a2150 0x7f933c0a4610 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:49.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.793+0000 7f93594e7700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93541070b0 0x7f935419cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:49.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.793+0000 7f93594e7700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f935406a610 0x7f93541982d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:49.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.793+0000 7f93594e7700 1 --2- 172.21.15.60:0/1612803909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f935406afc0 0x7f9354198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:49.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.793+0000 7f93594e7700 1 -- 172.21.15.60:0/1612803909 >> 172.21.15.60:0/1612803909 conn(0x7f93540779d0 msgr2=0x7f93540fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:49.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.793+0000 7f93594e7700 1 -- 172.21.15.60:0/1612803909 shutdown_connections 2024-02-26T04:57:49.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:49.793+0000 7f93594e7700 1 -- 172.21.15.60:0/1612803909 wait complete. 2024-02-26T04:57:50.457 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:50 smithi060 ceph-mon[32382]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:50.457 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:50 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1612803909' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T04:57:50.489 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-26T04:57:50.489 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T04:57:50.489 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-02-26T04:57:50.489 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-26T04:57:50.489 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T04:57:50.489 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-26T04:57:50.489 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-26T04:57:50.489 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T04:57:50.489 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T04:57:50.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:50.490 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-02-26T04:57:50.507 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T04:57:50.507 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T04:57:50.507 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T04:57:50.507 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-02-26T04:57:50.507 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-02-26T04:57:50.507 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-02-26T04:57:50.507 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T04:57:50.507 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-02-26T04:57:50.507 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-02-26T04:57:50.508 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T04:57:50.508 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-02-26T04:57:50.508 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-02-26T04:57:50.508 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T04:57:50.508 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-02-26T04:57:50.508 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-02-26T04:57:50.508 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-26T04:57:50.508 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41995sec preferred_lft 41995sec 2024-02-26T04:57:50.508 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-02-26T04:57:50.508 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-02-26T04:57:50.508 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T04:57:50.508 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-02-26T04:57:50.508 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-02-26T04:57:50.509 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-26T04:57:50.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:50.509 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T04:57:50.509 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T04:57:50.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add name ceph-brx type bridge 2024-02-26T04:57:50.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr flush dev ceph-brx 2024-02-26T04:57:50.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set ceph-brx up 2024-02-26T04:57:50.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-26T04:57:50.509 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T04:57:50.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:50.623 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T04:57:50.650 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-02-26T04:57:50.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:50.652 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-02-26T04:57:50.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:50 smithi156 ceph-mon[35805]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:50.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:50 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1612803909' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T04:57:50.708 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-02-26T04:57:50.708 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T04:57:50.708 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-26T04:57:50.708 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-26T04:57:50.708 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T04:57:50.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:50.708 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T04:57:50.708 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T04:57:50.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-26T04:57:50.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-26T04:57:50.709 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-26T04:57:50.709 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T04:57:50.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:50 smithi191 ceph-mon[36352]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:57:50.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:50 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1612803909' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-26T04:57:50.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:50.803 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-02-26T04:57:50.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:50.818 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2024-02-26T04:57:50.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:50.872 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T04:57:50.873 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T04:57:50.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-26T04:57:50.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-26T04:57:50.873 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T04:57:50.955 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-26T04:57:50.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:50.955 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T04:57:50.955 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T04:57:50.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-26T04:57:50.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-26T04:57:50.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-26T04:57:50.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-26T04:57:50.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-26T04:57:50.955 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T04:57:51.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:51.081 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-02-26T04:57:51.081 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-02-26T04:57:51.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-02-26T04:57:51.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.0 master ceph-brx 2024-02-26T04:57:51.081 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-02-26T04:57:51.122 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T04:57:51.122 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-26T04:57:51.181 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-26T04:57:51.181 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T04:57:51.181 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-26T04:57:51.236 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-02-26T04:57:51.236 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-02-26T04:57:51.237 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/b6609776-6af2-4a43-be85-51b16d9004c9 2024-02-26T04:57:51.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:51.237 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/b6609776-6af2-4a43-be85-51b16d9004c9 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-02-26T04:57:51.322 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-02-26T04:57:51.322 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-02-26T04:57:51.322 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-02-26T04:57:51.460 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-02-26T04:57:51.460 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-02-26T04:57:51.460 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-02-26T04:57:51.460 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.156:6789,172.21.15.191:6789" 2024-02-26T04:57:51.460 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.156:6789,172.21.15.191:6789:/volumes/qa/sv_1/b6609776-6af2-4a43-be85-51b16d9004c9 2024-02-26T04:57:51.461 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=f12e2206-d461-11ee-95c3-87774f69a715" will pass to kernel 2024-02-26T04:57:51.461 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-26T04:57:51.461 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-02-26T04:57:51.461 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T04:57:51.461 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T04:57:51.461 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T04:57:51.465 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T04:57:51.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:51.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-26T04:57:51.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:51.493 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T04:57:51.551 INFO:teuthology.orchestra.run.smithi060.stdout:client25030 2024-02-26T04:57:51.551 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:57:51.552 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T04:57:51.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:51.607 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T04:57:51.663 INFO:teuthology.orchestra.run.smithi060.stdout:client25030 2024-02-26T04:57:51.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:51.664 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T04:57:51.720 INFO:teuthology.orchestra.run.smithi060.stdout:client25030 2024-02-26T04:57:51.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:51.720 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/f12e2206-d461-11ee-95c3-87774f69a715.client25030/status') 2024-02-26T04:57:51.787 INFO:teuthology.orchestra.run.smithi060.stdout:instance: client.25030 (3)172.21.15.60:0/95757998 2024-02-26T04:57:51.787 INFO:teuthology.orchestra.run.smithi060.stdout:blocklisted: false 2024-02-26T04:57:51.787 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records in 2024-02-26T04:57:51.787 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records out 2024-02-26T04:57:51.787 INFO:teuthology.orchestra.run.smithi060.stderr:69 bytes copied, 2.0273e-05 s, 3.4 MB/s 2024-02-26T04:57:51.789 INFO:teuthology.run_tasks:Running task exec... 2024-02-26T04:57:51.800 INFO:teuthology.task.exec:Executing custom commands... 2024-02-26T04:57:51.800 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T04:57:51.801 DEBUG:teuthology.orchestra.run.smithi060:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-02-26T04:57:51.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.931+0000 7f52f23ed640 1 -- 172.21.15.60:0/2866181383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52ec1011c0 msgr2=0x7f52ec1015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:51.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.931+0000 7f52f23ed640 1 --2- 172.21.15.60:0/2866181383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52ec1011c0 0x7f52ec1015e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009900 tx=0x7f52dc02bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:51.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.931+0000 7f52f23ed640 1 -- 172.21.15.60:0/2866181383 shutdown_connections 2024-02-26T04:57:51.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.931+0000 7f52f23ed640 1 --2- 172.21.15.60:0/2866181383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52ec102d70 0x7f52ec109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.931+0000 7f52f23ed640 1 --2- 172.21.15.60:0/2866181383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ec1023b0 0x7f52ec102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.931+0000 7f52f23ed640 1 --2- 172.21.15.60:0/2866181383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52ec1011c0 0x7f52ec1015e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.931+0000 7f52f23ed640 1 -- 172.21.15.60:0/2866181383 >> 172.21.15.60:0/2866181383 conn(0x7f52ec0fc710 msgr2=0x7f52ec0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.931+0000 7f52f23ed640 1 -- 172.21.15.60:0/2866181383 shutdown_connections 2024-02-26T04:57:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.931+0000 7f52f23ed640 1 -- 172.21.15.60:0/2866181383 wait complete. 2024-02-26T04:57:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.931+0000 7f52f23ed640 1 Processor -- start 2024-02-26T04:57:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.931+0000 7f52f23ed640 1 -- start start 2024-02-26T04:57:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52f23ed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52ec1011c0 0x7f52ec074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52f23ed640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52ec1023b0 0x7f52ec073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52f23ed640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ec102d70 0x7f52ec073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52f23ed640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec074210 con 0x7f52ec1011c0 2024-02-26T04:57:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52f23ed640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec074380 con 0x7f52ec1023b0 2024-02-26T04:57:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52f23ed640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec1a1250 con 0x7f52ec102d70 2024-02-26T04:57:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52ebfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52ec1011c0 0x7f52ec074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52ebfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52ec1011c0 0x7f52ec074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43964/0 (socket says 172.21.15.60:43964) 2024-02-26T04:57:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52ebfff640 1 -- 172.21.15.60:0/1702195239 learned_addr learned my addr 172.21.15.60:0/1702195239 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52ebfff640 1 -- 172.21.15.60:0/1702195239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ec102d70 msgr2=0x7f52ec073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52ebfff640 1 --2- 172.21.15.60:0/1702195239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ec102d70 0x7f52ec073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52ebfff640 1 -- 172.21.15.60:0/1702195239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52ec1023b0 msgr2=0x7f52ec073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52eb7fe640 1 --2- 172.21.15.60:0/1702195239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52ec1023b0 0x7f52ec073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52ebfff640 1 --2- 172.21.15.60:0/1702195239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52ec1023b0 0x7f52ec073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52ebfff640 1 -- 172.21.15.60:0/1702195239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc009560 con 0x7f52ec1011c0 2024-02-26T04:57:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52eb7fe640 1 --2- 172.21.15.60:0/1702195239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52ec1023b0 0x7f52ec073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52ebfff640 1 --2- 172.21.15.60:0/1702195239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52ec1011c0 0x7f52ec074f30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f52dc02cd90 tx=0x7f52dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52e97fa640 1 -- 172.21.15.60:0/1702195239 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc004510 con 0x7f52ec1011c0 2024-02-26T04:57:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52f23ed640 1 -- 172.21.15.60:0/1702195239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec1a14a0 con 0x7f52ec1011c0 2024-02-26T04:57:51.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52e97fa640 1 -- 172.21.15.60:0/1702195239 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52dc0054e0 con 0x7f52ec1011c0 2024-02-26T04:57:51.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52e97fa640 1 -- 172.21.15.60:0/1702195239 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc03e450 con 0x7f52ec1011c0 2024-02-26T04:57:51.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.932+0000 7f52f23ed640 1 -- 172.21.15.60:0/1702195239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec1a1a00 con 0x7f52ec1011c0 2024-02-26T04:57:51.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.934+0000 7f52f23ed640 1 -- 172.21.15.60:0/1702195239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ec108480 con 0x7f52ec1011c0 2024-02-26T04:57:51.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.935+0000 7f52e97fa640 1 -- 172.21.15.60:0/1702195239 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f52dc003710 con 0x7f52ec1011c0 2024-02-26T04:57:51.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.935+0000 7f52e97fa640 1 --2- 172.21.15.60:0/1702195239 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f52d80a2160 0x7f52d80a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:51.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.935+0000 7f52eb7fe640 1 --2- 172.21.15.60:0/1702195239 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f52d80a2160 0x7f52d80a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:51.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.935+0000 7f52e97fa640 1 -- 172.21.15.60:0/1702195239 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f52dc0f05e0 con 0x7f52ec1011c0 2024-02-26T04:57:51.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.936+0000 7f52eb7fe640 1 --2- 172.21.15.60:0/1702195239 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f52d80a2160 0x7f52d80a4620 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f52ec074ca0 tx=0x7f52d4005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:51.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:51.940+0000 7f52e97fa640 1 -- 172.21.15.60:0/1702195239 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f52dc0b9170 con 0x7f52ec1011c0 2024-02-26T04:57:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.161+0000 7f52f23ed640 1 -- 172.21.15.60:0/1702195239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7f52ec02cda0 con 0x7f52ec1011c0 2024-02-26T04:57:52.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.290+0000 7f52e97fa640 1 -- 172.21.15.60:0/1702195239 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v23) v1 ==== 117+0+0 (secure 0 0 0) 0x7f52dc0bc830 con 0x7f52ec1011c0 2024-02-26T04:57:52.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.292+0000 7f52f23ed640 1 -- 172.21.15.60:0/1702195239 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f52d80a2160 msgr2=0x7f52d80a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:52.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.292+0000 7f52f23ed640 1 --2- 172.21.15.60:0/1702195239 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f52d80a2160 0x7f52d80a4620 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f52ec074ca0 tx=0x7f52d4005d30 comp rx=0 tx=0).stop 2024-02-26T04:57:52.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.292+0000 7f52f23ed640 1 -- 172.21.15.60:0/1702195239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52ec1011c0 msgr2=0x7f52ec074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:52.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.292+0000 7f52f23ed640 1 --2- 172.21.15.60:0/1702195239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52ec1011c0 0x7f52ec074f30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f52dc02cd90 tx=0x7f52dc0042f0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.293+0000 7f52f23ed640 1 -- 172.21.15.60:0/1702195239 shutdown_connections 2024-02-26T04:57:52.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.293+0000 7f52f23ed640 1 --2- 172.21.15.60:0/1702195239 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f52d80a2160 0x7f52d80a4620 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.293+0000 7f52f23ed640 1 --2- 172.21.15.60:0/1702195239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ec102d70 0x7f52ec073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.293+0000 7f52f23ed640 1 --2- 172.21.15.60:0/1702195239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52ec1023b0 0x7f52ec073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.293+0000 7f52f23ed640 1 --2- 172.21.15.60:0/1702195239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52ec1011c0 0x7f52ec074f30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.293+0000 7f52f23ed640 1 -- 172.21.15.60:0/1702195239 >> 172.21.15.60:0/1702195239 conn(0x7f52ec0fc710 msgr2=0x7f52ec105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:52.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.293+0000 7f52f23ed640 1 -- 172.21.15.60:0/1702195239 shutdown_connections 2024-02-26T04:57:52.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.293+0000 7f52f23ed640 1 -- 172.21.15.60:0/1702195239 wait complete. 2024-02-26T04:57:52.307 INFO:teuthology.run_tasks:Running task check-counter... 2024-02-26T04:57:52.339 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-02-26T04:57:52.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:57:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 -- 172.21.15.60:0/2199336237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec104550 msgr2=0x7f98ec106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 --2- 172.21.15.60:0/2199336237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec104550 0x7f98ec106940 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009900 tx=0x7f98dc02bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 -- 172.21.15.60:0/2199336237 shutdown_connections 2024-02-26T04:57:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 --2- 172.21.15.60:0/2199336237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98ec106e80 0x7f98ec1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 --2- 172.21.15.60:0/2199336237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec104550 0x7f98ec106940 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 --2- 172.21.15.60:0/2199336237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98ec101c20 0x7f98ec104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 -- 172.21.15.60:0/2199336237 >> 172.21.15.60:0/2199336237 conn(0x7f98ec0fb610 msgr2=0x7f98ec0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 -- 172.21.15.60:0/2199336237 shutdown_connections 2024-02-26T04:57:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 -- 172.21.15.60:0/2199336237 wait complete. 2024-02-26T04:57:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 Processor -- start 2024-02-26T04:57:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 -- start start 2024-02-26T04:57:52.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98ec101c20 0x7f98ec197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:52.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec104550 0x7f98ec197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:52.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98ec106e80 0x7f98ec19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:52.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec19ed00 con 0x7f98ec104550 2024-02-26T04:57:52.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec19ee70 con 0x7f98ec106e80 2024-02-26T04:57:52.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.453+0000 7f98f43bb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec19f030 con 0x7f98ec101c20 2024-02-26T04:57:52.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98f1956640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec104550 0x7f98ec197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:52.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98f1956640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec104550 0x7f98ec197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43982/0 (socket says 172.21.15.60:43982) 2024-02-26T04:57:52.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98f1956640 1 -- 172.21.15.60:0/1373352570 learned_addr learned my addr 172.21.15.60:0/1373352570 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:52.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98f2157640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98ec101c20 0x7f98ec197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:52.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98f2958640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98ec106e80 0x7f98ec19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:52.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98f1956640 1 -- 172.21.15.60:0/1373352570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98ec101c20 msgr2=0x7f98ec197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:52.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98f1956640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98ec101c20 0x7f98ec197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98f1956640 1 -- 172.21.15.60:0/1373352570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98ec106e80 msgr2=0x7f98ec19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:52.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98f1956640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98ec106e80 0x7f98ec19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98f1956640 1 -- 172.21.15.60:0/1373352570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc009560 con 0x7f98ec104550 2024-02-26T04:57:52.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98f2958640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98ec106e80 0x7f98ec19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:52.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98f1956640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec104550 0x7f98ec197750 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f98dc036040 tx=0x7f98dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:52.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98e37fe640 1 -- 172.21.15.60:0/1373352570 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0044e0 con 0x7f98ec104550 2024-02-26T04:57:52.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98e37fe640 1 -- 172.21.15.60:0/1373352570 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98dc0054e0 con 0x7f98ec104550 2024-02-26T04:57:52.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98e37fe640 1 -- 172.21.15.60:0/1373352570 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc03e450 con 0x7f98ec104550 2024-02-26T04:57:52.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98f43bb640 1 -- 172.21.15.60:0/1373352570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ec19f2b0 con 0x7f98ec104550 2024-02-26T04:57:52.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.454+0000 7f98f43bb640 1 -- 172.21.15.60:0/1373352570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ec19f810 con 0x7f98ec104550 2024-02-26T04:57:52.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.455+0000 7f98f43bb640 1 -- 172.21.15.60:0/1373352570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ec0fd100 con 0x7f98ec104550 2024-02-26T04:57:52.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.456+0000 7f98e37fe640 1 -- 172.21.15.60:0/1373352570 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f98dc003710 con 0x7f98ec104550 2024-02-26T04:57:52.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.457+0000 7f98e37fe640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f98d80a2140 0x7f98d80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:52.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.457+0000 7f98f2157640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f98d80a2140 0x7f98d80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:52.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.457+0000 7f98e37fe640 1 -- 172.21.15.60:0/1373352570 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f98dc0f0b10 con 0x7f98ec104550 2024-02-26T04:57:52.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.458+0000 7f98f2157640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f98d80a2140 0x7f98d80a4600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f98e8005e10 tx=0x7f98e8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:52.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.461+0000 7f98e37fe640 1 -- 172.21.15.60:0/1373352570 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f98dc0b96a0 con 0x7f98ec104550 2024-02-26T04:57:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.644+0000 7f98f43bb640 1 -- 172.21.15.60:0/1373352570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98ec10cdc0 con 0x7f98ec104550 2024-02-26T04:57:52.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.646+0000 7f98e37fe640 1 -- 172.21.15.60:0/1373352570 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24972 (secure 0 0 0) 0x7f98dc0bcd60 con 0x7f98ec104550 2024-02-26T04:57:52.649 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:52.649 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:57:21.168776+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:57:52.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.648+0000 7f98f43bb640 1 -- 172.21.15.60:0/1373352570 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f98d80a2140 msgr2=0x7f98d80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:52.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.648+0000 7f98f43bb640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f98d80a2140 0x7f98d80a4600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f98e8005e10 tx=0x7f98e8005da0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.648+0000 7f98f43bb640 1 -- 172.21.15.60:0/1373352570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec104550 msgr2=0x7f98ec197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:52.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.648+0000 7f98f43bb640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec104550 0x7f98ec197750 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f98dc036040 tx=0x7f98dc0042f0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.648+0000 7f98f43bb640 1 -- 172.21.15.60:0/1373352570 shutdown_connections 2024-02-26T04:57:52.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.648+0000 7f98f43bb640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f98d80a2140 0x7f98d80a4600 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.648+0000 7f98f43bb640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98ec106e80 0x7f98ec19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.648+0000 7f98f43bb640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec104550 0x7f98ec197750 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.648+0000 7f98f43bb640 1 --2- 172.21.15.60:0/1373352570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98ec101c20 0x7f98ec197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.648+0000 7f98f43bb640 1 -- 172.21.15.60:0/1373352570 >> 172.21.15.60:0/1373352570 conn(0x7f98ec0fb610 msgr2=0x7f98ec102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:52.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.648+0000 7f98f43bb640 1 -- 172.21.15.60:0/1373352570 shutdown_connections 2024-02-26T04:57:52.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.648+0000 7f98f43bb640 1 -- 172.21.15.60:0/1373352570 wait complete. 2024-02-26T04:57:52.661 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:52 smithi060 ceph-mon[32382]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:57:52.661 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:52 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1702195239' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-02-26T04:57:52.662 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T04:50:44.572921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T04:55:25.530720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T04:55:26.741604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T04:55:30.453569+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T04:57:52.663 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T04:57:52.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:52 smithi156 ceph-mon[35805]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:57:52.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:52 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1702195239' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-02-26T04:57:52.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:52 smithi191 ceph-mon[36352]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:57:52.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:52 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1702195239' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-02-26T04:57:52.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.766+0000 7f7446ed5640 1 -- 172.21.15.60:0/3534937582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74401045b0 msgr2=0x7f74401069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:52.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.766+0000 7f7446ed5640 1 --2- 172.21.15.60:0/3534937582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74401045b0 0x7f74401069a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f742c009900 tx=0x7f742c02bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:52.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.766+0000 7f7446ed5640 1 -- 172.21.15.60:0/3534937582 shutdown_connections 2024-02-26T04:57:52.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.766+0000 7f7446ed5640 1 --2- 172.21.15.60:0/3534937582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7440106ee0 0x7f74401092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.766+0000 7f7446ed5640 1 --2- 172.21.15.60:0/3534937582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74401045b0 0x7f74401069a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.766+0000 7f7446ed5640 1 --2- 172.21.15.60:0/3534937582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7440101c80 0x7f7440104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.766+0000 7f7446ed5640 1 -- 172.21.15.60:0/3534937582 >> 172.21.15.60:0/3534937582 conn(0x7f74400fb630 msgr2=0x7f74400fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:52.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.766+0000 7f7446ed5640 1 -- 172.21.15.60:0/3534937582 shutdown_connections 2024-02-26T04:57:52.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.766+0000 7f7446ed5640 1 -- 172.21.15.60:0/3534937582 wait complete. 2024-02-26T04:57:52.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.766+0000 7f7446ed5640 1 Processor -- start 2024-02-26T04:57:52.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.766+0000 7f7446ed5640 1 -- start start 2024-02-26T04:57:52.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7446ed5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7440101c80 0x7f7440197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:52.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7446ed5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74401045b0 0x7f74401977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:52.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7446ed5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7440106ee0 0x7f744019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:52.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7446ed5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744019ed50 con 0x7f7440101c80 2024-02-26T04:57:52.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7446ed5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744019eec0 con 0x7f74401045b0 2024-02-26T04:57:52.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7446ed5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744019f080 con 0x7f7440106ee0 2024-02-26T04:57:52.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7444c71640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7440101c80 0x7f7440197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:52.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7444c71640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7440101c80 0x7f7440197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43996/0 (socket says 172.21.15.60:43996) 2024-02-26T04:57:52.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7444c71640 1 -- 172.21.15.60:0/590092318 learned_addr learned my addr 172.21.15.60:0/590092318 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:52.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f743ffff640 1 --2- 172.21.15.60:0/590092318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74401045b0 0x7f74401977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:52.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7445472640 1 --2- 172.21.15.60:0/590092318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7440106ee0 0x7f744019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:52.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7444c71640 1 -- 172.21.15.60:0/590092318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7440106ee0 msgr2=0x7f744019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:52.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7444c71640 1 --2- 172.21.15.60:0/590092318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7440106ee0 0x7f744019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7444c71640 1 -- 172.21.15.60:0/590092318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74401045b0 msgr2=0x7f74401977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:52.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7444c71640 1 --2- 172.21.15.60:0/590092318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74401045b0 0x7f74401977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7444c71640 1 -- 172.21.15.60:0/590092318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c009560 con 0x7f7440101c80 2024-02-26T04:57:52.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7444c71640 1 --2- 172.21.15.60:0/590092318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7440101c80 0x7f7440197260 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f743400e9f0 tx=0x7f743400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:52.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f743dffb640 1 -- 172.21.15.60:0/590092318 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434009720 con 0x7f7440101c80 2024-02-26T04:57:52.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f7446ed5640 1 -- 172.21.15.60:0/590092318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744019f360 con 0x7f7440101c80 2024-02-26T04:57:52.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f743dffb640 1 -- 172.21.15.60:0/590092318 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f743400cd80 con 0x7f7440101c80 2024-02-26T04:57:52.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.767+0000 7f743dffb640 1 -- 172.21.15.60:0/590092318 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434010690 con 0x7f7440101c80 2024-02-26T04:57:52.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.768+0000 7f7446ed5640 1 -- 172.21.15.60:0/590092318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744019f840 con 0x7f7440101c80 2024-02-26T04:57:52.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.769+0000 7f7446ed5640 1 -- 172.21.15.60:0/590092318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74400fd120 con 0x7f7440101c80 2024-02-26T04:57:52.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.770+0000 7f743dffb640 1 -- 172.21.15.60:0/590092318 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f74340046c0 con 0x7f7440101c80 2024-02-26T04:57:52.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.770+0000 7f743dffb640 1 --2- 172.21.15.60:0/590092318 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f74300a20c0 0x7f74300a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:52.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.770+0000 7f743dffb640 1 -- 172.21.15.60:0/590092318 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f7434014070 con 0x7f7440101c80 2024-02-26T04:57:52.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.770+0000 7f743ffff640 1 --2- 172.21.15.60:0/590092318 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f74300a20c0 0x7f74300a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:52.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.771+0000 7f743ffff640 1 --2- 172.21.15.60:0/590092318 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f74300a20c0 0x7f74300a4580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f742c005ab0 tx=0x7f742c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:52.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.774+0000 7f743dffb640 1 -- 172.21.15.60:0/590092318 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7434097280 con 0x7f7440101c80 2024-02-26T04:57:52.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.955+0000 7f7446ed5640 1 -- 172.21.15.60:0/590092318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f744010cbb0 con 0x7f7440101c80 2024-02-26T04:57:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.956+0000 7f743dffb640 1 -- 172.21.15.60:0/590092318 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f743409a940 con 0x7f7440101c80 2024-02-26T04:57:52.958 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T04:57:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.958+0000 7f7446ed5640 1 -- 172.21.15.60:0/590092318 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f74300a20c0 msgr2=0x7f74300a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.958+0000 7f7446ed5640 1 --2- 172.21.15.60:0/590092318 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f74300a20c0 0x7f74300a4580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f742c005ab0 tx=0x7f742c005fd0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.958+0000 7f7446ed5640 1 -- 172.21.15.60:0/590092318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7440101c80 msgr2=0x7f7440197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.958+0000 7f7446ed5640 1 --2- 172.21.15.60:0/590092318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7440101c80 0x7f7440197260 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f743400e9f0 tx=0x7f743400edf0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.958+0000 7f7446ed5640 1 -- 172.21.15.60:0/590092318 shutdown_connections 2024-02-26T04:57:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.958+0000 7f7446ed5640 1 --2- 172.21.15.60:0/590092318 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f74300a20c0 0x7f74300a4580 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.958+0000 7f7446ed5640 1 --2- 172.21.15.60:0/590092318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7440106ee0 0x7f744019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.958+0000 7f7446ed5640 1 --2- 172.21.15.60:0/590092318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74401045b0 0x7f74401977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.958+0000 7f7446ed5640 1 --2- 172.21.15.60:0/590092318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7440101c80 0x7f7440197260 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:52.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.958+0000 7f7446ed5640 1 -- 172.21.15.60:0/590092318 >> 172.21.15.60:0/590092318 conn(0x7f74400fb630 msgr2=0x7f7440102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:52.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.958+0000 7f7446ed5640 1 -- 172.21.15.60:0/590092318 shutdown_connections 2024-02-26T04:57:52.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:52.958+0000 7f7446ed5640 1 -- 172.21.15.60:0/590092318 wait complete. 2024-02-26T04:57:52.972 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T04:57:53.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.075+0000 7f61b24de640 1 -- 172.21.15.60:0/1802333487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61ac101c40 msgr2=0x7f61ac1084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.075+0000 7f61b24de640 1 --2- 172.21.15.60:0/1802333487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61ac101c40 0x7f61ac1084d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f61a4009ca0 tx=0x7f61a402c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.075+0000 7f61b24de640 1 -- 172.21.15.60:0/1802333487 shutdown_connections 2024-02-26T04:57:53.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.075+0000 7f61b24de640 1 --2- 172.21.15.60:0/1802333487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61ac101c40 0x7f61ac1084d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.075+0000 7f61b24de640 1 --2- 172.21.15.60:0/1802333487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61ac101280 0x7f61ac101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.075+0000 7f61b24de640 1 --2- 172.21.15.60:0/1802333487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61ac100090 0x7f61ac1004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.075+0000 7f61b24de640 1 -- 172.21.15.60:0/1802333487 >> 172.21.15.60:0/1802333487 conn(0x7f61ac0fb620 msgr2=0x7f61ac0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:53.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.075+0000 7f61b24de640 1 -- 172.21.15.60:0/1802333487 shutdown_connections 2024-02-26T04:57:53.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.075+0000 7f61b24de640 1 -- 172.21.15.60:0/1802333487 wait complete. 2024-02-26T04:57:53.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.076+0000 7f61b24de640 1 Processor -- start 2024-02-26T04:57:53.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.076+0000 7f61b24de640 1 -- start start 2024-02-26T04:57:53.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.076+0000 7f61b24de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61ac100090 0x7f61ac1960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:53.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.076+0000 7f61b24de640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61ac101280 0x7f61ac1965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:53.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.076+0000 7f61b24de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61ac101c40 0x7f61ac19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.076+0000 7f61b24de640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac19dba0 con 0x7f61ac101c40 2024-02-26T04:57:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.076+0000 7f61b24de640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac19dd10 con 0x7f61ac101280 2024-02-26T04:57:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.076+0000 7f61b24de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac19ded0 con 0x7f61ac100090 2024-02-26T04:57:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.076+0000 7f61b0a7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61ac101c40 0x7f61ac19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.076+0000 7f61b0a7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61ac101c40 0x7f61ac19d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44014/0 (socket says 172.21.15.60:44014) 2024-02-26T04:57:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.076+0000 7f61b0a7b640 1 -- 172.21.15.60:0/1659840043 learned_addr learned my addr 172.21.15.60:0/1659840043 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.077+0000 7f61ab7fe640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61ac101280 0x7f61ac1965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.077+0000 7f61abfff640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61ac100090 0x7f61ac1960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.077+0000 7f61b0a7b640 1 -- 172.21.15.60:0/1659840043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61ac100090 msgr2=0x7f61ac1960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.077+0000 7f61b0a7b640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61ac100090 0x7f61ac1960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.077+0000 7f61b0a7b640 1 -- 172.21.15.60:0/1659840043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61ac101280 msgr2=0x7f61ac1965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.077+0000 7f61b0a7b640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61ac101280 0x7f61ac1965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.077+0000 7f61b0a7b640 1 -- 172.21.15.60:0/1659840043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a4009900 con 0x7f61ac101c40 2024-02-26T04:57:53.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.077+0000 7f61ab7fe640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61ac101280 0x7f61ac1965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:53.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.077+0000 7f61b0a7b640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61ac101c40 0x7f61ac19d660 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f61a4037040 tx=0x7f61a4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:53.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.077+0000 7f61a97fa640 1 -- 172.21.15.60:0/1659840043 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a40329b0 con 0x7f61ac101c40 2024-02-26T04:57:53.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.077+0000 7f61a97fa640 1 -- 172.21.15.60:0/1659840043 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61a4003e50 con 0x7f61ac101c40 2024-02-26T04:57:53.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.077+0000 7f61b24de640 1 -- 172.21.15.60:0/1659840043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ac19e150 con 0x7f61ac101c40 2024-02-26T04:57:53.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.077+0000 7f61a97fa640 1 -- 172.21.15.60:0/1659840043 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4040450 con 0x7f61ac101c40 2024-02-26T04:57:53.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.077+0000 7f61b24de640 1 -- 172.21.15.60:0/1659840043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac19e600 con 0x7f61ac101c40 2024-02-26T04:57:53.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.078+0000 7f61b24de640 1 -- 172.21.15.60:0/1659840043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ac1073b0 con 0x7f61ac101c40 2024-02-26T04:57:53.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.079+0000 7f61a97fa640 1 -- 172.21.15.60:0/1659840043 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f61a400a040 con 0x7f61ac101c40 2024-02-26T04:57:53.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.080+0000 7f61a97fa640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f61980a20c0 0x7f61980a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:53.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.080+0000 7f61abfff640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f61980a20c0 0x7f61980a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:53.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.080+0000 7f61a97fa640 1 -- 172.21.15.60:0/1659840043 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f61a40f0f80 con 0x7f61ac101c40 2024-02-26T04:57:53.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.081+0000 7f61abfff640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f61980a20c0 0x7f61980a4580 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f61ac1010e0 tx=0x7f619c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.085+0000 7f61a97fa640 1 -- 172.21.15.60:0/1659840043 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f61a40b9ba0 con 0x7f61ac101c40 2024-02-26T04:57:53.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.263+0000 7f61b24de640 1 -- 172.21.15.60:0/1659840043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f61ac02cda0 con 0x7f61ac101c40 2024-02-26T04:57:53.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.263+0000 7f61a97fa640 1 -- 172.21.15.60:0/1659840043 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f61a4034e60 con 0x7f61ac101c40 2024-02-26T04:57:53.266 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:57:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.265+0000 7f61b24de640 1 -- 172.21.15.60:0/1659840043 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f61980a20c0 msgr2=0x7f61980a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.265+0000 7f61b24de640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f61980a20c0 0x7f61980a4580 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f61ac1010e0 tx=0x7f619c005f90 comp rx=0 tx=0).stop 2024-02-26T04:57:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.265+0000 7f61b24de640 1 -- 172.21.15.60:0/1659840043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61ac101c40 msgr2=0x7f61ac19d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.265+0000 7f61b24de640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61ac101c40 0x7f61ac19d660 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f61a4037040 tx=0x7f61a4009360 comp rx=0 tx=0).stop 2024-02-26T04:57:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.266+0000 7f61b24de640 1 -- 172.21.15.60:0/1659840043 shutdown_connections 2024-02-26T04:57:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.266+0000 7f61b24de640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f61980a20c0 0x7f61980a4580 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.266+0000 7f61b24de640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61ac101c40 0x7f61ac19d660 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.266+0000 7f61b24de640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61ac101280 0x7f61ac1965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.266+0000 7f61b24de640 1 --2- 172.21.15.60:0/1659840043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f61ac100090 0x7f61ac1960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.266+0000 7f61b24de640 1 -- 172.21.15.60:0/1659840043 >> 172.21.15.60:0/1659840043 conn(0x7f61ac0fb620 msgr2=0x7f61ac104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.266+0000 7f61b24de640 1 -- 172.21.15.60:0/1659840043 shutdown_connections 2024-02-26T04:57:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.266+0000 7f61b24de640 1 -- 172.21.15.60:0/1659840043 wait complete. 2024-02-26T04:57:53.279 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T04:57:53.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.383+0000 7ff70e2de640 1 -- 172.21.15.60:0/1795566928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff708101cb0 msgr2=0x7ff7081040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.383+0000 7ff70e2de640 1 --2- 172.21.15.60:0/1795566928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff708101cb0 0x7ff7081040a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009900 tx=0x7ff6fc02bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.383+0000 7ff70e2de640 1 -- 172.21.15.60:0/1795566928 shutdown_connections 2024-02-26T04:57:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.383+0000 7ff70e2de640 1 --2- 172.21.15.60:0/1795566928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff708106f10 0x7ff708109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.383+0000 7ff70e2de640 1 --2- 172.21.15.60:0/1795566928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7081045e0 0x7ff7081069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.383+0000 7ff70e2de640 1 --2- 172.21.15.60:0/1795566928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff708101cb0 0x7ff7081040a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.383+0000 7ff70e2de640 1 -- 172.21.15.60:0/1795566928 >> 172.21.15.60:0/1795566928 conn(0x7ff7080fb620 msgr2=0x7ff7080fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.383+0000 7ff70e2de640 1 -- 172.21.15.60:0/1795566928 shutdown_connections 2024-02-26T04:57:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.383+0000 7ff70e2de640 1 -- 172.21.15.60:0/1795566928 wait complete. 2024-02-26T04:57:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.383+0000 7ff70e2de640 1 Processor -- start 2024-02-26T04:57:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.383+0000 7ff70e2de640 1 -- start start 2024-02-26T04:57:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.383+0000 7ff70e2de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff708101cb0 0x7ff708197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.383+0000 7ff70e2de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7081045e0 0x7ff708197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:53.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff70e2de640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff708106f10 0x7ff70819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:53.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff70e2de640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70819ee10 con 0x7ff708101cb0 2024-02-26T04:57:53.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff70e2de640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70819ef80 con 0x7ff708106f10 2024-02-26T04:57:53.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff70e2de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70819f140 con 0x7ff7081045e0 2024-02-26T04:57:53.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff707fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff708101cb0 0x7ff708197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:53.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff707fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff708101cb0 0x7ff708197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44030/0 (socket says 172.21.15.60:44030) 2024-02-26T04:57:53.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff707fff640 1 -- 172.21.15.60:0/3431964075 learned_addr learned my addr 172.21.15.60:0/3431964075 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:53.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff70c87b640 1 --2- 172.21.15.60:0/3431964075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff708106f10 0x7ff70819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:53.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff7077fe640 1 --2- 172.21.15.60:0/3431964075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7081045e0 0x7ff708197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:53.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff707fff640 1 -- 172.21.15.60:0/3431964075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7081045e0 msgr2=0x7ff708197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff707fff640 1 --2- 172.21.15.60:0/3431964075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7081045e0 0x7ff708197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff707fff640 1 -- 172.21.15.60:0/3431964075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff708106f10 msgr2=0x7ff70819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff707fff640 1 --2- 172.21.15.60:0/3431964075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff708106f10 0x7ff70819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff707fff640 1 -- 172.21.15.60:0/3431964075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc009560 con 0x7ff708101cb0 2024-02-26T04:57:53.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff707fff640 1 --2- 172.21.15.60:0/3431964075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff708101cb0 0x7ff708197320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc006010 tx=0x7ff6fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:53.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff7057fa640 1 -- 172.21.15.60:0/3431964075 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0044e0 con 0x7ff708101cb0 2024-02-26T04:57:53.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff70e2de640 1 -- 172.21.15.60:0/3431964075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70819f3c0 con 0x7ff708101cb0 2024-02-26T04:57:53.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff7057fa640 1 -- 172.21.15.60:0/3431964075 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6fc0054e0 con 0x7ff708101cb0 2024-02-26T04:57:53.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff7057fa640 1 -- 172.21.15.60:0/3431964075 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc03e450 con 0x7ff708101cb0 2024-02-26T04:57:53.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.384+0000 7ff70e2de640 1 -- 172.21.15.60:0/3431964075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70819f950 con 0x7ff708101cb0 2024-02-26T04:57:53.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.385+0000 7ff70e2de640 1 -- 172.21.15.60:0/3431964075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7080fd110 con 0x7ff708101cb0 2024-02-26T04:57:53.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.386+0000 7ff7057fa640 1 -- 172.21.15.60:0/3431964075 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff6fc0035f0 con 0x7ff708101cb0 2024-02-26T04:57:53.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.387+0000 7ff7057fa640 1 --2- 172.21.15.60:0/3431964075 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6f00a20c0 0x7ff6f00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:53.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.387+0000 7ff7077fe640 1 --2- 172.21.15.60:0/3431964075 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6f00a20c0 0x7ff6f00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:53.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.387+0000 7ff7057fa640 1 -- 172.21.15.60:0/3431964075 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff6fc0f05c0 con 0x7ff708101cb0 2024-02-26T04:57:53.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.388+0000 7ff7077fe640 1 --2- 172.21.15.60:0/3431964075 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6f00a20c0 0x7ff6f00a4580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff708198830 tx=0x7ff6f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:53.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.392+0000 7ff7057fa640 1 -- 172.21.15.60:0/3431964075 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff6fc0b90f0 con 0x7ff708101cb0 2024-02-26T04:57:53.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.574+0000 7ff70e2de640 1 -- 172.21.15.60:0/3431964075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff70802cdd0 con 0x7ff708101cb0 2024-02-26T04:57:53.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.574+0000 7ff7057fa640 1 -- 172.21.15.60:0/3431964075 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff6fc0bc7b0 con 0x7ff708101cb0 2024-02-26T04:57:53.577 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T04:57:53.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.576+0000 7ff70e2de640 1 -- 172.21.15.60:0/3431964075 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6f00a20c0 msgr2=0x7ff6f00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.576+0000 7ff70e2de640 1 --2- 172.21.15.60:0/3431964075 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6f00a20c0 0x7ff6f00a4580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff708198830 tx=0x7ff6f8008040 comp rx=0 tx=0).stop 2024-02-26T04:57:53.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.576+0000 7ff70e2de640 1 -- 172.21.15.60:0/3431964075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff708101cb0 msgr2=0x7ff708197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.576+0000 7ff70e2de640 1 --2- 172.21.15.60:0/3431964075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff708101cb0 0x7ff708197320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc006010 tx=0x7ff6fc0042f0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.576+0000 7ff70e2de640 1 -- 172.21.15.60:0/3431964075 shutdown_connections 2024-02-26T04:57:53.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.576+0000 7ff70e2de640 1 --2- 172.21.15.60:0/3431964075 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6f00a20c0 0x7ff6f00a4580 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.576+0000 7ff70e2de640 1 --2- 172.21.15.60:0/3431964075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff708106f10 0x7ff70819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.576+0000 7ff70e2de640 1 --2- 172.21.15.60:0/3431964075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7081045e0 0x7ff708197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.576+0000 7ff70e2de640 1 --2- 172.21.15.60:0/3431964075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff708101cb0 0x7ff708197320 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.576+0000 7ff70e2de640 1 -- 172.21.15.60:0/3431964075 >> 172.21.15.60:0/3431964075 conn(0x7ff7080fb620 msgr2=0x7ff7081026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:53.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.576+0000 7ff70e2de640 1 -- 172.21.15.60:0/3431964075 shutdown_connections 2024-02-26T04:57:53.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.576+0000 7ff70e2de640 1 -- 172.21.15.60:0/3431964075 wait complete. 2024-02-26T04:57:53.588 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:53 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1702195239' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-02-26T04:57:53.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:53 smithi060 ceph-mon[32382]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T04:57:53.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:53 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1373352570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:53.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:53 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/590092318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:53.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:53 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1659840043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:53.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T04:57:53.680 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:53 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1702195239' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-02-26T04:57:53.680 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:53 smithi156 ceph-mon[35805]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T04:57:53.680 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:53 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1373352570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:53.680 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:53 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/590092318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:53.680 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:53 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1659840043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:53.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.694+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1753108577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b0101c50 msgr2=0x7fb3b01084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.694+0000 7fb3b65cf640 1 --2- 172.21.15.60:0/1753108577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b0101c50 0x7fb3b01084e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009ca0 tx=0x7fb3a802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.694+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1753108577 shutdown_connections 2024-02-26T04:57:53.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.694+0000 7fb3b65cf640 1 --2- 172.21.15.60:0/1753108577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b0101c50 0x7fb3b01084e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.694+0000 7fb3b65cf640 1 --2- 172.21.15.60:0/1753108577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b0101290 0x7fb3b0101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.694+0000 7fb3b65cf640 1 --2- 172.21.15.60:0/1753108577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3b01000a0 0x7fb3b01004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.694+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1753108577 >> 172.21.15.60:0/1753108577 conn(0x7fb3b00fb630 msgr2=0x7fb3b00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:53.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.694+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1753108577 shutdown_connections 2024-02-26T04:57:53.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.694+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1753108577 wait complete. 2024-02-26T04:57:53.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.694+0000 7fb3b65cf640 1 Processor -- start 2024-02-26T04:57:53.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.694+0000 7fb3b65cf640 1 -- start start 2024-02-26T04:57:53.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3b65cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b01000a0 0x7fb3b01971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3b65cf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3b0101290 0x7fb3b01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3b65cf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b0101c50 0x7fb3b019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3b65cf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b019ec90 con 0x7fb3b01000a0 2024-02-26T04:57:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3b65cf640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b019ee00 con 0x7fb3b0101c50 2024-02-26T04:57:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3b65cf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b019efc0 con 0x7fb3b0101290 2024-02-26T04:57:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3affff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b01000a0 0x7fb3b01971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3affff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b01000a0 0x7fb3b01971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44044/0 (socket says 172.21.15.60:44044) 2024-02-26T04:57:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3affff640 1 -- 172.21.15.60:0/1884897861 learned_addr learned my addr 172.21.15.60:0/1884897861 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3af7fe640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3b0101290 0x7fb3b01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3b4b6c640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b0101c50 0x7fb3b019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3affff640 1 -- 172.21.15.60:0/1884897861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3b0101290 msgr2=0x7fb3b01976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3affff640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3b0101290 0x7fb3b01976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3affff640 1 -- 172.21.15.60:0/1884897861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b0101c50 msgr2=0x7fb3b019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3affff640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b0101c50 0x7fb3b019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3affff640 1 -- 172.21.15.60:0/1884897861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a8009900 con 0x7fb3b01000a0 2024-02-26T04:57:53.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3b4b6c640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b0101c50 0x7fb3b019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:53.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3affff640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b01000a0 0x7fb3b01971a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000e9c0 tx=0x7fb3a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:53.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3ad7fa640 1 -- 172.21.15.60:0/1884897861 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a00096e0 con 0x7fb3b01000a0 2024-02-26T04:57:53.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3ad7fa640 1 -- 172.21.15.60:0/1884897861 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3a000cd80 con 0x7fb3b01000a0 2024-02-26T04:57:53.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1884897861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b019f2a0 con 0x7fb3b01000a0 2024-02-26T04:57:53.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3ad7fa640 1 -- 172.21.15.60:0/1884897861 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0010430 con 0x7fb3b01000a0 2024-02-26T04:57:53.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.695+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1884897861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b019f760 con 0x7fb3b01000a0 2024-02-26T04:57:53.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.697+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1884897861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b0107360 con 0x7fb3b01000a0 2024-02-26T04:57:53.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.697+0000 7fb3ad7fa640 1 -- 172.21.15.60:0/1884897861 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb3a0003680 con 0x7fb3b01000a0 2024-02-26T04:57:53.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.698+0000 7fb3ad7fa640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb3980a20c0 0x7fb3980a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:53.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.698+0000 7fb3ad7fa640 1 -- 172.21.15.60:0/1884897861 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fb3a0014070 con 0x7fb3b01000a0 2024-02-26T04:57:53.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.698+0000 7fb3af7fe640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb3980a20c0 0x7fb3980a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:53.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.698+0000 7fb3af7fe640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb3980a20c0 0x7fb3980a4580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb3b01986b0 tx=0x7fb3a400b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:53.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.702+0000 7fb3ad7fa640 1 -- 172.21.15.60:0/1884897861 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb3a0097140 con 0x7fb3b01000a0 2024-02-26T04:57:53.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:53 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1702195239' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-02-26T04:57:53.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:53 smithi191 ceph-mon[36352]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-02-26T04:57:53.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:53 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1373352570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:53.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:53 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/590092318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:53.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:53 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1659840043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:53.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.884+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1884897861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb3b002ce40 con 0x7fb3b01000a0 2024-02-26T04:57:53.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.885+0000 7fb3ad7fa640 1 -- 172.21.15.60:0/1884897861 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb3a009a800 con 0x7fb3b01000a0 2024-02-26T04:57:53.887 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:57:53.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.887+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1884897861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb3980a20c0 msgr2=0x7fb3980a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.887+0000 7fb3b65cf640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb3980a20c0 0x7fb3980a4580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb3b01986b0 tx=0x7fb3a400b590 comp rx=0 tx=0).stop 2024-02-26T04:57:53.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.887+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1884897861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b01000a0 msgr2=0x7fb3b01971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:53.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.887+0000 7fb3b65cf640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b01000a0 0x7fb3b01971a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000e9c0 tx=0x7fb3a000ee80 comp rx=0 tx=0).stop 2024-02-26T04:57:53.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.887+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1884897861 shutdown_connections 2024-02-26T04:57:53.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.887+0000 7fb3b65cf640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb3980a20c0 0x7fb3980a4580 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.887+0000 7fb3b65cf640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b0101c50 0x7fb3b019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.887+0000 7fb3b65cf640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3b0101290 0x7fb3b01976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.887+0000 7fb3b65cf640 1 --2- 172.21.15.60:0/1884897861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b01000a0 0x7fb3b01971a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:53.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.887+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1884897861 >> 172.21.15.60:0/1884897861 conn(0x7fb3b00fb630 msgr2=0x7fb3b0104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:53.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.887+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1884897861 shutdown_connections 2024-02-26T04:57:53.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:53.887+0000 7fb3b65cf640 1 -- 172.21.15.60:0/1884897861 wait complete. 2024-02-26T04:57:53.901 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:57:54.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.000+0000 7f8afc71e640 1 -- 172.21.15.60:0/224299823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af4100120 msgr2=0x7f8af4100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.000+0000 7f8afc71e640 1 --2- 172.21.15.60:0/224299823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af4100120 0x7f8af4100540 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009900 tx=0x7f8af002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.000+0000 7f8afc71e640 1 -- 172.21.15.60:0/224299823 shutdown_connections 2024-02-26T04:57:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.000+0000 7f8afc71e640 1 --2- 172.21.15.60:0/224299823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8af4101cd0 0x7f8af4108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.000+0000 7f8afc71e640 1 --2- 172.21.15.60:0/224299823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8af4101310 0x7f8af4101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.000+0000 7f8afc71e640 1 --2- 172.21.15.60:0/224299823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af4100120 0x7f8af4100540 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.000+0000 7f8afc71e640 1 -- 172.21.15.60:0/224299823 >> 172.21.15.60:0/224299823 conn(0x7f8af40fb6b0 msgr2=0x7f8af40fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.000+0000 7f8afc71e640 1 -- 172.21.15.60:0/224299823 shutdown_connections 2024-02-26T04:57:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.000+0000 7f8afc71e640 1 -- 172.21.15.60:0/224299823 wait complete. 2024-02-26T04:57:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.001+0000 7f8afc71e640 1 Processor -- start 2024-02-26T04:57:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.001+0000 7f8afc71e640 1 -- start start 2024-02-26T04:57:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.001+0000 7f8afc71e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8af4100120 0x7f8af4197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.001+0000 7f8afc71e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8af4101310 0x7f8af41977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.001+0000 7f8afc71e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af4101cd0 0x7f8af419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.001+0000 7f8afc71e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af419ed50 con 0x7f8af4101cd0 2024-02-26T04:57:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.001+0000 7f8afc71e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af419eec0 con 0x7f8af4101310 2024-02-26T04:57:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.001+0000 7f8afc71e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af419f080 con 0x7f8af4100120 2024-02-26T04:57:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.001+0000 7f8afacbb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af4101cd0 0x7f8af419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.001+0000 7f8afacbb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af4101cd0 0x7f8af419e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44074/0 (socket says 172.21.15.60:44074) 2024-02-26T04:57:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.001+0000 7f8afacbb640 1 -- 172.21.15.60:0/3261987498 learned_addr learned my addr 172.21.15.60:0/3261987498 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.001+0000 7f8af9cb9640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8af4101310 0x7f8af41977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.002+0000 7f8afa4ba640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8af4100120 0x7f8af4197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.002+0000 7f8afacbb640 1 -- 172.21.15.60:0/3261987498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8af4100120 msgr2=0x7f8af4197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.002+0000 7f8afacbb640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8af4100120 0x7f8af4197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.002+0000 7f8afacbb640 1 -- 172.21.15.60:0/3261987498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8af4101310 msgr2=0x7f8af41977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.002+0000 7f8afacbb640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8af4101310 0x7f8af41977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.002+0000 7f8afacbb640 1 -- 172.21.15.60:0/3261987498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af0009560 con 0x7f8af4101cd0 2024-02-26T04:57:54.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.002+0000 7f8afa4ba640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8af4100120 0x7f8af4197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:54.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.002+0000 7f8afacbb640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af4101cd0 0x7f8af419e810 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00eb10 tx=0x7f8aec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:54.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.002+0000 7f8aeb7fe640 1 -- 172.21.15.60:0/3261987498 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec004b20 con 0x7f8af4101cd0 2024-02-26T04:57:54.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.002+0000 7f8afc71e640 1 -- 172.21.15.60:0/3261987498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af419f360 con 0x7f8af4101cd0 2024-02-26T04:57:54.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.002+0000 7f8aeb7fe640 1 -- 172.21.15.60:0/3261987498 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8aec004c80 con 0x7f8af4101cd0 2024-02-26T04:57:54.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.002+0000 7f8afc71e640 1 -- 172.21.15.60:0/3261987498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af419f8c0 con 0x7f8af4101cd0 2024-02-26T04:57:54.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.002+0000 7f8aeb7fe640 1 -- 172.21.15.60:0/3261987498 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec018630 con 0x7f8af4101cd0 2024-02-26T04:57:54.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.003+0000 7f8afc71e640 1 -- 172.21.15.60:0/3261987498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af410bac0 con 0x7f8af4101cd0 2024-02-26T04:57:54.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.004+0000 7f8aeb7fe640 1 -- 172.21.15.60:0/3261987498 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8aec018810 con 0x7f8af4101cd0 2024-02-26T04:57:54.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.005+0000 7f8aeb7fe640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8ae00a20c0 0x7f8ae00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.005+0000 7f8aeb7fe640 1 -- 172.21.15.60:0/3261987498 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8aec014070 con 0x7f8af4101cd0 2024-02-26T04:57:54.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.005+0000 7f8afa4ba640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8ae00a20c0 0x7f8ae00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.005+0000 7f8afa4ba640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8ae00a20c0 0x7f8ae00a4580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8af0036040 tx=0x7f8af0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:54.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.008+0000 7f8aeb7fe640 1 -- 172.21.15.60:0/3261987498 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8aec097950 con 0x7f8af4101cd0 2024-02-26T04:57:54.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.189+0000 7f8afc71e640 1 -- 172.21.15.60:0/3261987498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8af419fde0 con 0x7f8af4101cd0 2024-02-26T04:57:54.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.191+0000 7f8aeb7fe640 1 -- 172.21.15.60:0/3261987498 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24972 (secure 0 0 0) 0x7f8aec09b010 con 0x7f8af4101cd0 2024-02-26T04:57:54.193 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:54.194 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:57:21.168776+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:57:54.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.193+0000 7f8afc71e640 1 -- 172.21.15.60:0/3261987498 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8ae00a20c0 msgr2=0x7f8ae00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.193+0000 7f8afc71e640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8ae00a20c0 0x7f8ae00a4580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8af0036040 tx=0x7f8af0000bc0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.193+0000 7f8afc71e640 1 -- 172.21.15.60:0/3261987498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af4101cd0 msgr2=0x7f8af419e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.193+0000 7f8afc71e640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af4101cd0 0x7f8af419e810 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00eb10 tx=0x7f8aec00c2d0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.193+0000 7f8afc71e640 1 -- 172.21.15.60:0/3261987498 shutdown_connections 2024-02-26T04:57:54.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.193+0000 7f8afc71e640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8ae00a20c0 0x7f8ae00a4580 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.193+0000 7f8afc71e640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af4101cd0 0x7f8af419e810 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.193+0000 7f8afc71e640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8af4101310 0x7f8af41977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.193+0000 7f8afc71e640 1 --2- 172.21.15.60:0/3261987498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8af4100120 0x7f8af4197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.193+0000 7f8afc71e640 1 -- 172.21.15.60:0/3261987498 >> 172.21.15.60:0/3261987498 conn(0x7f8af40fb6b0 msgr2=0x7f8af40fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:54.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.193+0000 7f8afc71e640 1 -- 172.21.15.60:0/3261987498 shutdown_connections 2024-02-26T04:57:54.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.193+0000 7f8afc71e640 1 -- 172.21.15.60:0/3261987498 wait complete. 2024-02-26T04:57:54.207 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T04:50:44.572921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T04:55:25.530720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T04:55:26.741604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T04:55:30.453569+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T04:57:54.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T04:57:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.311+0000 7ff985ac4640 1 -- 172.21.15.60:0/3042962135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff980101190 msgr2=0x7ff9801015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.311+0000 7ff985ac4640 1 --2- 172.21.15.60:0/3042962135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff980101190 0x7ff9801015b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff970009900 tx=0x7ff97002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.311+0000 7ff985ac4640 1 -- 172.21.15.60:0/3042962135 shutdown_connections 2024-02-26T04:57:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.311+0000 7ff985ac4640 1 --2- 172.21.15.60:0/3042962135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff980102d40 0x7ff9801095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.311+0000 7ff985ac4640 1 --2- 172.21.15.60:0/3042962135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff980102380 0x7ff980102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.311+0000 7ff985ac4640 1 --2- 172.21.15.60:0/3042962135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff980101190 0x7ff9801015b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.311+0000 7ff985ac4640 1 -- 172.21.15.60:0/3042962135 >> 172.21.15.60:0/3042962135 conn(0x7ff9800fc720 msgr2=0x7ff9800feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.311+0000 7ff985ac4640 1 -- 172.21.15.60:0/3042962135 shutdown_connections 2024-02-26T04:57:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.311+0000 7ff985ac4640 1 -- 172.21.15.60:0/3042962135 wait complete. 2024-02-26T04:57:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.312+0000 7ff985ac4640 1 Processor -- start 2024-02-26T04:57:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.312+0000 7ff985ac4640 1 -- start start 2024-02-26T04:57:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.312+0000 7ff985ac4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff980101190 0x7ff9801972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.312+0000 7ff985ac4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff980102380 0x7ff9801977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.312+0000 7ff985ac4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff980102d40 0x7ff98019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.312+0000 7ff985ac4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98019ed90 con 0x7ff980101190 2024-02-26T04:57:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.312+0000 7ff985ac4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98019ef00 con 0x7ff980102380 2024-02-26T04:57:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.312+0000 7ff985ac4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98019f0c0 con 0x7ff980102d40 2024-02-26T04:57:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.312+0000 7ff97f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff980101190 0x7ff9801972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.312+0000 7ff97f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff980101190 0x7ff9801972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44092/0 (socket says 172.21.15.60:44092) 2024-02-26T04:57:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.312+0000 7ff97f7fe640 1 -- 172.21.15.60:0/1954917218 learned_addr learned my addr 172.21.15.60:0/1954917218 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.312+0000 7ff97effd640 1 --2- 172.21.15.60:0/1954917218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff980102380 0x7ff9801977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.312+0000 7ff97ffff640 1 --2- 172.21.15.60:0/1954917218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff980102d40 0x7ff98019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.313+0000 7ff97f7fe640 1 -- 172.21.15.60:0/1954917218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff980102d40 msgr2=0x7ff98019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.313+0000 7ff97f7fe640 1 --2- 172.21.15.60:0/1954917218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff980102d40 0x7ff98019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.313+0000 7ff97f7fe640 1 -- 172.21.15.60:0/1954917218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff980102380 msgr2=0x7ff9801977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.313+0000 7ff97f7fe640 1 --2- 172.21.15.60:0/1954917218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff980102380 0x7ff9801977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.313+0000 7ff97f7fe640 1 -- 172.21.15.60:0/1954917218 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff970009560 con 0x7ff980101190 2024-02-26T04:57:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.313+0000 7ff97f7fe640 1 --2- 172.21.15.60:0/1954917218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff980101190 0x7ff9801972a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff970005ab0 tx=0x7ff970004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.313+0000 7ff97cff9640 1 -- 172.21.15.60:0/1954917218 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970004aa0 con 0x7ff980101190 2024-02-26T04:57:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.313+0000 7ff97cff9640 1 -- 172.21.15.60:0/1954917218 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9700052d0 con 0x7ff980101190 2024-02-26T04:57:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.313+0000 7ff985ac4640 1 -- 172.21.15.60:0/1954917218 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98019f340 con 0x7ff980101190 2024-02-26T04:57:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.313+0000 7ff97cff9640 1 -- 172.21.15.60:0/1954917218 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970003e80 con 0x7ff980101190 2024-02-26T04:57:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.313+0000 7ff985ac4640 1 -- 172.21.15.60:0/1954917218 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98019f850 con 0x7ff980101190 2024-02-26T04:57:54.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.314+0000 7ff985ac4640 1 -- 172.21.15.60:0/1954917218 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff980108550 con 0x7ff980101190 2024-02-26T04:57:54.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.315+0000 7ff97cff9640 1 -- 172.21.15.60:0/1954917218 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff970005440 con 0x7ff980101190 2024-02-26T04:57:54.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.316+0000 7ff97cff9640 1 --2- 172.21.15.60:0/1954917218 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff9680a20c0 0x7ff9680a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.316+0000 7ff97effd640 1 --2- 172.21.15.60:0/1954917218 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff9680a20c0 0x7ff9680a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.316+0000 7ff97cff9640 1 -- 172.21.15.60:0/1954917218 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff9700f0540 con 0x7ff980101190 2024-02-26T04:57:54.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.316+0000 7ff97effd640 1 --2- 172.21.15.60:0/1954917218 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff9680a20c0 0x7ff9680a4580 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff9801987b0 tx=0x7ff97400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:54.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.320+0000 7ff97cff9640 1 -- 172.21.15.60:0/1954917218 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff9700b9070 con 0x7ff980101190 2024-02-26T04:57:54.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:54 smithi060 ceph-mon[32382]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:57:54.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:54 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3431964075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:54.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:54 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1884897861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:54.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:54 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3261987498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.502+0000 7ff985ac4640 1 -- 172.21.15.60:0/1954917218 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff98002cdd0 con 0x7ff980101190 2024-02-26T04:57:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.503+0000 7ff97cff9640 1 -- 172.21.15.60:0/1954917218 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9700bc730 con 0x7ff980101190 2024-02-26T04:57:54.505 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T04:57:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.505+0000 7ff985ac4640 1 -- 172.21.15.60:0/1954917218 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff9680a20c0 msgr2=0x7ff9680a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.505+0000 7ff985ac4640 1 --2- 172.21.15.60:0/1954917218 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff9680a20c0 0x7ff9680a4580 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff9801987b0 tx=0x7ff97400b390 comp rx=0 tx=0).stop 2024-02-26T04:57:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.505+0000 7ff985ac4640 1 -- 172.21.15.60:0/1954917218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff980101190 msgr2=0x7ff9801972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.505+0000 7ff985ac4640 1 --2- 172.21.15.60:0/1954917218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff980101190 0x7ff9801972a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff970005ab0 tx=0x7ff970004270 comp rx=0 tx=0).stop 2024-02-26T04:57:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.505+0000 7ff985ac4640 1 -- 172.21.15.60:0/1954917218 shutdown_connections 2024-02-26T04:57:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.505+0000 7ff985ac4640 1 --2- 172.21.15.60:0/1954917218 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff9680a20c0 0x7ff9680a4580 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.505+0000 7ff985ac4640 1 --2- 172.21.15.60:0/1954917218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff980102d40 0x7ff98019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.505+0000 7ff985ac4640 1 --2- 172.21.15.60:0/1954917218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff980102380 0x7ff9801977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.505+0000 7ff985ac4640 1 --2- 172.21.15.60:0/1954917218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff980101190 0x7ff9801972a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.505+0000 7ff985ac4640 1 -- 172.21.15.60:0/1954917218 >> 172.21.15.60:0/1954917218 conn(0x7ff9800fc720 msgr2=0x7ff980105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.505+0000 7ff985ac4640 1 -- 172.21.15.60:0/1954917218 shutdown_connections 2024-02-26T04:57:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.505+0000 7ff985ac4640 1 -- 172.21.15.60:0/1954917218 wait complete. 2024-02-26T04:57:54.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T04:57:54.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.621+0000 7f342861c640 1 -- 172.21.15.60:0/1635828346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34201011a0 msgr2=0x7f34201015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.621+0000 7f342861c640 1 --2- 172.21.15.60:0/1635828346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34201011a0 0x7f34201015c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f341c009900 tx=0x7f341c02bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:54.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.621+0000 7f342861c640 1 -- 172.21.15.60:0/1635828346 shutdown_connections 2024-02-26T04:57:54.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.621+0000 7f342861c640 1 --2- 172.21.15.60:0/1635828346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420102d50 0x7f34201095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.621+0000 7f342861c640 1 --2- 172.21.15.60:0/1635828346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3420102390 0x7f3420102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.621+0000 7f342861c640 1 --2- 172.21.15.60:0/1635828346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34201011a0 0x7f34201015c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.621+0000 7f342861c640 1 -- 172.21.15.60:0/1635828346 >> 172.21.15.60:0/1635828346 conn(0x7f34200fc6f0 msgr2=0x7f34200feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:54.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.621+0000 7f342861c640 1 -- 172.21.15.60:0/1635828346 shutdown_connections 2024-02-26T04:57:54.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.621+0000 7f342861c640 1 -- 172.21.15.60:0/1635828346 wait complete. 2024-02-26T04:57:54.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f342861c640 1 Processor -- start 2024-02-26T04:57:54.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f342861c640 1 -- start start 2024-02-26T04:57:54.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f342861c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34201011a0 0x7f3420074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f342861c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420102390 0x7f3420073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f342861c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3420102d50 0x7f3420073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f342861c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3420074210 con 0x7f34201011a0 2024-02-26T04:57:54.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f342861c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3420074380 con 0x7f3420102d50 2024-02-26T04:57:54.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f342861c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34201a1270 con 0x7f3420102390 2024-02-26T04:57:54.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f34263b8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34201011a0 0x7f3420074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f34263b8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34201011a0 0x7f3420074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44116/0 (socket says 172.21.15.60:44116) 2024-02-26T04:57:54.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f34263b8640 1 -- 172.21.15.60:0/1314751997 learned_addr learned my addr 172.21.15.60:0/1314751997 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:54.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f3425bb7640 1 --2- 172.21.15.60:0/1314751997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420102390 0x7f3420073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f34263b8640 1 -- 172.21.15.60:0/1314751997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420102390 msgr2=0x7f3420073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f34263b8640 1 --2- 172.21.15.60:0/1314751997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420102390 0x7f3420073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f34263b8640 1 -- 172.21.15.60:0/1314751997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3420102d50 msgr2=0x7f3420073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f34263b8640 1 --2- 172.21.15.60:0/1314751997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3420102d50 0x7f3420073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f34263b8640 1 -- 172.21.15.60:0/1314751997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c009560 con 0x7f34201011a0 2024-02-26T04:57:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.622+0000 7f3425bb7640 1 --2- 172.21.15.60:0/1314751997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420102390 0x7f3420073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.623+0000 7f34263b8640 1 --2- 172.21.15.60:0/1314751997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34201011a0 0x7f3420074f30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f341c036040 tx=0x7f341c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.623+0000 7f34177fe640 1 -- 172.21.15.60:0/1314751997 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c004df0 con 0x7f34201011a0 2024-02-26T04:57:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.623+0000 7f34177fe640 1 -- 172.21.15.60:0/1314751997 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f341c005430 con 0x7f34201011a0 2024-02-26T04:57:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.623+0000 7f342861c640 1 -- 172.21.15.60:0/1314751997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34201a14c0 con 0x7f34201011a0 2024-02-26T04:57:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.623+0000 7f34177fe640 1 -- 172.21.15.60:0/1314751997 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c03e450 con 0x7f34201011a0 2024-02-26T04:57:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.623+0000 7f342861c640 1 -- 172.21.15.60:0/1314751997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34201a19d0 con 0x7f34201011a0 2024-02-26T04:57:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.624+0000 7f342861c640 1 -- 172.21.15.60:0/1314751997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3420108510 con 0x7f34201011a0 2024-02-26T04:57:54.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.625+0000 7f34177fe640 1 -- 172.21.15.60:0/1314751997 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f341c003680 con 0x7f34201011a0 2024-02-26T04:57:54.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.625+0000 7f34177fe640 1 --2- 172.21.15.60:0/1314751997 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f340c0a20c0 0x7f340c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.625+0000 7f3425bb7640 1 --2- 172.21.15.60:0/1314751997 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f340c0a20c0 0x7f340c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.626+0000 7f34177fe640 1 -- 172.21.15.60:0/1314751997 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f341c0f0cc0 con 0x7f34201011a0 2024-02-26T04:57:54.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.626+0000 7f3425bb7640 1 --2- 172.21.15.60:0/1314751997 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f340c0a20c0 0x7f340c0a4580 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f34100058c0 tx=0x7f3410005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:54.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.629+0000 7f34177fe640 1 -- 172.21.15.60:0/1314751997 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f341c0b97f0 con 0x7f34201011a0 2024-02-26T04:57:54.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:54 smithi156 ceph-mon[35805]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:57:54.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:54 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3431964075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:54.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:54 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1884897861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:54.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:54 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3261987498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:54.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:54 smithi191 ceph-mon[36352]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:57:54.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:54 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3431964075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:54.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:54 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1884897861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:54.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:54 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3261987498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:54.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.808+0000 7f342861c640 1 -- 172.21.15.60:0/1314751997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f342002cea0 con 0x7f34201011a0 2024-02-26T04:57:54.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.809+0000 7f34177fe640 1 -- 172.21.15.60:0/1314751997 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f341c0bceb0 con 0x7f34201011a0 2024-02-26T04:57:54.811 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:57:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.810+0000 7f342861c640 1 -- 172.21.15.60:0/1314751997 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f340c0a20c0 msgr2=0x7f340c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.810+0000 7f342861c640 1 --2- 172.21.15.60:0/1314751997 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f340c0a20c0 0x7f340c0a4580 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f34100058c0 tx=0x7f3410005850 comp rx=0 tx=0).stop 2024-02-26T04:57:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.810+0000 7f342861c640 1 -- 172.21.15.60:0/1314751997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34201011a0 msgr2=0x7f3420074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.810+0000 7f342861c640 1 --2- 172.21.15.60:0/1314751997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34201011a0 0x7f3420074f30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f341c036040 tx=0x7f341c004270 comp rx=0 tx=0).stop 2024-02-26T04:57:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.810+0000 7f342861c640 1 -- 172.21.15.60:0/1314751997 shutdown_connections 2024-02-26T04:57:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.810+0000 7f342861c640 1 --2- 172.21.15.60:0/1314751997 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f340c0a20c0 0x7f340c0a4580 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.810+0000 7f342861c640 1 --2- 172.21.15.60:0/1314751997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3420102d50 0x7f3420073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.810+0000 7f342861c640 1 --2- 172.21.15.60:0/1314751997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420102390 0x7f3420073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.810+0000 7f342861c640 1 --2- 172.21.15.60:0/1314751997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34201011a0 0x7f3420074f30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.810+0000 7f342861c640 1 -- 172.21.15.60:0/1314751997 >> 172.21.15.60:0/1314751997 conn(0x7f34200fc6f0 msgr2=0x7f34201056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:54.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.811+0000 7f342861c640 1 -- 172.21.15.60:0/1314751997 shutdown_connections 2024-02-26T04:57:54.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.811+0000 7f342861c640 1 -- 172.21.15.60:0/1314751997 wait complete. 2024-02-26T04:57:54.824 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T04:57:54.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.927+0000 7f6165236640 1 -- 172.21.15.60:0/2713160155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6160100080 msgr2=0x7f61601004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.927+0000 7f6165236640 1 --2- 172.21.15.60:0/2713160155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6160100080 0x7f61601004a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6150009900 tx=0x7f615002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:54.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.927+0000 7f6165236640 1 -- 172.21.15.60:0/2713160155 shutdown_connections 2024-02-26T04:57:54.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.927+0000 7f6165236640 1 --2- 172.21.15.60:0/2713160155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6160101c30 0x7f61601084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.927+0000 7f6165236640 1 --2- 172.21.15.60:0/2713160155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6160101270 0x7f61601016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.927+0000 7f6165236640 1 --2- 172.21.15.60:0/2713160155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6160100080 0x7f61601004a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.927+0000 7f6165236640 1 -- 172.21.15.60:0/2713160155 >> 172.21.15.60:0/2713160155 conn(0x7f61600fb610 msgr2=0x7f61600fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:54.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.927+0000 7f6165236640 1 -- 172.21.15.60:0/2713160155 shutdown_connections 2024-02-26T04:57:54.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.927+0000 7f6165236640 1 -- 172.21.15.60:0/2713160155 wait complete. 2024-02-26T04:57:54.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6165236640 1 Processor -- start 2024-02-26T04:57:54.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6165236640 1 -- start start 2024-02-26T04:57:54.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6165236640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6160100080 0x7f6160197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6165236640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6160101270 0x7f6160197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6165236640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6160101c30 0x7f616019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6165236640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616019ed00 con 0x7f6160101270 2024-02-26T04:57:54.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6165236640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616019ee70 con 0x7f6160100080 2024-02-26T04:57:54.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6165236640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616019f030 con 0x7f6160101c30 2024-02-26T04:57:54.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6157fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6160101270 0x7f6160197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6157fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6160101270 0x7f6160197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44126/0 (socket says 172.21.15.60:44126) 2024-02-26T04:57:54.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6157fff640 1 -- 172.21.15.60:0/1088831720 learned_addr learned my addr 172.21.15.60:0/1088831720 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f615ed9d640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6160100080 0x7f6160197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f615f59e640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6160101c30 0x7f616019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6157fff640 1 -- 172.21.15.60:0/1088831720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6160101c30 msgr2=0x7f616019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6157fff640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6160101c30 0x7f616019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6157fff640 1 -- 172.21.15.60:0/1088831720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6160100080 msgr2=0x7f6160197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6157fff640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6160100080 0x7f6160197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f6157fff640 1 -- 172.21.15.60:0/1088831720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6150009560 con 0x7f6160101270 2024-02-26T04:57:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.928+0000 7f615f59e640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6160101c30 0x7f616019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.929+0000 7f6157fff640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6160101270 0x7f6160197750 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f614800c890 tx=0x7f614800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.929+0000 7f615cd99640 1 -- 172.21.15.60:0/1088831720 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148004220 con 0x7f6160101270 2024-02-26T04:57:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.929+0000 7f6165236640 1 -- 172.21.15.60:0/1088831720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616019f310 con 0x7f6160101270 2024-02-26T04:57:54.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.929+0000 7f615cd99640 1 -- 172.21.15.60:0/1088831720 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6148007e90 con 0x7f6160101270 2024-02-26T04:57:54.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.929+0000 7f615cd99640 1 -- 172.21.15.60:0/1088831720 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148004a00 con 0x7f6160101270 2024-02-26T04:57:54.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.929+0000 7f6165236640 1 -- 172.21.15.60:0/1088831720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616019f870 con 0x7f6160101270 2024-02-26T04:57:54.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.930+0000 7f6165236640 1 -- 172.21.15.60:0/1088831720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616010ba50 con 0x7f6160101270 2024-02-26T04:57:54.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.931+0000 7f615cd99640 1 -- 172.21.15.60:0/1088831720 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f61480079b0 con 0x7f6160101270 2024-02-26T04:57:54.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.931+0000 7f615cd99640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f614c0a20c0 0x7f614c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:54.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.931+0000 7f615ed9d640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f614c0a20c0 0x7f614c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:54.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.931+0000 7f615cd99640 1 -- 172.21.15.60:0/1088831720 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f61480ce250 con 0x7f6160101270 2024-02-26T04:57:54.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.932+0000 7f615ed9d640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f614c0a20c0 0x7f614c0a4580 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6150006010 tx=0x7f6150000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:54.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:54.935+0000 7f615cd99640 1 -- 172.21.15.60:0/1088831720 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6148096de0 con 0x7f6160101270 2024-02-26T04:57:55.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.130+0000 7f6165236640 1 -- 172.21.15.60:0/1088831720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f616010bbe0 con 0x7f6160101270 2024-02-26T04:57:55.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.130+0000 7f615cd99640 1 -- 172.21.15.60:0/1088831720 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f614801d020 con 0x7f6160101270 2024-02-26T04:57:55.133 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T04:57:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.132+0000 7f6165236640 1 -- 172.21.15.60:0/1088831720 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f614c0a20c0 msgr2=0x7f614c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.132+0000 7f6165236640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f614c0a20c0 0x7f614c0a4580 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6150006010 tx=0x7f6150000bc0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.132+0000 7f6165236640 1 -- 172.21.15.60:0/1088831720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6160101270 msgr2=0x7f6160197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.132+0000 7f6165236640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6160101270 0x7f6160197750 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f614800c890 tx=0x7f614800cd50 comp rx=0 tx=0).stop 2024-02-26T04:57:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.132+0000 7f6165236640 1 -- 172.21.15.60:0/1088831720 shutdown_connections 2024-02-26T04:57:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.132+0000 7f6165236640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f614c0a20c0 0x7f614c0a4580 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.132+0000 7f6165236640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6160101c30 0x7f616019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.132+0000 7f6165236640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6160101270 0x7f6160197750 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.132+0000 7f6165236640 1 --2- 172.21.15.60:0/1088831720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6160100080 0x7f6160197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.132+0000 7f6165236640 1 -- 172.21.15.60:0/1088831720 >> 172.21.15.60:0/1088831720 conn(0x7f61600fb610 msgr2=0x7f61600682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.132+0000 7f6165236640 1 -- 172.21.15.60:0/1088831720 shutdown_connections 2024-02-26T04:57:55.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.132+0000 7f6165236640 1 -- 172.21.15.60:0/1088831720 wait complete. 2024-02-26T04:57:55.146 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T04:57:55.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.249+0000 7f0af8484640 1 -- 172.21.15.60:0/834540154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0102380 msgr2=0x7f0af0102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.249+0000 7f0af8484640 1 --2- 172.21.15.60:0/834540154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0102380 0x7f0af0102800 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009900 tx=0x7f0ae002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:55.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.249+0000 7f0af8484640 1 -- 172.21.15.60:0/834540154 shutdown_connections 2024-02-26T04:57:55.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.249+0000 7f0af8484640 1 --2- 172.21.15.60:0/834540154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af0102d40 0x7f0af01095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.249+0000 7f0af8484640 1 --2- 172.21.15.60:0/834540154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0102380 0x7f0af0102800 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.249+0000 7f0af8484640 1 --2- 172.21.15.60:0/834540154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af0101190 0x7f0af01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.249+0000 7f0af8484640 1 -- 172.21.15.60:0/834540154 >> 172.21.15.60:0/834540154 conn(0x7f0af00fc720 msgr2=0x7f0af00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:55.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.249+0000 7f0af8484640 1 -- 172.21.15.60:0/834540154 shutdown_connections 2024-02-26T04:57:55.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.249+0000 7f0af8484640 1 -- 172.21.15.60:0/834540154 wait complete. 2024-02-26T04:57:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af8484640 1 Processor -- start 2024-02-26T04:57:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af8484640 1 -- start start 2024-02-26T04:57:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af8484640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af0101190 0x7f0af0199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af8484640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af0102380 0x7f0af0199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af8484640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0102d40 0x7f0af01a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af8484640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af01a0f20 con 0x7f0af0102d40 2024-02-26T04:57:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af8484640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af01a1090 con 0x7f0af0101190 2024-02-26T04:57:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af8484640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af01a1250 con 0x7f0af0102380 2024-02-26T04:57:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af6a21640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0102d40 0x7f0af01a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af6a21640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0102d40 0x7f0af01a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44140/0 (socket says 172.21.15.60:44140) 2024-02-26T04:57:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af6a21640 1 -- 172.21.15.60:0/889553695 learned_addr learned my addr 172.21.15.60:0/889553695 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af6220640 1 --2- 172.21.15.60:0/889553695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af0101190 0x7f0af0199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af5a1f640 1 --2- 172.21.15.60:0/889553695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af0102380 0x7f0af0199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af6a21640 1 -- 172.21.15.60:0/889553695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af0102380 msgr2=0x7f0af0199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af6a21640 1 --2- 172.21.15.60:0/889553695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af0102380 0x7f0af0199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af6a21640 1 -- 172.21.15.60:0/889553695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af0101190 msgr2=0x7f0af0199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af6a21640 1 --2- 172.21.15.60:0/889553695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af0101190 0x7f0af0199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.250+0000 7f0af6a21640 1 -- 172.21.15.60:0/889553695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0009560 con 0x7f0af0102d40 2024-02-26T04:57:55.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.251+0000 7f0af6a21640 1 --2- 172.21.15.60:0/889553695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0102d40 0x7f0af01a09e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800eb10 tx=0x7f0ae800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:55.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.251+0000 7f0ae77fe640 1 -- 172.21.15.60:0/889553695 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8004b20 con 0x7f0af0102d40 2024-02-26T04:57:55.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.251+0000 7f0ae77fe640 1 -- 172.21.15.60:0/889553695 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ae8004c80 con 0x7f0af0102d40 2024-02-26T04:57:55.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.251+0000 7f0af8484640 1 -- 172.21.15.60:0/889553695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af01a1530 con 0x7f0af0102d40 2024-02-26T04:57:55.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.251+0000 7f0ae77fe640 1 -- 172.21.15.60:0/889553695 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8010670 con 0x7f0af0102d40 2024-02-26T04:57:55.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.251+0000 7f0af8484640 1 -- 172.21.15.60:0/889553695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af01a1a40 con 0x7f0af0102d40 2024-02-26T04:57:55.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.252+0000 7f0af8484640 1 -- 172.21.15.60:0/889553695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af010cba0 con 0x7f0af0102d40 2024-02-26T04:57:55.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.253+0000 7f0ae77fe640 1 -- 172.21.15.60:0/889553695 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0ae8004df0 con 0x7f0af0102d40 2024-02-26T04:57:55.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.253+0000 7f0ae77fe640 1 --2- 172.21.15.60:0/889553695 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0adc0a20c0 0x7f0adc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:55.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.253+0000 7f0ae77fe640 1 -- 172.21.15.60:0/889553695 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f0ae8014070 con 0x7f0af0102d40 2024-02-26T04:57:55.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.253+0000 7f0af6220640 1 --2- 172.21.15.60:0/889553695 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0adc0a20c0 0x7f0adc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:55.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.254+0000 7f0af6220640 1 --2- 172.21.15.60:0/889553695 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0adc0a20c0 0x7f0adc0a4580 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0af01021e0 tx=0x7f0aec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:55.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.257+0000 7f0ae77fe640 1 -- 172.21.15.60:0/889553695 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ae8097460 con 0x7f0af0102d40 2024-02-26T04:57:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.439+0000 7f0af8484640 1 -- 172.21.15.60:0/889553695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0af01a1cc0 con 0x7f0af0102d40 2024-02-26T04:57:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.439+0000 7f0ae77fe640 1 -- 172.21.15.60:0/889553695 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0af01a1cc0 con 0x7f0af0102d40 2024-02-26T04:57:55.442 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:57:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.441+0000 7f0af8484640 1 -- 172.21.15.60:0/889553695 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0adc0a20c0 msgr2=0x7f0adc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.441+0000 7f0af8484640 1 --2- 172.21.15.60:0/889553695 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0adc0a20c0 0x7f0adc0a4580 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0af01021e0 tx=0x7f0aec005f90 comp rx=0 tx=0).stop 2024-02-26T04:57:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.441+0000 7f0af8484640 1 -- 172.21.15.60:0/889553695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0102d40 msgr2=0x7f0af01a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.441+0000 7f0af8484640 1 --2- 172.21.15.60:0/889553695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0102d40 0x7f0af01a09e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800eb10 tx=0x7f0ae800c2d0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.441+0000 7f0af8484640 1 -- 172.21.15.60:0/889553695 shutdown_connections 2024-02-26T04:57:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.441+0000 7f0af8484640 1 --2- 172.21.15.60:0/889553695 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0adc0a20c0 0x7f0adc0a4580 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.441+0000 7f0af8484640 1 --2- 172.21.15.60:0/889553695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0102d40 0x7f0af01a09e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.441+0000 7f0af8484640 1 --2- 172.21.15.60:0/889553695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0af0102380 0x7f0af0199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.441+0000 7f0af8484640 1 --2- 172.21.15.60:0/889553695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0af0101190 0x7f0af0199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.441+0000 7f0af8484640 1 -- 172.21.15.60:0/889553695 >> 172.21.15.60:0/889553695 conn(0x7f0af00fc720 msgr2=0x7f0af00733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.441+0000 7f0af8484640 1 -- 172.21.15.60:0/889553695 shutdown_connections 2024-02-26T04:57:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.441+0000 7f0af8484640 1 -- 172.21.15.60:0/889553695 wait complete. 2024-02-26T04:57:55.454 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:55 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1954917218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:55.454 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:55 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1314751997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:55.454 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:55 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1088831720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:55.455 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-02-26T04:57:55.455 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:57:55.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.555+0000 7fa0d54de640 1 -- 172.21.15.60:0/3006340501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d01045b0 msgr2=0x7fa0d01069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.555+0000 7fa0d54de640 1 --2- 172.21.15.60:0/3006340501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d01045b0 0x7fa0d01069a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009900 tx=0x7fa0c402bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:55.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.555+0000 7fa0d54de640 1 -- 172.21.15.60:0/3006340501 shutdown_connections 2024-02-26T04:57:55.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.555+0000 7fa0d54de640 1 --2- 172.21.15.60:0/3006340501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0d0106ee0 0x7fa0d01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.555+0000 7fa0d54de640 1 --2- 172.21.15.60:0/3006340501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d01045b0 0x7fa0d01069a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.555+0000 7fa0d54de640 1 --2- 172.21.15.60:0/3006340501 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0d0101c80 0x7fa0d0104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.555+0000 7fa0d54de640 1 -- 172.21.15.60:0/3006340501 >> 172.21.15.60:0/3006340501 conn(0x7fa0d00fb610 msgr2=0x7fa0d00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:55.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.555+0000 7fa0d54de640 1 -- 172.21.15.60:0/3006340501 shutdown_connections 2024-02-26T04:57:55.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.555+0000 7fa0d54de640 1 -- 172.21.15.60:0/3006340501 wait complete. 2024-02-26T04:57:55.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.555+0000 7fa0d54de640 1 Processor -- start 2024-02-26T04:57:55.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0d54de640 1 -- start start 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0d54de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0d0101c80 0x7fa0d01015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0d54de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d01045b0 0x7fa0d00ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0d54de640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0d0106ee0 0x7fa0d0100170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0d54de640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d01008d0 con 0x7fa0d01045b0 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0d54de640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d0100a40 con 0x7fa0d0106ee0 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0d54de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d01a1310 con 0x7fa0d0101c80 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0ce7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d01045b0 0x7fa0d00ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0ce7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d01045b0 0x7fa0d00ffc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44162/0 (socket says 172.21.15.60:44162) 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0ce7fc640 1 -- 172.21.15.60:0/4002182226 learned_addr learned my addr 172.21.15.60:0/4002182226 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0ceffd640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0d0101c80 0x7fa0d01015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0cf7fe640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0d0106ee0 0x7fa0d0100170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0ce7fc640 1 -- 172.21.15.60:0/4002182226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0d0101c80 msgr2=0x7fa0d01015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0ce7fc640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0d0101c80 0x7fa0d01015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0ce7fc640 1 -- 172.21.15.60:0/4002182226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0d0106ee0 msgr2=0x7fa0d0100170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0ce7fc640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0d0106ee0 0x7fa0d0100170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0ce7fc640 1 -- 172.21.15.60:0/4002182226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4009560 con 0x7fa0d01045b0 2024-02-26T04:57:55.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0ceffd640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0d0101c80 0x7fa0d01015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:55.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0cf7fe640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0d0106ee0 0x7fa0d0100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:55.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.556+0000 7fa0ce7fc640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d01045b0 0x7fa0d00ffc30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4036040 tx=0x7fa0c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:55.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.557+0000 7fa0b7fff640 1 -- 172.21.15.60:0/4002182226 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4004df0 con 0x7fa0d01045b0 2024-02-26T04:57:55.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.557+0000 7fa0b7fff640 1 -- 172.21.15.60:0/4002182226 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0c4005430 con 0x7fa0d01045b0 2024-02-26T04:57:55.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.557+0000 7fa0d54de640 1 -- 172.21.15.60:0/4002182226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d01a1560 con 0x7fa0d01045b0 2024-02-26T04:57:55.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.557+0000 7fa0b7fff640 1 -- 172.21.15.60:0/4002182226 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c403e450 con 0x7fa0d01045b0 2024-02-26T04:57:55.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.557+0000 7fa0d54de640 1 -- 172.21.15.60:0/4002182226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d01a1a70 con 0x7fa0d01045b0 2024-02-26T04:57:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.558+0000 7fa0d54de640 1 -- 172.21.15.60:0/4002182226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d00fd100 con 0x7fa0d01045b0 2024-02-26T04:57:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.559+0000 7fa0b7fff640 1 -- 172.21.15.60:0/4002182226 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fa0c4003680 con 0x7fa0d01045b0 2024-02-26T04:57:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.559+0000 7fa0b7fff640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa0b80a20c0 0x7fa0b80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.559+0000 7fa0b7fff640 1 -- 172.21.15.60:0/4002182226 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fa0c40f06c0 con 0x7fa0d01045b0 2024-02-26T04:57:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.559+0000 7fa0ceffd640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa0b80a20c0 0x7fa0b80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.560+0000 7fa0ceffd640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa0b80a20c0 0x7fa0b80a4580 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00058c0 tx=0x7fa0c0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:55.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.564+0000 7fa0b7fff640 1 -- 172.21.15.60:0/4002182226 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa0c40b9250 con 0x7fa0d01045b0 2024-02-26T04:57:55.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:55 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1954917218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:55.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:55 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1314751997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:55.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:55 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1088831720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:55.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:55 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1954917218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:55.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:55 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1314751997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:55.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:55 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1088831720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:55.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.781+0000 7fa0d54de640 1 -- 172.21.15.60:0/4002182226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0d002cdd0 con 0x7fa0d01045b0 2024-02-26T04:57:55.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.781+0000 7fa0b7fff640 1 -- 172.21.15.60:0/4002182226 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11347 (secure 0 0 0) 0x7fa0c40bc910 con 0x7fa0d01045b0 2024-02-26T04:57:55.784 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:55.784 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:57:52.162616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24476,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T04:57:55.785 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-02-26T04:57:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.783+0000 7fa0d54de640 1 -- 172.21.15.60:0/4002182226 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa0b80a20c0 msgr2=0x7fa0b80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.783+0000 7fa0d54de640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa0b80a20c0 0x7fa0b80a4580 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00058c0 tx=0x7fa0c0005850 comp rx=0 tx=0).stop 2024-02-26T04:57:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.783+0000 7fa0d54de640 1 -- 172.21.15.60:0/4002182226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d01045b0 msgr2=0x7fa0d00ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.783+0000 7fa0d54de640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d01045b0 0x7fa0d00ffc30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4036040 tx=0x7fa0c4004270 comp rx=0 tx=0).stop 2024-02-26T04:57:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.783+0000 7fa0d54de640 1 -- 172.21.15.60:0/4002182226 shutdown_connections 2024-02-26T04:57:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.783+0000 7fa0d54de640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa0b80a20c0 0x7fa0b80a4580 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.783+0000 7fa0d54de640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0d0106ee0 0x7fa0d0100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.783+0000 7fa0d54de640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d01045b0 0x7fa0d00ffc30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.783+0000 7fa0d54de640 1 --2- 172.21.15.60:0/4002182226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0d0101c80 0x7fa0d01015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.783+0000 7fa0d54de640 1 -- 172.21.15.60:0/4002182226 >> 172.21.15.60:0/4002182226 conn(0x7fa0d00fb610 msgr2=0x7fa0d01025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:55.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.783+0000 7fa0d54de640 1 -- 172.21.15.60:0/4002182226 shutdown_connections 2024-02-26T04:57:55.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.783+0000 7fa0d54de640 1 -- 172.21.15.60:0/4002182226 wait complete. 2024-02-26T04:57:55.797 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-02-26T04:57:55.797 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-02-26T04:57:55.797 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T04:57:55.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.897+0000 7f2a7f78c640 1 -- 172.21.15.60:0/2511368585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a78101290 msgr2=0x7f2a78101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.897+0000 7f2a7f78c640 1 --2- 172.21.15.60:0/2511368585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a78101290 0x7f2a78101710 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009900 tx=0x7f2a6802bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:55.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.897+0000 7f2a7f78c640 1 -- 172.21.15.60:0/2511368585 shutdown_connections 2024-02-26T04:57:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.897+0000 7f2a7f78c640 1 --2- 172.21.15.60:0/2511368585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a78101c50 0x7f2a781084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.897+0000 7f2a7f78c640 1 --2- 172.21.15.60:0/2511368585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a78101290 0x7f2a78101710 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.897+0000 7f2a7f78c640 1 --2- 172.21.15.60:0/2511368585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a781000a0 0x7f2a781004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.897+0000 7f2a7f78c640 1 -- 172.21.15.60:0/2511368585 >> 172.21.15.60:0/2511368585 conn(0x7f2a780fb610 msgr2=0x7f2a780fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.898+0000 7f2a7f78c640 1 -- 172.21.15.60:0/2511368585 shutdown_connections 2024-02-26T04:57:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.898+0000 7f2a7f78c640 1 -- 172.21.15.60:0/2511368585 wait complete. 2024-02-26T04:57:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.898+0000 7f2a7f78c640 1 Processor -- start 2024-02-26T04:57:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.898+0000 7f2a7f78c640 1 -- start start 2024-02-26T04:57:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.898+0000 7f2a7f78c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a781000a0 0x7f2a78197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.898+0000 7f2a7f78c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a78101290 0x7f2a78197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.898+0000 7f2a7f78c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a78101c50 0x7f2a7819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.898+0000 7f2a7f78c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7819ed40 con 0x7f2a78101290 2024-02-26T04:57:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.898+0000 7f2a7f78c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7819eeb0 con 0x7f2a781000a0 2024-02-26T04:57:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.898+0000 7f2a7f78c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7819f070 con 0x7f2a78101c50 2024-02-26T04:57:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.898+0000 7f2a7cd27640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a78101290 0x7f2a78197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.898+0000 7f2a7cd27640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a78101290 0x7f2a78197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44180/0 (socket says 172.21.15.60:44180) 2024-02-26T04:57:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.898+0000 7f2a7cd27640 1 -- 172.21.15.60:0/517404366 learned_addr learned my addr 172.21.15.60:0/517404366 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.899+0000 7f2a7d528640 1 --2- 172.21.15.60:0/517404366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a781000a0 0x7f2a78197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.899+0000 7f2a7dd29640 1 --2- 172.21.15.60:0/517404366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a78101c50 0x7f2a7819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.899+0000 7f2a7cd27640 1 -- 172.21.15.60:0/517404366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a78101c50 msgr2=0x7f2a7819e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.899+0000 7f2a7cd27640 1 --2- 172.21.15.60:0/517404366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a78101c50 0x7f2a7819e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.899+0000 7f2a7cd27640 1 -- 172.21.15.60:0/517404366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a781000a0 msgr2=0x7f2a78197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.899+0000 7f2a7cd27640 1 --2- 172.21.15.60:0/517404366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a781000a0 0x7f2a78197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.899+0000 7f2a7cd27640 1 -- 172.21.15.60:0/517404366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a68009560 con 0x7f2a78101290 2024-02-26T04:57:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.899+0000 7f2a7cd27640 1 --2- 172.21.15.60:0/517404366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a78101290 0x7f2a78197790 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2a68005ab0 tx=0x7f2a680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.899+0000 7f2a6e7fc640 1 -- 172.21.15.60:0/517404366 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a680044e0 con 0x7f2a78101290 2024-02-26T04:57:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.899+0000 7f2a7f78c640 1 -- 172.21.15.60:0/517404366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7819f2f0 con 0x7f2a78101290 2024-02-26T04:57:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.899+0000 7f2a6e7fc640 1 -- 172.21.15.60:0/517404366 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a680054e0 con 0x7f2a78101290 2024-02-26T04:57:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.899+0000 7f2a6e7fc640 1 -- 172.21.15.60:0/517404366 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6803e450 con 0x7f2a78101290 2024-02-26T04:57:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.899+0000 7f2a7f78c640 1 -- 172.21.15.60:0/517404366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7819f8d0 con 0x7f2a78101290 2024-02-26T04:57:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.900+0000 7f2a7f78c640 1 -- 172.21.15.60:0/517404366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7810baf0 con 0x7f2a78101290 2024-02-26T04:57:55.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.901+0000 7f2a6e7fc640 1 -- 172.21.15.60:0/517404366 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2a680035f0 con 0x7f2a78101290 2024-02-26T04:57:55.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.902+0000 7f2a6e7fc640 1 --2- 172.21.15.60:0/517404366 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2a640a20c0 0x7f2a640a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:55.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.902+0000 7f2a7d528640 1 --2- 172.21.15.60:0/517404366 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2a640a20c0 0x7f2a640a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:55.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.902+0000 7f2a6e7fc640 1 -- 172.21.15.60:0/517404366 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f2a680f06d0 con 0x7f2a78101290 2024-02-26T04:57:55.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.902+0000 7f2a7d528640 1 --2- 172.21.15.60:0/517404366 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2a640a20c0 0x7f2a640a4580 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2a740058c0 tx=0x7f2a74005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:55.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:55.906+0000 7f2a6e7fc640 1 -- 172.21.15.60:0/517404366 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a680b9260 con 0x7f2a78101290 2024-02-26T04:57:56.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.082+0000 7f2a7f78c640 1 -- 172.21.15.60:0/517404366 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a780fd750 con 0x7f2a640a20c0 2024-02-26T04:57:56.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.095+0000 7f2a6e7fc640 1 -- 172.21.15.60:0/517404366 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428039 (secure 0 0 0) 0x7f2a780fd750 con 0x7f2a640a20c0 2024-02-26T04:57:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:56.106 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-02-26T04:57:56.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.099+0000 7f2a7f78c640 1 -- 172.21.15.60:0/517404366 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2a640a20c0 msgr2=0x7f2a640a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.099+0000 7f2a7f78c640 1 --2- 172.21.15.60:0/517404366 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2a640a20c0 0x7f2a640a4580 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2a740058c0 tx=0x7f2a74005850 comp rx=0 tx=0).stop 2024-02-26T04:57:56.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.099+0000 7f2a7f78c640 1 -- 172.21.15.60:0/517404366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a78101290 msgr2=0x7f2a78197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.099+0000 7f2a7f78c640 1 --2- 172.21.15.60:0/517404366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a78101290 0x7f2a78197790 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2a68005ab0 tx=0x7f2a680042f0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.099+0000 7f2a7f78c640 1 -- 172.21.15.60:0/517404366 shutdown_connections 2024-02-26T04:57:56.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.099+0000 7f2a7f78c640 1 --2- 172.21.15.60:0/517404366 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f2a640a20c0 0x7f2a640a4580 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.099+0000 7f2a7f78c640 1 --2- 172.21.15.60:0/517404366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a78101c50 0x7f2a7819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.099+0000 7f2a7f78c640 1 --2- 172.21.15.60:0/517404366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a78101290 0x7f2a78197790 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.099+0000 7f2a7f78c640 1 --2- 172.21.15.60:0/517404366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a781000a0 0x7f2a78197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.099+0000 7f2a7f78c640 1 -- 172.21.15.60:0/517404366 >> 172.21.15.60:0/517404366 conn(0x7f2a780fb610 msgr2=0x7f2a780fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:56.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.099+0000 7f2a7f78c640 1 -- 172.21.15.60:0/517404366 shutdown_connections 2024-02-26T04:57:56.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.099+0000 7f2a7f78c640 1 -- 172.21.15.60:0/517404366 wait complete. 2024-02-26T04:57:56.112 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-02-26T04:57:54.345010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657817,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":854,"num_read_kb":545,"num_write":349,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181318,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":783,"ondisk_log_size":783,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386636,"kb_used_data":12440,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555188,"statfs":{"total":1151940427776,"available":1151544512512,"internally_reserved":0,"allocated":12738560,"data_stored":8191309,"data_compressed":430314,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003196"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184951+0000","last_change":"2024-02-26T04:55:28.208832+0000","last_active":"2024-02-26T04:57:21.184951+0000","last_peered":"2024-02-26T04:57:21.184951+0000","last_clean":"2024-02-26T04:57:21.184951+0000","last_became_active":"2024-02-26T04:55:28.208454+0000","last_became_peered":"2024-02-26T04:55:28.208454+0000","last_unstale":"2024-02-26T04:57:21.184951+0000","last_undegraded":"2024-02-26T04:57:21.184951+0000","last_fullsized":"2024-02-26T04:57:21.184951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:55:54.132384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391600+0000","last_change":"2024-02-26T04:55:28.203876+0000","last_active":"2024-02-26T04:57:21.391600+0000","last_peered":"2024-02-26T04:57:21.391600+0000","last_clean":"2024-02-26T04:57:21.391600+0000","last_became_active":"2024-02-26T04:55:28.203399+0000","last_became_peered":"2024-02-26T04:55:28.203399+0000","last_unstale":"2024-02-26T04:57:21.391600+0000","last_undegraded":"2024-02-26T04:57:21.391600+0000","last_fullsized":"2024-02-26T04:57:21.391600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:31:40.927929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391340+0000","last_change":"2024-02-26T04:55:28.211388+0000","last_active":"2024-02-26T04:57:21.391340+0000","last_peered":"2024-02-26T04:57:21.391340+0000","last_clean":"2024-02-26T04:57:21.391340+0000","last_became_active":"2024-02-26T04:55:28.209007+0000","last_became_peered":"2024-02-26T04:55:28.209007+0000","last_unstale":"2024-02-26T04:57:21.391340+0000","last_undegraded":"2024-02-26T04:57:21.391340+0000","last_fullsized":"2024-02-26T04:57:21.391340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:37:01.197589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177443+0000","last_change":"2024-02-26T04:55:28.216378+0000","last_active":"2024-02-26T04:57:21.177443+0000","last_peered":"2024-02-26T04:57:21.177443+0000","last_clean":"2024-02-26T04:57:21.177443+0000","last_became_active":"2024-02-26T04:55:28.216074+0000","last_became_peered":"2024-02-26T04:55:28.216074+0000","last_unstale":"2024-02-26T04:57:21.177443+0000","last_undegraded":"2024-02-26T04:57:21.177443+0000","last_fullsized":"2024-02-26T04:57:21.177443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:15:20.460699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180626+0000","last_change":"2024-02-26T04:55:28.213026+0000","last_active":"2024-02-26T04:57:21.180626+0000","last_peered":"2024-02-26T04:57:21.180626+0000","last_clean":"2024-02-26T04:57:21.180626+0000","last_became_active":"2024-02-26T04:55:28.212706+0000","last_became_peered":"2024-02-26T04:55:28.212706+0000","last_unstale":"2024-02-26T04:57:21.180626+0000","last_undegraded":"2024-02-26T04:57:21.180626+0000","last_fullsized":"2024-02-26T04:57:21.180626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:57:35.961956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.245681+0000","last_change":"2024-02-26T04:55:28.202123+0000","last_active":"2024-02-26T04:57:21.245681+0000","last_peered":"2024-02-26T04:57:21.245681+0000","last_clean":"2024-02-26T04:57:21.245681+0000","last_became_active":"2024-02-26T04:55:28.201733+0000","last_became_peered":"2024-02-26T04:55:28.201733+0000","last_unstale":"2024-02-26T04:57:21.245681+0000","last_undegraded":"2024-02-26T04:57:21.245681+0000","last_fullsized":"2024-02-26T04:57:21.245681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:43:11.300006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180147+0000","last_change":"2024-02-26T04:56:20.508346+0000","last_active":"2024-02-26T04:57:21.180147+0000","last_peered":"2024-02-26T04:57:21.180147+0000","last_clean":"2024-02-26T04:57:21.180147+0000","last_became_active":"2024-02-26T04:56:20.507941+0000","last_became_peered":"2024-02-26T04:56:20.507941+0000","last_unstale":"2024-02-26T04:57:21.180147+0000","last_undegraded":"2024-02-26T04:57:21.180147+0000","last_fullsized":"2024-02-26T04:57:21.180147+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:35:27.425477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.185909+0000","last_change":"2024-02-26T04:55:28.214631+0000","last_active":"2024-02-26T04:57:21.185909+0000","last_peered":"2024-02-26T04:57:21.185909+0000","last_clean":"2024-02-26T04:57:21.185909+0000","last_became_active":"2024-02-26T04:55:28.214290+0000","last_became_peered":"2024-02-26T04:55:28.214290+0000","last_unstale":"2024-02-26T04:57:21.185909+0000","last_undegraded":"2024-02-26T04:57:21.185909+0000","last_fullsized":"2024-02-26T04:57:21.185909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:30:11.494833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179697+0000","last_change":"2024-02-26T04:56:20.508783+0000","last_active":"2024-02-26T04:57:21.179697+0000","last_peered":"2024-02-26T04:57:21.179697+0000","last_clean":"2024-02-26T04:57:21.179697+0000","last_became_active":"2024-02-26T04:56:20.508372+0000","last_became_peered":"2024-02-26T04:56:20.508372+0000","last_unstale":"2024-02-26T04:57:21.179697+0000","last_undegraded":"2024-02-26T04:57:21.179697+0000","last_fullsized":"2024-02-26T04:57:21.179697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:32.415022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.246135+0000","last_change":"2024-02-26T04:55:28.204176+0000","last_active":"2024-02-26T04:57:21.246135+0000","last_peered":"2024-02-26T04:57:21.246135+0000","last_clean":"2024-02-26T04:57:21.246135+0000","last_became_active":"2024-02-26T04:55:28.203821+0000","last_became_peered":"2024-02-26T04:55:28.203821+0000","last_unstale":"2024-02-26T04:57:21.246135+0000","last_undegraded":"2024-02-26T04:57:21.246135+0000","last_fullsized":"2024-02-26T04:57:21.246135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:24:03.132650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180742+0000","last_change":"2024-02-26T04:55:28.215958+0000","last_active":"2024-02-26T04:57:21.180742+0000","last_peered":"2024-02-26T04:57:21.180742+0000","last_clean":"2024-02-26T04:57:21.180742+0000","last_became_active":"2024-02-26T04:55:28.215594+0000","last_became_peered":"2024-02-26T04:55:28.215594+0000","last_unstale":"2024-02-26T04:57:21.180742+0000","last_undegraded":"2024-02-26T04:57:21.180742+0000","last_fullsized":"2024-02-26T04:57:21.180742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:45:47.915379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177362+0000","last_change":"2024-02-26T04:55:28.218392+0000","last_active":"2024-02-26T04:57:21.177362+0000","last_peered":"2024-02-26T04:57:21.177362+0000","last_clean":"2024-02-26T04:57:21.177362+0000","last_became_active":"2024-02-26T04:55:28.217933+0000","last_became_peered":"2024-02-26T04:55:28.217933+0000","last_unstale":"2024-02-26T04:57:21.177362+0000","last_undegraded":"2024-02-26T04:57:21.177362+0000","last_fullsized":"2024-02-26T04:57:21.177362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:30:38.880649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176503+0000","last_change":"2024-02-26T04:55:28.197099+0000","last_active":"2024-02-26T04:57:21.176503+0000","last_peered":"2024-02-26T04:57:21.176503+0000","last_clean":"2024-02-26T04:57:21.176503+0000","last_became_active":"2024-02-26T04:55:28.196755+0000","last_became_peered":"2024-02-26T04:55:28.196755+0000","last_unstale":"2024-02-26T04:57:21.176503+0000","last_undegraded":"2024-02-26T04:57:21.176503+0000","last_fullsized":"2024-02-26T04:57:21.176503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:37:31.685845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391224+0000","last_change":"2024-02-26T04:55:28.212064+0000","last_active":"2024-02-26T04:57:21.391224+0000","last_peered":"2024-02-26T04:57:21.391224+0000","last_clean":"2024-02-26T04:57:21.391224+0000","last_became_active":"2024-02-26T04:55:28.208742+0000","last_became_peered":"2024-02-26T04:55:28.208742+0000","last_unstale":"2024-02-26T04:57:21.391224+0000","last_undegraded":"2024-02-26T04:57:21.391224+0000","last_fullsized":"2024-02-26T04:57:21.391224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:38:19.742468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391162+0000","last_change":"2024-02-26T04:56:20.511014+0000","last_active":"2024-02-26T04:57:21.391162+0000","last_peered":"2024-02-26T04:57:21.391162+0000","last_clean":"2024-02-26T04:57:21.391162+0000","last_became_active":"2024-02-26T04:56:20.510463+0000","last_became_peered":"2024-02-26T04:56:20.510463+0000","last_unstale":"2024-02-26T04:57:21.391162+0000","last_undegraded":"2024-02-26T04:57:21.391162+0000","last_fullsized":"2024-02-26T04:57:21.391162+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:44:59.574588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363448+0000","last_change":"2024-02-26T04:55:28.222810+0000","last_active":"2024-02-26T04:57:21.363448+0000","last_peered":"2024-02-26T04:57:21.363448+0000","last_clean":"2024-02-26T04:57:21.363448+0000","last_became_active":"2024-02-26T04:55:28.222341+0000","last_became_peered":"2024-02-26T04:55:28.222341+0000","last_unstale":"2024-02-26T04:57:21.363448+0000","last_undegraded":"2024-02-26T04:57:21.363448+0000","last_fullsized":"2024-02-26T04:57:21.363448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:59:21.627517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184837+0000","last_change":"2024-02-26T04:55:28.208377+0000","last_active":"2024-02-26T04:57:21.184837+0000","last_peered":"2024-02-26T04:57:21.184837+0000","last_clean":"2024-02-26T04:57:21.184837+0000","last_became_active":"2024-02-26T04:55:28.207932+0000","last_became_peered":"2024-02-26T04:55:28.207932+0000","last_unstale":"2024-02-26T04:57:21.184837+0000","last_undegraded":"2024-02-26T04:57:21.184837+0000","last_fullsized":"2024-02-26T04:57:21.184837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:15:39.586191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179812+0000","last_change":"2024-02-26T04:55:28.201105+0000","last_active":"2024-02-26T04:57:21.179812+0000","last_peered":"2024-02-26T04:57:21.179812+0000","last_clean":"2024-02-26T04:57:21.179812+0000","last_became_active":"2024-02-26T04:55:28.200796+0000","last_became_peered":"2024-02-26T04:55:28.200796+0000","last_unstale":"2024-02-26T04:57:21.179812+0000","last_undegraded":"2024-02-26T04:57:21.179812+0000","last_fullsized":"2024-02-26T04:57:21.179812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:53:43.713055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181623+0000","last_change":"2024-02-26T04:55:28.215725+0000","last_active":"2024-02-26T04:57:21.181623+0000","last_peered":"2024-02-26T04:57:21.181623+0000","last_clean":"2024-02-26T04:57:21.181623+0000","last_became_active":"2024-02-26T04:55:28.215341+0000","last_became_peered":"2024-02-26T04:55:28.215341+0000","last_unstale":"2024-02-26T04:57:21.181623+0000","last_undegraded":"2024-02-26T04:57:21.181623+0000","last_fullsized":"2024-02-26T04:57:21.181623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:26:26.960474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180646+0000","last_change":"2024-02-26T04:55:32.233303+0000","last_active":"2024-02-26T04:57:21.180646+0000","last_peered":"2024-02-26T04:57:21.180646+0000","last_clean":"2024-02-26T04:57:21.180646+0000","last_became_active":"2024-02-26T04:55:32.232901+0000","last_became_peered":"2024-02-26T04:55:32.232901+0000","last_unstale":"2024-02-26T04:57:21.180646+0000","last_undegraded":"2024-02-26T04:57:21.180646+0000","last_fullsized":"2024-02-26T04:57:21.180646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:37:49.659668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.194950+0000","last_change":"2024-02-26T04:57:21.194629+0000","last_active":"2024-02-26T04:57:21.194950+0000","last_peered":"2024-02-26T04:57:21.194950+0000","last_clean":"2024-02-26T04:57:21.194950+0000","last_became_active":"2024-02-26T04:57:21.186661+0000","last_became_peered":"2024-02-26T04:57:21.186661+0000","last_unstale":"2024-02-26T04:57:21.194950+0000","last_undegraded":"2024-02-26T04:57:21.194950+0000","last_fullsized":"2024-02-26T04:57:21.194950+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:08:47.904102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179621+0000","last_change":"2024-02-26T04:56:20.511029+0000","last_active":"2024-02-26T04:57:21.179621+0000","last_peered":"2024-02-26T04:57:21.179621+0000","last_clean":"2024-02-26T04:57:21.179621+0000","last_became_active":"2024-02-26T04:56:20.510590+0000","last_became_peered":"2024-02-26T04:56:20.510590+0000","last_unstale":"2024-02-26T04:57:21.179621+0000","last_undegraded":"2024-02-26T04:57:21.179621+0000","last_fullsized":"2024-02-26T04:57:21.179621+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:01:25.641444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176799+0000","last_change":"2024-02-26T04:55:32.233142+0000","last_active":"2024-02-26T04:57:21.176799+0000","last_peered":"2024-02-26T04:57:21.176799+0000","last_clean":"2024-02-26T04:57:21.176799+0000","last_became_active":"2024-02-26T04:55:32.232722+0000","last_became_peered":"2024-02-26T04:55:32.232722+0000","last_unstale":"2024-02-26T04:57:21.176799+0000","last_undegraded":"2024-02-26T04:57:21.176799+0000","last_fullsized":"2024-02-26T04:57:21.176799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:30:20.906104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186620+0000","last_change":"2024-02-26T04:55:27.203420+0000","last_active":"2024-02-26T04:57:21.186620+0000","last_peered":"2024-02-26T04:57:21.186620+0000","last_clean":"2024-02-26T04:57:21.186620+0000","last_became_active":"2024-02-26T04:55:27.202968+0000","last_became_peered":"2024-02-26T04:55:27.202968+0000","last_unstale":"2024-02-26T04:57:21.186620+0000","last_undegraded":"2024-02-26T04:57:21.186620+0000","last_fullsized":"2024-02-26T04:57:21.186620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:44.496528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391314+0000","last_change":"2024-02-26T04:55:28.217422+0000","last_active":"2024-02-26T04:57:21.391314+0000","last_peered":"2024-02-26T04:57:21.391314+0000","last_clean":"2024-02-26T04:57:21.391314+0000","last_became_active":"2024-02-26T04:55:28.217103+0000","last_became_peered":"2024-02-26T04:55:28.217103+0000","last_unstale":"2024-02-26T04:57:21.391314+0000","last_undegraded":"2024-02-26T04:57:21.391314+0000","last_fullsized":"2024-02-26T04:57:21.391314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:08:37.075467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176904+0000","last_change":"2024-02-26T04:55:32.233322+0000","last_active":"2024-02-26T04:57:21.176904+0000","last_peered":"2024-02-26T04:57:21.176904+0000","last_clean":"2024-02-26T04:57:21.176904+0000","last_became_active":"2024-02-26T04:55:32.232679+0000","last_became_peered":"2024-02-26T04:55:32.232679+0000","last_unstale":"2024-02-26T04:57:21.176904+0000","last_undegraded":"2024-02-26T04:57:21.176904+0000","last_fullsized":"2024-02-26T04:57:21.176904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:10.775924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.192327+0000","last_change":"2024-02-26T04:57:21.192327+0000","last_active":"2024-02-26T04:57:21.192327+0000","last_peered":"2024-02-26T04:57:21.192327+0000","last_clean":"2024-02-26T04:57:21.192327+0000","last_became_active":"2024-02-26T04:57:21.186299+0000","last_became_peered":"2024-02-26T04:57:21.186299+0000","last_unstale":"2024-02-26T04:57:21.192327+0000","last_undegraded":"2024-02-26T04:57:21.192327+0000","last_fullsized":"2024-02-26T04:57:21.192327+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:15.566202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363709+0000","last_change":"2024-02-26T04:55:28.210669+0000","last_active":"2024-02-26T04:57:21.363709+0000","last_peered":"2024-02-26T04:57:21.363709+0000","last_clean":"2024-02-26T04:57:21.363709+0000","last_became_active":"2024-02-26T04:55:28.210322+0000","last_became_peered":"2024-02-26T04:55:28.210322+0000","last_unstale":"2024-02-26T04:57:21.363709+0000","last_undegraded":"2024-02-26T04:57:21.363709+0000","last_fullsized":"2024-02-26T04:57:21.363709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:57:29.121762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176799+0000","last_change":"2024-02-26T04:55:32.229077+0000","last_active":"2024-02-26T04:57:21.176799+0000","last_peered":"2024-02-26T04:57:21.176799+0000","last_clean":"2024-02-26T04:57:21.176799+0000","last_became_active":"2024-02-26T04:55:32.228670+0000","last_became_peered":"2024-02-26T04:55:32.228670+0000","last_unstale":"2024-02-26T04:57:21.176799+0000","last_undegraded":"2024-02-26T04:57:21.176799+0000","last_fullsized":"2024-02-26T04:57:21.176799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:35:20.843351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181671+0000","last_change":"2024-02-26T04:55:27.206088+0000","last_active":"2024-02-26T04:57:21.181671+0000","last_peered":"2024-02-26T04:57:21.181671+0000","last_clean":"2024-02-26T04:57:21.181671+0000","last_became_active":"2024-02-26T04:55:27.205696+0000","last_became_peered":"2024-02-26T04:55:27.205696+0000","last_unstale":"2024-02-26T04:57:21.181671+0000","last_undegraded":"2024-02-26T04:57:21.181671+0000","last_fullsized":"2024-02-26T04:57:21.181671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:14:03.552251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178333+0000","last_change":"2024-02-26T04:56:20.508805+0000","last_active":"2024-02-26T04:57:21.178333+0000","last_peered":"2024-02-26T04:57:21.178333+0000","last_clean":"2024-02-26T04:57:21.178333+0000","last_became_active":"2024-02-26T04:56:20.507796+0000","last_became_peered":"2024-02-26T04:56:20.507796+0000","last_unstale":"2024-02-26T04:57:21.178333+0000","last_undegraded":"2024-02-26T04:57:21.178333+0000","last_fullsized":"2024-02-26T04:57:21.178333+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:25.453111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.187012+0000","last_change":"2024-02-26T04:57:21.187012+0000","last_active":"2024-02-26T04:57:21.187012+0000","last_peered":"2024-02-26T04:57:21.187012+0000","last_clean":"2024-02-26T04:57:21.187012+0000","last_became_active":"2024-02-26T04:57:21.186512+0000","last_became_peered":"2024-02-26T04:57:21.186512+0000","last_unstale":"2024-02-26T04:57:21.187012+0000","last_undegraded":"2024-02-26T04:57:21.187012+0000","last_fullsized":"2024-02-26T04:57:21.187012+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:36:14.768552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:49.315378+0000","last_change":"2024-02-26T04:55:27.201261+0000","last_active":"2024-02-26T04:57:49.315378+0000","last_peered":"2024-02-26T04:57:49.315378+0000","last_clean":"2024-02-26T04:57:49.315378+0000","last_became_active":"2024-02-26T04:55:27.200892+0000","last_became_peered":"2024-02-26T04:55:27.200892+0000","last_unstale":"2024-02-26T04:57:49.315378+0000","last_undegraded":"2024-02-26T04:57:49.315378+0000","last_fullsized":"2024-02-26T04:57:49.315378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:34:32.774143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181394+0000","last_change":"2024-02-26T04:55:28.200445+0000","last_active":"2024-02-26T04:57:21.181394+0000","last_peered":"2024-02-26T04:57:21.181394+0000","last_clean":"2024-02-26T04:57:21.181394+0000","last_became_active":"2024-02-26T04:55:28.199959+0000","last_became_peered":"2024-02-26T04:55:28.199959+0000","last_unstale":"2024-02-26T04:57:21.181394+0000","last_undegraded":"2024-02-26T04:57:21.181394+0000","last_fullsized":"2024-02-26T04:57:21.181394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:36:27.397644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181032+0000","last_change":"2024-02-26T04:55:32.232619+0000","last_active":"2024-02-26T04:57:21.181032+0000","last_peered":"2024-02-26T04:57:21.181032+0000","last_clean":"2024-02-26T04:57:21.181032+0000","last_became_active":"2024-02-26T04:55:32.232112+0000","last_became_peered":"2024-02-26T04:55:32.232112+0000","last_unstale":"2024-02-26T04:57:21.181032+0000","last_undegraded":"2024-02-26T04:57:21.181032+0000","last_fullsized":"2024-02-26T04:57:21.181032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:19.741393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.245662+0000","last_change":"2024-02-26T04:55:27.197376+0000","last_active":"2024-02-26T04:57:21.245662+0000","last_peered":"2024-02-26T04:57:21.245662+0000","last_clean":"2024-02-26T04:57:21.245662+0000","last_became_active":"2024-02-26T04:55:27.195258+0000","last_became_peered":"2024-02-26T04:55:27.195258+0000","last_unstale":"2024-02-26T04:57:21.245662+0000","last_undegraded":"2024-02-26T04:57:21.245662+0000","last_fullsized":"2024-02-26T04:57:21.245662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:41:51.284525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178375+0000","last_change":"2024-02-26T04:55:28.214337+0000","last_active":"2024-02-26T04:57:21.178375+0000","last_peered":"2024-02-26T04:57:21.178375+0000","last_clean":"2024-02-26T04:57:21.178375+0000","last_became_active":"2024-02-26T04:55:28.213966+0000","last_became_peered":"2024-02-26T04:55:28.213966+0000","last_unstale":"2024-02-26T04:57:21.178375+0000","last_undegraded":"2024-02-26T04:57:21.178375+0000","last_fullsized":"2024-02-26T04:57:21.178375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:42.974084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184714+0000","last_change":"2024-02-26T04:55:32.235579+0000","last_active":"2024-02-26T04:57:21.184714+0000","last_peered":"2024-02-26T04:57:21.184714+0000","last_clean":"2024-02-26T04:57:21.184714+0000","last_became_active":"2024-02-26T04:55:32.235241+0000","last_became_peered":"2024-02-26T04:55:32.235241+0000","last_unstale":"2024-02-26T04:57:21.184714+0000","last_undegraded":"2024-02-26T04:57:21.184714+0000","last_fullsized":"2024-02-26T04:57:21.184714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:06.319874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176279+0000","last_change":"2024-02-26T04:55:27.197689+0000","last_active":"2024-02-26T04:57:21.176279+0000","last_peered":"2024-02-26T04:57:21.176279+0000","last_clean":"2024-02-26T04:57:21.176279+0000","last_became_active":"2024-02-26T04:55:27.197285+0000","last_became_peered":"2024-02-26T04:55:27.197285+0000","last_unstale":"2024-02-26T04:57:21.176279+0000","last_undegraded":"2024-02-26T04:57:21.176279+0000","last_fullsized":"2024-02-26T04:57:21.176279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:45:36.688902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.185917+0000","last_change":"2024-02-26T04:55:28.201902+0000","last_active":"2024-02-26T04:57:21.185917+0000","last_peered":"2024-02-26T04:57:21.185917+0000","last_clean":"2024-02-26T04:57:21.185917+0000","last_became_active":"2024-02-26T04:55:28.201604+0000","last_became_peered":"2024-02-26T04:55:28.201604+0000","last_unstale":"2024-02-26T04:57:21.185917+0000","last_undegraded":"2024-02-26T04:57:21.185917+0000","last_fullsized":"2024-02-26T04:57:21.185917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:48:57.711555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391379+0000","last_change":"2024-02-26T04:55:32.230153+0000","last_active":"2024-02-26T04:57:21.391379+0000","last_peered":"2024-02-26T04:57:21.391379+0000","last_clean":"2024-02-26T04:57:21.391379+0000","last_became_active":"2024-02-26T04:55:32.229695+0000","last_became_peered":"2024-02-26T04:55:32.229695+0000","last_unstale":"2024-02-26T04:57:21.391379+0000","last_undegraded":"2024-02-26T04:57:21.391379+0000","last_fullsized":"2024-02-26T04:57:21.391379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:15:39.575727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177682+0000","last_change":"2024-02-26T04:55:27.200943+0000","last_active":"2024-02-26T04:57:21.177682+0000","last_peered":"2024-02-26T04:57:21.177682+0000","last_clean":"2024-02-26T04:57:21.177682+0000","last_became_active":"2024-02-26T04:55:27.200565+0000","last_became_peered":"2024-02-26T04:55:27.200565+0000","last_unstale":"2024-02-26T04:57:21.177682+0000","last_undegraded":"2024-02-26T04:57:21.177682+0000","last_fullsized":"2024-02-26T04:57:21.177682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:54:19.670631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179297+0000","last_change":"2024-02-26T04:55:28.212232+0000","last_active":"2024-02-26T04:57:21.179297+0000","last_peered":"2024-02-26T04:57:21.179297+0000","last_clean":"2024-02-26T04:57:21.179297+0000","last_became_active":"2024-02-26T04:55:28.211875+0000","last_became_peered":"2024-02-26T04:55:28.211875+0000","last_unstale":"2024-02-26T04:57:21.179297+0000","last_undegraded":"2024-02-26T04:57:21.179297+0000","last_fullsized":"2024-02-26T04:57:21.179297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:55.379271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186082+0000","last_change":"2024-02-26T04:55:32.234961+0000","last_active":"2024-02-26T04:57:21.186082+0000","last_peered":"2024-02-26T04:57:21.186082+0000","last_clean":"2024-02-26T04:57:21.186082+0000","last_became_active":"2024-02-26T04:55:32.234455+0000","last_became_peered":"2024-02-26T04:55:32.234455+0000","last_unstale":"2024-02-26T04:57:21.186082+0000","last_undegraded":"2024-02-26T04:57:21.186082+0000","last_fullsized":"2024-02-26T04:57:21.186082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:00:44.322341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":150,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:48.313130+0000","last_change":"2024-02-26T04:55:27.199036+0000","last_active":"2024-02-26T04:57:48.313130+0000","last_peered":"2024-02-26T04:57:48.313130+0000","last_clean":"2024-02-26T04:57:48.313130+0000","last_became_active":"2024-02-26T04:55:27.198694+0000","last_became_peered":"2024-02-26T04:55:27.198694+0000","last_unstale":"2024-02-26T04:57:48.313130+0000","last_undegraded":"2024-02-26T04:57:48.313130+0000","last_fullsized":"2024-02-26T04:57:48.313130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:31:41.935123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181847+0000","last_change":"2024-02-26T04:56:20.208105+0000","last_active":"2024-02-26T04:57:21.181847+0000","last_peered":"2024-02-26T04:57:21.181847+0000","last_clean":"2024-02-26T04:57:21.181847+0000","last_became_active":"2024-02-26T04:56:20.207691+0000","last_became_peered":"2024-02-26T04:56:20.207691+0000","last_unstale":"2024-02-26T04:57:21.181847+0000","last_undegraded":"2024-02-26T04:57:21.181847+0000","last_fullsized":"2024-02-26T04:57:21.181847+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:49:49.019089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,9],"acting":[8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179290+0000","last_change":"2024-02-26T04:55:32.228033+0000","last_active":"2024-02-26T04:57:21.179290+0000","last_peered":"2024-02-26T04:57:21.179290+0000","last_clean":"2024-02-26T04:57:21.179290+0000","last_became_active":"2024-02-26T04:55:32.224273+0000","last_became_peered":"2024-02-26T04:55:32.224273+0000","last_unstale":"2024-02-26T04:57:21.179290+0000","last_undegraded":"2024-02-26T04:57:21.179290+0000","last_fullsized":"2024-02-26T04:57:21.179290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:25:13.618421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180910+0000","last_change":"2024-02-26T04:55:27.202833+0000","last_active":"2024-02-26T04:57:21.180910+0000","last_peered":"2024-02-26T04:57:21.180910+0000","last_clean":"2024-02-26T04:57:21.180910+0000","last_became_active":"2024-02-26T04:55:27.202167+0000","last_became_peered":"2024-02-26T04:55:27.202167+0000","last_unstale":"2024-02-26T04:57:21.180910+0000","last_undegraded":"2024-02-26T04:57:21.180910+0000","last_fullsized":"2024-02-26T04:57:21.180910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:55:46.605763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391476+0000","last_change":"2024-02-26T04:56:20.207930+0000","last_active":"2024-02-26T04:57:21.391476+0000","last_peered":"2024-02-26T04:57:21.391476+0000","last_clean":"2024-02-26T04:57:21.391476+0000","last_became_active":"2024-02-26T04:56:20.207586+0000","last_became_peered":"2024-02-26T04:56:20.207586+0000","last_unstale":"2024-02-26T04:57:21.391476+0000","last_undegraded":"2024-02-26T04:57:21.391476+0000","last_fullsized":"2024-02-26T04:57:21.391476+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:37:57.141645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.365354+0000","last_change":"2024-02-26T04:57:21.365354+0000","last_active":"2024-02-26T04:57:21.365354+0000","last_peered":"2024-02-26T04:57:21.365354+0000","last_clean":"2024-02-26T04:57:21.365354+0000","last_became_active":"2024-02-26T04:57:21.364768+0000","last_became_peered":"2024-02-26T04:57:21.364768+0000","last_unstale":"2024-02-26T04:57:21.365354+0000","last_undegraded":"2024-02-26T04:57:21.365354+0000","last_fullsized":"2024-02-26T04:57:21.365354+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:33:05.881044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178363+0000","last_change":"2024-02-26T04:55:27.200389+0000","last_active":"2024-02-26T04:57:21.178363+0000","last_peered":"2024-02-26T04:57:21.178363+0000","last_clean":"2024-02-26T04:57:21.178363+0000","last_became_active":"2024-02-26T04:55:27.200019+0000","last_became_peered":"2024-02-26T04:55:27.200019+0000","last_unstale":"2024-02-26T04:57:21.178363+0000","last_undegraded":"2024-02-26T04:57:21.178363+0000","last_fullsized":"2024-02-26T04:57:21.178363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:59:11.820988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363892+0000","last_change":"2024-02-26T04:55:28.220309+0000","last_active":"2024-02-26T04:57:21.363892+0000","last_peered":"2024-02-26T04:57:21.363892+0000","last_clean":"2024-02-26T04:57:21.363892+0000","last_became_active":"2024-02-26T04:55:28.219987+0000","last_became_peered":"2024-02-26T04:55:28.219987+0000","last_unstale":"2024-02-26T04:57:21.363892+0000","last_undegraded":"2024-02-26T04:57:21.363892+0000","last_fullsized":"2024-02-26T04:57:21.363892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:39:34.713802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179321+0000","last_change":"2024-02-26T04:55:32.236817+0000","last_active":"2024-02-26T04:57:21.179321+0000","last_peered":"2024-02-26T04:57:21.179321+0000","last_clean":"2024-02-26T04:57:21.179321+0000","last_became_active":"2024-02-26T04:55:32.235818+0000","last_became_peered":"2024-02-26T04:55:32.235818+0000","last_unstale":"2024-02-26T04:57:21.179321+0000","last_undegraded":"2024-02-26T04:57:21.179321+0000","last_fullsized":"2024-02-26T04:57:21.179321+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:56:54.121684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":309,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:49.336238+0000","last_change":"2024-02-26T04:55:27.201582+0000","last_active":"2024-02-26T04:57:49.336238+0000","last_peered":"2024-02-26T04:57:49.336238+0000","last_clean":"2024-02-26T04:57:49.336238+0000","last_became_active":"2024-02-26T04:55:27.201011+0000","last_became_peered":"2024-02-26T04:55:27.201011+0000","last_unstale":"2024-02-26T04:57:49.336238+0000","last_undegraded":"2024-02-26T04:57:49.336238+0000","last_fullsized":"2024-02-26T04:57:49.336238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:07:01.888924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":241,"num_read_kb":183,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183993+0000","last_change":"2024-02-26T04:55:28.199313+0000","last_active":"2024-02-26T04:57:21.183993+0000","last_peered":"2024-02-26T04:57:21.183993+0000","last_clean":"2024-02-26T04:57:21.183993+0000","last_became_active":"2024-02-26T04:55:28.198973+0000","last_became_peered":"2024-02-26T04:55:28.198973+0000","last_unstale":"2024-02-26T04:57:21.183993+0000","last_undegraded":"2024-02-26T04:57:21.183993+0000","last_fullsized":"2024-02-26T04:57:21.183993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:33:49.005899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178664+0000","last_change":"2024-02-26T04:55:32.234529+0000","last_active":"2024-02-26T04:57:21.178664+0000","last_peered":"2024-02-26T04:57:21.178664+0000","last_clean":"2024-02-26T04:57:21.178664+0000","last_became_active":"2024-02-26T04:55:32.234105+0000","last_became_peered":"2024-02-26T04:55:32.234105+0000","last_unstale":"2024-02-26T04:57:21.178664+0000","last_undegraded":"2024-02-26T04:57:21.178664+0000","last_fullsized":"2024-02-26T04:57:21.178664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:17:40.909267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180944+0000","last_change":"2024-02-26T04:55:27.209178+0000","last_active":"2024-02-26T04:57:21.180944+0000","last_peered":"2024-02-26T04:57:21.180944+0000","last_clean":"2024-02-26T04:57:21.180944+0000","last_became_active":"2024-02-26T04:55:27.208834+0000","last_became_peered":"2024-02-26T04:55:27.208834+0000","last_unstale":"2024-02-26T04:57:21.180944+0000","last_undegraded":"2024-02-26T04:57:21.180944+0000","last_fullsized":"2024-02-26T04:57:21.180944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:28.187333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176297+0000","last_change":"2024-02-26T04:55:28.215208+0000","last_active":"2024-02-26T04:57:21.176297+0000","last_peered":"2024-02-26T04:57:21.176297+0000","last_clean":"2024-02-26T04:57:21.176297+0000","last_became_active":"2024-02-26T04:55:28.212135+0000","last_became_peered":"2024-02-26T04:55:28.212135+0000","last_unstale":"2024-02-26T04:57:21.176297+0000","last_undegraded":"2024-02-26T04:57:21.176297+0000","last_fullsized":"2024-02-26T04:57:21.176297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:51.269557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.182783+0000","last_change":"2024-02-26T04:55:32.235439+0000","last_active":"2024-02-26T04:57:21.182783+0000","last_peered":"2024-02-26T04:57:21.182783+0000","last_clean":"2024-02-26T04:57:21.182783+0000","last_became_active":"2024-02-26T04:55:32.234987+0000","last_became_peered":"2024-02-26T04:55:32.234987+0000","last_unstale":"2024-02-26T04:57:21.182783+0000","last_undegraded":"2024-02-26T04:57:21.182783+0000","last_fullsized":"2024-02-26T04:57:21.182783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:31:00.637846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177962+0000","last_change":"2024-02-26T04:55:27.205498+0000","last_active":"2024-02-26T04:57:21.177962+0000","last_peered":"2024-02-26T04:57:21.177962+0000","last_clean":"2024-02-26T04:57:21.177962+0000","last_became_active":"2024-02-26T04:55:27.205179+0000","last_became_peered":"2024-02-26T04:55:27.205179+0000","last_unstale":"2024-02-26T04:57:21.177962+0000","last_undegraded":"2024-02-26T04:57:21.177962+0000","last_fullsized":"2024-02-26T04:57:21.177962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:02:55.491896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180574+0000","last_change":"2024-02-26T04:55:28.201110+0000","last_active":"2024-02-26T04:57:21.180574+0000","last_peered":"2024-02-26T04:57:21.180574+0000","last_clean":"2024-02-26T04:57:21.180574+0000","last_became_active":"2024-02-26T04:55:28.200725+0000","last_became_peered":"2024-02-26T04:55:28.200725+0000","last_unstale":"2024-02-26T04:57:21.180574+0000","last_undegraded":"2024-02-26T04:57:21.180574+0000","last_fullsized":"2024-02-26T04:57:21.180574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:50.377925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.392866+0000","last_change":"2024-02-26T04:57:21.392457+0000","last_active":"2024-02-26T04:57:21.392866+0000","last_peered":"2024-02-26T04:57:21.392866+0000","last_clean":"2024-02-26T04:57:21.392866+0000","last_became_active":"2024-02-26T04:57:21.392057+0000","last_became_peered":"2024-02-26T04:57:21.392057+0000","last_unstale":"2024-02-26T04:57:21.392866+0000","last_undegraded":"2024-02-26T04:57:21.392866+0000","last_fullsized":"2024-02-26T04:57:21.392866+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:23:42.695300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":154,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:50.317246+0000","last_change":"2024-02-26T04:55:27.197338+0000","last_active":"2024-02-26T04:57:50.317246+0000","last_peered":"2024-02-26T04:57:50.317246+0000","last_clean":"2024-02-26T04:57:50.317246+0000","last_became_active":"2024-02-26T04:55:27.196898+0000","last_became_peered":"2024-02-26T04:55:27.196898+0000","last_unstale":"2024-02-26T04:57:50.317246+0000","last_undegraded":"2024-02-26T04:57:50.317246+0000","last_fullsized":"2024-02-26T04:57:50.317246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:27:09.635379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180780+0000","last_change":"2024-02-26T04:55:28.200665+0000","last_active":"2024-02-26T04:57:21.180780+0000","last_peered":"2024-02-26T04:57:21.180780+0000","last_clean":"2024-02-26T04:57:21.180780+0000","last_became_active":"2024-02-26T04:55:28.200329+0000","last_became_peered":"2024-02-26T04:55:28.200329+0000","last_unstale":"2024-02-26T04:57:21.180780+0000","last_undegraded":"2024-02-26T04:57:21.180780+0000","last_fullsized":"2024-02-26T04:57:21.180780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:27.090259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176631+0000","last_change":"2024-02-26T04:55:32.225861+0000","last_active":"2024-02-26T04:57:21.176631+0000","last_peered":"2024-02-26T04:57:21.176631+0000","last_clean":"2024-02-26T04:57:21.176631+0000","last_became_active":"2024-02-26T04:55:32.225187+0000","last_became_peered":"2024-02-26T04:55:32.225187+0000","last_unstale":"2024-02-26T04:57:21.176631+0000","last_undegraded":"2024-02-26T04:57:21.176631+0000","last_fullsized":"2024-02-26T04:57:21.176631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:14:09.077383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180544+0000","last_change":"2024-02-26T04:55:27.200321+0000","last_active":"2024-02-26T04:57:21.180544+0000","last_peered":"2024-02-26T04:57:21.180544+0000","last_clean":"2024-02-26T04:57:21.180544+0000","last_became_active":"2024-02-26T04:55:27.199978+0000","last_became_peered":"2024-02-26T04:55:27.199978+0000","last_unstale":"2024-02-26T04:57:21.180544+0000","last_undegraded":"2024-02-26T04:57:21.180544+0000","last_fullsized":"2024-02-26T04:57:21.180544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:08:20.381010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177958+0000","last_change":"2024-02-26T04:55:28.213940+0000","last_active":"2024-02-26T04:57:21.177958+0000","last_peered":"2024-02-26T04:57:21.177958+0000","last_clean":"2024-02-26T04:57:21.177958+0000","last_became_active":"2024-02-26T04:55:28.213590+0000","last_became_peered":"2024-02-26T04:55:28.213590+0000","last_unstale":"2024-02-26T04:57:21.177958+0000","last_undegraded":"2024-02-26T04:57:21.177958+0000","last_fullsized":"2024-02-26T04:57:21.177958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:29:29.151391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177711+0000","last_change":"2024-02-26T04:55:32.232669+0000","last_active":"2024-02-26T04:57:21.177711+0000","last_peered":"2024-02-26T04:57:21.177711+0000","last_clean":"2024-02-26T04:57:21.177711+0000","last_became_active":"2024-02-26T04:55:32.232288+0000","last_became_peered":"2024-02-26T04:55:32.232288+0000","last_unstale":"2024-02-26T04:57:21.177711+0000","last_undegraded":"2024-02-26T04:57:21.177711+0000","last_fullsized":"2024-02-26T04:57:21.177711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:31:42.436516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.182351+0000","last_change":"2024-02-26T04:55:27.205993+0000","last_active":"2024-02-26T04:57:21.182351+0000","last_peered":"2024-02-26T04:57:21.182351+0000","last_clean":"2024-02-26T04:57:21.182351+0000","last_became_active":"2024-02-26T04:55:27.205594+0000","last_became_peered":"2024-02-26T04:55:27.205594+0000","last_unstale":"2024-02-26T04:57:21.182351+0000","last_undegraded":"2024-02-26T04:57:21.182351+0000","last_fullsized":"2024-02-26T04:57:21.182351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:44:19.993925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184074+0000","last_change":"2024-02-26T04:55:28.203429+0000","last_active":"2024-02-26T04:57:21.184074+0000","last_peered":"2024-02-26T04:57:21.184074+0000","last_clean":"2024-02-26T04:57:21.184074+0000","last_became_active":"2024-02-26T04:55:28.203002+0000","last_became_peered":"2024-02-26T04:55:28.203002+0000","last_unstale":"2024-02-26T04:57:21.184074+0000","last_undegraded":"2024-02-26T04:57:21.184074+0000","last_fullsized":"2024-02-26T04:57:21.184074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:08:05.872241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181329+0000","last_change":"2024-02-26T04:55:32.239323+0000","last_active":"2024-02-26T04:57:21.181329+0000","last_peered":"2024-02-26T04:57:21.181329+0000","last_clean":"2024-02-26T04:57:21.181329+0000","last_became_active":"2024-02-26T04:55:32.238843+0000","last_became_peered":"2024-02-26T04:55:32.238843+0000","last_unstale":"2024-02-26T04:57:21.181329+0000","last_undegraded":"2024-02-26T04:57:21.181329+0000","last_fullsized":"2024-02-26T04:57:21.181329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:39:56.645429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179549+0000","last_change":"2024-02-26T04:55:27.203222+0000","last_active":"2024-02-26T04:57:21.179549+0000","last_peered":"2024-02-26T04:57:21.179549+0000","last_clean":"2024-02-26T04:57:21.179549+0000","last_became_active":"2024-02-26T04:55:27.202845+0000","last_became_peered":"2024-02-26T04:55:27.202845+0000","last_unstale":"2024-02-26T04:57:21.179549+0000","last_undegraded":"2024-02-26T04:57:21.179549+0000","last_fullsized":"2024-02-26T04:57:21.179549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:16:07.524260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.246028+0000","last_change":"2024-02-26T04:55:28.209122+0000","last_active":"2024-02-26T04:57:21.246028+0000","last_peered":"2024-02-26T04:57:21.246028+0000","last_clean":"2024-02-26T04:57:21.246028+0000","last_became_active":"2024-02-26T04:55:28.207598+0000","last_became_peered":"2024-02-26T04:55:28.207598+0000","last_unstale":"2024-02-26T04:57:21.246028+0000","last_undegraded":"2024-02-26T04:57:21.246028+0000","last_fullsized":"2024-02-26T04:57:21.246028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:02:10.763515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176657+0000","last_change":"2024-02-26T04:55:32.227333+0000","last_active":"2024-02-26T04:57:21.176657+0000","last_peered":"2024-02-26T04:57:21.176657+0000","last_clean":"2024-02-26T04:57:21.176657+0000","last_became_active":"2024-02-26T04:55:32.226956+0000","last_became_peered":"2024-02-26T04:55:32.226956+0000","last_unstale":"2024-02-26T04:57:21.176657+0000","last_undegraded":"2024-02-26T04:57:21.176657+0000","last_fullsized":"2024-02-26T04:57:21.176657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:51:40.710242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180411+0000","last_change":"2024-02-26T04:55:27.198781+0000","last_active":"2024-02-26T04:57:21.180411+0000","last_peered":"2024-02-26T04:57:21.180411+0000","last_clean":"2024-02-26T04:57:21.180411+0000","last_became_active":"2024-02-26T04:55:27.198288+0000","last_became_peered":"2024-02-26T04:55:27.198288+0000","last_unstale":"2024-02-26T04:57:21.180411+0000","last_undegraded":"2024-02-26T04:57:21.180411+0000","last_fullsized":"2024-02-26T04:57:21.180411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:52:18.665816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.245833+0000","last_change":"2024-02-26T04:55:28.213044+0000","last_active":"2024-02-26T04:57:21.245833+0000","last_peered":"2024-02-26T04:57:21.245833+0000","last_clean":"2024-02-26T04:57:21.245833+0000","last_became_active":"2024-02-26T04:55:28.212712+0000","last_became_peered":"2024-02-26T04:55:28.212712+0000","last_unstale":"2024-02-26T04:57:21.245833+0000","last_undegraded":"2024-02-26T04:57:21.245833+0000","last_fullsized":"2024-02-26T04:57:21.245833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:02:09.155551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176495+0000","last_change":"2024-02-26T04:55:32.231867+0000","last_active":"2024-02-26T04:57:21.176495+0000","last_peered":"2024-02-26T04:57:21.176495+0000","last_clean":"2024-02-26T04:57:21.176495+0000","last_became_active":"2024-02-26T04:55:32.231450+0000","last_became_peered":"2024-02-26T04:55:32.231450+0000","last_unstale":"2024-02-26T04:57:21.176495+0000","last_undegraded":"2024-02-26T04:57:21.176495+0000","last_fullsized":"2024-02-26T04:57:21.176495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:12:14.367427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177833+0000","last_change":"2024-02-26T04:55:28.207746+0000","last_active":"2024-02-26T04:57:21.177833+0000","last_peered":"2024-02-26T04:57:21.177833+0000","last_clean":"2024-02-26T04:57:21.177833+0000","last_became_active":"2024-02-26T04:55:28.207438+0000","last_became_peered":"2024-02-26T04:55:28.207438+0000","last_unstale":"2024-02-26T04:57:21.177833+0000","last_undegraded":"2024-02-26T04:57:21.177833+0000","last_fullsized":"2024-02-26T04:57:21.177833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:09.409635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":156,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:51.454112+0000","last_change":"2024-02-26T04:55:27.192233+0000","last_active":"2024-02-26T04:57:51.454112+0000","last_peered":"2024-02-26T04:57:51.454112+0000","last_clean":"2024-02-26T04:57:51.454112+0000","last_became_active":"2024-02-26T04:55:27.191868+0000","last_became_peered":"2024-02-26T04:55:27.191868+0000","last_unstale":"2024-02-26T04:57:51.454112+0000","last_undegraded":"2024-02-26T04:57:51.454112+0000","last_fullsized":"2024-02-26T04:57:51.454112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:37:58.430504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181068+0000","last_change":"2024-02-26T04:55:32.226577+0000","last_active":"2024-02-26T04:57:21.181068+0000","last_peered":"2024-02-26T04:57:21.181068+0000","last_clean":"2024-02-26T04:57:21.181068+0000","last_became_active":"2024-02-26T04:55:32.226148+0000","last_became_peered":"2024-02-26T04:55:32.226148+0000","last_unstale":"2024-02-26T04:57:21.181068+0000","last_undegraded":"2024-02-26T04:57:21.181068+0000","last_fullsized":"2024-02-26T04:57:21.181068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:34:53.099211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186167+0000","last_change":"2024-02-26T04:55:28.217865+0000","last_active":"2024-02-26T04:57:21.186167+0000","last_peered":"2024-02-26T04:57:21.186167+0000","last_clean":"2024-02-26T04:57:21.186167+0000","last_became_active":"2024-02-26T04:55:28.217075+0000","last_became_peered":"2024-02-26T04:55:28.217075+0000","last_unstale":"2024-02-26T04:57:21.186167+0000","last_undegraded":"2024-02-26T04:57:21.186167+0000","last_fullsized":"2024-02-26T04:57:21.186167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:30:38.720694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177120+0000","last_change":"2024-02-26T04:55:27.204300+0000","last_active":"2024-02-26T04:57:21.177120+0000","last_peered":"2024-02-26T04:57:21.177120+0000","last_clean":"2024-02-26T04:57:21.177120+0000","last_became_active":"2024-02-26T04:55:27.203933+0000","last_became_peered":"2024-02-26T04:55:27.203933+0000","last_unstale":"2024-02-26T04:57:21.177120+0000","last_undegraded":"2024-02-26T04:57:21.177120+0000","last_fullsized":"2024-02-26T04:57:21.177120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:19:36.247209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180188+0000","last_change":"2024-02-26T04:54:13.145088+0000","last_active":"2024-02-26T04:57:21.180188+0000","last_peered":"2024-02-26T04:57:21.180188+0000","last_clean":"2024-02-26T04:57:21.180188+0000","last_became_active":"2024-02-26T04:54:13.144657+0000","last_became_peered":"2024-02-26T04:54:13.144657+0000","last_unstale":"2024-02-26T04:57:21.180188+0000","last_undegraded":"2024-02-26T04:57:21.180188+0000","last_fullsized":"2024-02-26T04:57:21.180188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:50:45.246330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:50:45.246330+0000","last_clean_scrub_stamp":"2024-02-26T04:50:45.246330+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:00:50.842020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178258+0000","last_change":"2024-02-26T04:55:32.240124+0000","last_active":"2024-02-26T04:57:21.178258+0000","last_peered":"2024-02-26T04:57:21.178258+0000","last_clean":"2024-02-26T04:57:21.178258+0000","last_became_active":"2024-02-26T04:55:32.239624+0000","last_became_peered":"2024-02-26T04:55:32.239624+0000","last_unstale":"2024-02-26T04:57:21.178258+0000","last_undegraded":"2024-02-26T04:57:21.178258+0000","last_fullsized":"2024-02-26T04:57:21.178258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:40:33.770917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176056+0000","last_change":"2024-02-26T04:55:28.204631+0000","last_active":"2024-02-26T04:57:21.176056+0000","last_peered":"2024-02-26T04:57:21.176056+0000","last_clean":"2024-02-26T04:57:21.176056+0000","last_became_active":"2024-02-26T04:55:28.204274+0000","last_became_peered":"2024-02-26T04:55:28.204274+0000","last_unstale":"2024-02-26T04:57:21.176056+0000","last_undegraded":"2024-02-26T04:57:21.176056+0000","last_fullsized":"2024-02-26T04:57:21.176056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:18:18.336015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.187620+0000","last_change":"2024-02-26T04:57:21.187202+0000","last_active":"2024-02-26T04:57:21.187620+0000","last_peered":"2024-02-26T04:57:21.187620+0000","last_clean":"2024-02-26T04:57:21.187620+0000","last_became_active":"2024-02-26T04:57:21.183709+0000","last_became_peered":"2024-02-26T04:57:21.183709+0000","last_unstale":"2024-02-26T04:57:21.187620+0000","last_undegraded":"2024-02-26T04:57:21.187620+0000","last_fullsized":"2024-02-26T04:57:21.187620+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:05.733443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183832+0000","last_change":"2024-02-26T04:55:32.228900+0000","last_active":"2024-02-26T04:57:21.183832+0000","last_peered":"2024-02-26T04:57:21.183832+0000","last_clean":"2024-02-26T04:57:21.183832+0000","last_became_active":"2024-02-26T04:55:32.228262+0000","last_became_peered":"2024-02-26T04:55:32.228262+0000","last_unstale":"2024-02-26T04:57:21.183832+0000","last_undegraded":"2024-02-26T04:57:21.183832+0000","last_fullsized":"2024-02-26T04:57:21.183832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:33.069756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183858+0000","last_change":"2024-02-26T04:55:27.205264+0000","last_active":"2024-02-26T04:57:21.183858+0000","last_peered":"2024-02-26T04:57:21.183858+0000","last_clean":"2024-02-26T04:57:21.183858+0000","last_became_active":"2024-02-26T04:55:27.204882+0000","last_became_peered":"2024-02-26T04:55:27.204882+0000","last_unstale":"2024-02-26T04:57:21.183858+0000","last_undegraded":"2024-02-26T04:57:21.183858+0000","last_fullsized":"2024-02-26T04:57:21.183858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:14:19.497725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176445+0000","last_change":"2024-02-26T04:55:28.200264+0000","last_active":"2024-02-26T04:57:21.176445+0000","last_peered":"2024-02-26T04:57:21.176445+0000","last_clean":"2024-02-26T04:57:21.176445+0000","last_became_active":"2024-02-26T04:55:28.199913+0000","last_became_peered":"2024-02-26T04:55:28.199913+0000","last_unstale":"2024-02-26T04:57:21.176445+0000","last_undegraded":"2024-02-26T04:57:21.176445+0000","last_fullsized":"2024-02-26T04:57:21.176445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:54:51.529149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.185839+0000","last_change":"2024-02-26T04:55:32.233362+0000","last_active":"2024-02-26T04:57:21.185839+0000","last_peered":"2024-02-26T04:57:21.185839+0000","last_clean":"2024-02-26T04:57:21.185839+0000","last_became_active":"2024-02-26T04:55:32.232907+0000","last_became_peered":"2024-02-26T04:55:32.232907+0000","last_unstale":"2024-02-26T04:57:21.185839+0000","last_undegraded":"2024-02-26T04:57:21.185839+0000","last_fullsized":"2024-02-26T04:57:21.185839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:49:11.889863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179790+0000","last_change":"2024-02-26T04:55:27.207371+0000","last_active":"2024-02-26T04:57:21.179790+0000","last_peered":"2024-02-26T04:57:21.179790+0000","last_clean":"2024-02-26T04:57:21.179790+0000","last_became_active":"2024-02-26T04:55:27.207047+0000","last_became_peered":"2024-02-26T04:55:27.207047+0000","last_unstale":"2024-02-26T04:57:21.179790+0000","last_undegraded":"2024-02-26T04:57:21.179790+0000","last_fullsized":"2024-02-26T04:57:21.179790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:29.575633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186642+0000","last_change":"2024-02-26T04:55:28.207132+0000","last_active":"2024-02-26T04:57:21.186642+0000","last_peered":"2024-02-26T04:57:21.186642+0000","last_clean":"2024-02-26T04:57:21.186642+0000","last_became_active":"2024-02-26T04:55:28.206376+0000","last_became_peered":"2024-02-26T04:55:28.206376+0000","last_unstale":"2024-02-26T04:57:21.186642+0000","last_undegraded":"2024-02-26T04:57:21.186642+0000","last_fullsized":"2024-02-26T04:57:21.186642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:17:53.802386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180403+0000","last_change":"2024-02-26T04:55:32.226381+0000","last_active":"2024-02-26T04:57:21.180403+0000","last_peered":"2024-02-26T04:57:21.180403+0000","last_clean":"2024-02-26T04:57:21.180403+0000","last_became_active":"2024-02-26T04:55:32.225940+0000","last_became_peered":"2024-02-26T04:55:32.225940+0000","last_unstale":"2024-02-26T04:57:21.180403+0000","last_undegraded":"2024-02-26T04:57:21.180403+0000","last_fullsized":"2024-02-26T04:57:21.180403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:51:20.190514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177325+0000","last_change":"2024-02-26T04:55:27.200246+0000","last_active":"2024-02-26T04:57:21.177325+0000","last_peered":"2024-02-26T04:57:21.177325+0000","last_clean":"2024-02-26T04:57:21.177325+0000","last_became_active":"2024-02-26T04:55:27.199906+0000","last_became_peered":"2024-02-26T04:55:27.199906+0000","last_unstale":"2024-02-26T04:57:21.177325+0000","last_undegraded":"2024-02-26T04:57:21.177325+0000","last_fullsized":"2024-02-26T04:57:21.177325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:29.037230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391466+0000","last_change":"2024-02-26T04:56:20.204222+0000","last_active":"2024-02-26T04:57:21.391466+0000","last_peered":"2024-02-26T04:57:21.391466+0000","last_clean":"2024-02-26T04:57:21.391466+0000","last_became_active":"2024-02-26T04:56:20.203726+0000","last_became_peered":"2024-02-26T04:56:20.203726+0000","last_unstale":"2024-02-26T04:57:21.391466+0000","last_undegraded":"2024-02-26T04:57:21.391466+0000","last_fullsized":"2024-02-26T04:57:21.391466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:37:59.371734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177174+0000","last_change":"2024-02-26T04:55:32.227765+0000","last_active":"2024-02-26T04:57:21.177174+0000","last_peered":"2024-02-26T04:57:21.177174+0000","last_clean":"2024-02-26T04:57:21.177174+0000","last_became_active":"2024-02-26T04:55:32.227346+0000","last_became_peered":"2024-02-26T04:55:32.227346+0000","last_unstale":"2024-02-26T04:57:21.177174+0000","last_undegraded":"2024-02-26T04:57:21.177174+0000","last_fullsized":"2024-02-26T04:57:21.177174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:05:12.634958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363549+0000","last_change":"2024-02-26T04:55:27.199979+0000","last_active":"2024-02-26T04:57:21.363549+0000","last_peered":"2024-02-26T04:57:21.363549+0000","last_clean":"2024-02-26T04:57:21.363549+0000","last_became_active":"2024-02-26T04:55:27.199618+0000","last_became_peered":"2024-02-26T04:55:27.199618+0000","last_unstale":"2024-02-26T04:57:21.363549+0000","last_undegraded":"2024-02-26T04:57:21.363549+0000","last_fullsized":"2024-02-26T04:57:21.363549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:20:59.649473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180646+0000","last_change":"2024-02-26T04:55:28.202253+0000","last_active":"2024-02-26T04:57:21.180646+0000","last_peered":"2024-02-26T04:57:21.180646+0000","last_clean":"2024-02-26T04:57:21.180646+0000","last_became_active":"2024-02-26T04:55:28.199825+0000","last_became_peered":"2024-02-26T04:55:28.199825+0000","last_unstale":"2024-02-26T04:57:21.180646+0000","last_undegraded":"2024-02-26T04:57:21.180646+0000","last_fullsized":"2024-02-26T04:57:21.180646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:21:59.148995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.188691+0000","last_change":"2024-02-26T04:57:21.188067+0000","last_active":"2024-02-26T04:57:21.188691+0000","last_peered":"2024-02-26T04:57:21.188691+0000","last_clean":"2024-02-26T04:57:21.188691+0000","last_became_active":"2024-02-26T04:57:21.187498+0000","last_became_peered":"2024-02-26T04:57:21.187498+0000","last_unstale":"2024-02-26T04:57:21.188691+0000","last_undegraded":"2024-02-26T04:57:21.188691+0000","last_fullsized":"2024-02-26T04:57:21.188691+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:00:12.060520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186148+0000","last_change":"2024-02-26T04:55:27.206529+0000","last_active":"2024-02-26T04:57:21.186148+0000","last_peered":"2024-02-26T04:57:21.186148+0000","last_clean":"2024-02-26T04:57:21.186148+0000","last_became_active":"2024-02-26T04:55:27.206203+0000","last_became_peered":"2024-02-26T04:55:27.206203+0000","last_unstale":"2024-02-26T04:57:21.186148+0000","last_undegraded":"2024-02-26T04:57:21.186148+0000","last_fullsized":"2024-02-26T04:57:21.186148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:24.491832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184260+0000","last_change":"2024-02-26T04:55:28.205915+0000","last_active":"2024-02-26T04:57:21.184260+0000","last_peered":"2024-02-26T04:57:21.184260+0000","last_clean":"2024-02-26T04:57:21.184260+0000","last_became_active":"2024-02-26T04:55:28.205576+0000","last_became_peered":"2024-02-26T04:55:28.205576+0000","last_unstale":"2024-02-26T04:57:21.184260+0000","last_undegraded":"2024-02-26T04:57:21.184260+0000","last_fullsized":"2024-02-26T04:57:21.184260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:18:54.639778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177053+0000","last_change":"2024-02-26T04:55:32.226253+0000","last_active":"2024-02-26T04:57:21.177053+0000","last_peered":"2024-02-26T04:57:21.177053+0000","last_clean":"2024-02-26T04:57:21.177053+0000","last_became_active":"2024-02-26T04:55:32.225815+0000","last_became_peered":"2024-02-26T04:55:32.225815+0000","last_unstale":"2024-02-26T04:57:21.177053+0000","last_undegraded":"2024-02-26T04:57:21.177053+0000","last_fullsized":"2024-02-26T04:57:21.177053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:22.764756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.196831+0000","last_change":"2024-02-26T04:57:21.196391+0000","last_active":"2024-02-26T04:57:21.196831+0000","last_peered":"2024-02-26T04:57:21.196831+0000","last_clean":"2024-02-26T04:57:21.196831+0000","last_became_active":"2024-02-26T04:57:21.187476+0000","last_became_peered":"2024-02-26T04:57:21.187476+0000","last_unstale":"2024-02-26T04:57:21.196831+0000","last_undegraded":"2024-02-26T04:57:21.196831+0000","last_fullsized":"2024-02-26T04:57:21.196831+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:04:51.219911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186401+0000","last_change":"2024-02-26T04:55:28.213065+0000","last_active":"2024-02-26T04:57:21.186401+0000","last_peered":"2024-02-26T04:57:21.186401+0000","last_clean":"2024-02-26T04:57:21.186401+0000","last_became_active":"2024-02-26T04:55:28.211112+0000","last_became_peered":"2024-02-26T04:55:28.211112+0000","last_unstale":"2024-02-26T04:57:21.186401+0000","last_undegraded":"2024-02-26T04:57:21.186401+0000","last_fullsized":"2024-02-26T04:57:21.186401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:41:10.070183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177186+0000","last_change":"2024-02-26T04:55:32.229838+0000","last_active":"2024-02-26T04:57:21.177186+0000","last_peered":"2024-02-26T04:57:21.177186+0000","last_clean":"2024-02-26T04:57:21.177186+0000","last_became_active":"2024-02-26T04:55:32.228696+0000","last_became_peered":"2024-02-26T04:55:32.228696+0000","last_unstale":"2024-02-26T04:57:21.177186+0000","last_undegraded":"2024-02-26T04:57:21.177186+0000","last_fullsized":"2024-02-26T04:57:21.177186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:59:42.359301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180801+0000","last_change":"2024-02-26T04:55:27.205688+0000","last_active":"2024-02-26T04:57:21.180801+0000","last_peered":"2024-02-26T04:57:21.180801+0000","last_clean":"2024-02-26T04:57:21.180801+0000","last_became_active":"2024-02-26T04:55:27.205306+0000","last_became_peered":"2024-02-26T04:55:27.205306+0000","last_unstale":"2024-02-26T04:57:21.180801+0000","last_undegraded":"2024-02-26T04:57:21.180801+0000","last_fullsized":"2024-02-26T04:57:21.180801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:23:50.203565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180575+0000","last_change":"2024-02-26T04:55:28.210073+0000","last_active":"2024-02-26T04:57:21.180575+0000","last_peered":"2024-02-26T04:57:21.180575+0000","last_clean":"2024-02-26T04:57:21.180575+0000","last_became_active":"2024-02-26T04:55:28.208475+0000","last_became_peered":"2024-02-26T04:55:28.208475+0000","last_unstale":"2024-02-26T04:57:21.180575+0000","last_undegraded":"2024-02-26T04:57:21.180575+0000","last_fullsized":"2024-02-26T04:57:21.180575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:06:49.513916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363553+0000","last_change":"2024-02-26T04:55:32.228708+0000","last_active":"2024-02-26T04:57:21.363553+0000","last_peered":"2024-02-26T04:57:21.363553+0000","last_clean":"2024-02-26T04:57:21.363553+0000","last_became_active":"2024-02-26T04:55:32.228304+0000","last_became_peered":"2024-02-26T04:55:32.228304+0000","last_unstale":"2024-02-26T04:57:21.363553+0000","last_undegraded":"2024-02-26T04:57:21.363553+0000","last_fullsized":"2024-02-26T04:57:21.363553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:53:02.390712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181305+0000","last_change":"2024-02-26T04:55:27.199761+0000","last_active":"2024-02-26T04:57:21.181305+0000","last_peered":"2024-02-26T04:57:21.181305+0000","last_clean":"2024-02-26T04:57:21.181305+0000","last_became_active":"2024-02-26T04:55:27.199113+0000","last_became_peered":"2024-02-26T04:55:27.199113+0000","last_unstale":"2024-02-26T04:57:21.181305+0000","last_undegraded":"2024-02-26T04:57:21.181305+0000","last_fullsized":"2024-02-26T04:57:21.181305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:14:10.631886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180844+0000","last_change":"2024-02-26T04:55:28.211484+0000","last_active":"2024-02-26T04:57:21.180844+0000","last_peered":"2024-02-26T04:57:21.180844+0000","last_clean":"2024-02-26T04:57:21.180844+0000","last_became_active":"2024-02-26T04:55:28.211131+0000","last_became_peered":"2024-02-26T04:55:28.211131+0000","last_unstale":"2024-02-26T04:57:21.180844+0000","last_undegraded":"2024-02-26T04:57:21.180844+0000","last_fullsized":"2024-02-26T04:57:21.180844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:54:09.393847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391396+0000","last_change":"2024-02-26T04:55:32.227493+0000","last_active":"2024-02-26T04:57:21.391396+0000","last_peered":"2024-02-26T04:57:21.391396+0000","last_clean":"2024-02-26T04:57:21.391396+0000","last_became_active":"2024-02-26T04:55:32.227106+0000","last_became_peered":"2024-02-26T04:55:32.227106+0000","last_unstale":"2024-02-26T04:57:21.391396+0000","last_undegraded":"2024-02-26T04:57:21.391396+0000","last_fullsized":"2024-02-26T04:57:21.391396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:33.040966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178567+0000","last_change":"2024-02-26T04:55:27.205331+0000","last_active":"2024-02-26T04:57:21.178567+0000","last_peered":"2024-02-26T04:57:21.178567+0000","last_clean":"2024-02-26T04:57:21.178567+0000","last_became_active":"2024-02-26T04:55:27.204998+0000","last_became_peered":"2024-02-26T04:55:27.204998+0000","last_unstale":"2024-02-26T04:57:21.178567+0000","last_undegraded":"2024-02-26T04:57:21.178567+0000","last_fullsized":"2024-02-26T04:57:21.178567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:10:03.671016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180697+0000","last_change":"2024-02-26T04:56:20.508854+0000","last_active":"2024-02-26T04:57:21.180697+0000","last_peered":"2024-02-26T04:57:21.180697+0000","last_clean":"2024-02-26T04:57:21.180697+0000","last_became_active":"2024-02-26T04:56:20.508396+0000","last_became_peered":"2024-02-26T04:56:20.508396+0000","last_unstale":"2024-02-26T04:57:21.180697+0000","last_undegraded":"2024-02-26T04:57:21.180697+0000","last_fullsized":"2024-02-26T04:57:21.180697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:47:33.249035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177528+0000","last_change":"2024-02-26T04:55:32.232222+0000","last_active":"2024-02-26T04:57:21.177528+0000","last_peered":"2024-02-26T04:57:21.177528+0000","last_clean":"2024-02-26T04:57:21.177528+0000","last_became_active":"2024-02-26T04:55:32.231836+0000","last_became_peered":"2024-02-26T04:55:32.231836+0000","last_unstale":"2024-02-26T04:57:21.177528+0000","last_undegraded":"2024-02-26T04:57:21.177528+0000","last_fullsized":"2024-02-26T04:57:21.177528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:18:00.766266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176570+0000","last_change":"2024-02-26T04:55:27.200385+0000","last_active":"2024-02-26T04:57:21.176570+0000","last_peered":"2024-02-26T04:57:21.176570+0000","last_clean":"2024-02-26T04:57:21.176570+0000","last_became_active":"2024-02-26T04:55:27.200046+0000","last_became_peered":"2024-02-26T04:55:27.200046+0000","last_unstale":"2024-02-26T04:57:21.176570+0000","last_undegraded":"2024-02-26T04:57:21.176570+0000","last_fullsized":"2024-02-26T04:57:21.176570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:01:58.112457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181168+0000","last_change":"2024-02-26T04:55:28.220702+0000","last_active":"2024-02-26T04:57:21.181168+0000","last_peered":"2024-02-26T04:57:21.181168+0000","last_clean":"2024-02-26T04:57:21.181168+0000","last_became_active":"2024-02-26T04:55:28.220386+0000","last_became_peered":"2024-02-26T04:55:28.220386+0000","last_unstale":"2024-02-26T04:57:21.181168+0000","last_undegraded":"2024-02-26T04:57:21.181168+0000","last_fullsized":"2024-02-26T04:57:21.181168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:40:37.774831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183841+0000","last_change":"2024-02-26T04:55:28.206409+0000","last_active":"2024-02-26T04:57:21.183841+0000","last_peered":"2024-02-26T04:57:21.183841+0000","last_clean":"2024-02-26T04:57:21.183841+0000","last_became_active":"2024-02-26T04:55:28.206046+0000","last_became_peered":"2024-02-26T04:55:28.206046+0000","last_unstale":"2024-02-26T04:57:21.183841+0000","last_undegraded":"2024-02-26T04:57:21.183841+0000","last_fullsized":"2024-02-26T04:57:21.183841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:35:43.277376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181921+0000","last_change":"2024-02-26T04:55:28.216146+0000","last_active":"2024-02-26T04:57:21.181921+0000","last_peered":"2024-02-26T04:57:21.181921+0000","last_clean":"2024-02-26T04:57:21.181921+0000","last_became_active":"2024-02-26T04:55:28.215800+0000","last_became_peered":"2024-02-26T04:55:28.215800+0000","last_unstale":"2024-02-26T04:57:21.181921+0000","last_undegraded":"2024-02-26T04:57:21.181921+0000","last_fullsized":"2024-02-26T04:57:21.181921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:18:47.902694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180868+0000","last_change":"2024-02-26T04:55:28.221587+0000","last_active":"2024-02-26T04:57:21.180868+0000","last_peered":"2024-02-26T04:57:21.180868+0000","last_clean":"2024-02-26T04:57:21.180868+0000","last_became_active":"2024-02-26T04:55:28.220990+0000","last_became_peered":"2024-02-26T04:55:28.220990+0000","last_unstale":"2024-02-26T04:57:21.180868+0000","last_undegraded":"2024-02-26T04:57:21.180868+0000","last_fullsized":"2024-02-26T04:57:21.180868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:24:13.302081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391622+0000","last_change":"2024-02-26T04:55:28.220586+0000","last_active":"2024-02-26T04:57:21.391622+0000","last_peered":"2024-02-26T04:57:21.391622+0000","last_clean":"2024-02-26T04:57:21.391622+0000","last_became_active":"2024-02-26T04:55:28.219585+0000","last_became_peered":"2024-02-26T04:55:28.219585+0000","last_unstale":"2024-02-26T04:57:21.391622+0000","last_undegraded":"2024-02-26T04:57:21.391622+0000","last_fullsized":"2024-02-26T04:57:21.391622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:41.786142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176530+0000","last_change":"2024-02-26T04:55:28.204599+0000","last_active":"2024-02-26T04:57:21.176530+0000","last_peered":"2024-02-26T04:57:21.176530+0000","last_clean":"2024-02-26T04:57:21.176530+0000","last_became_active":"2024-02-26T04:55:28.204252+0000","last_became_peered":"2024-02-26T04:55:28.204252+0000","last_unstale":"2024-02-26T04:57:21.176530+0000","last_undegraded":"2024-02-26T04:57:21.176530+0000","last_fullsized":"2024-02-26T04:57:21.176530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:30:19.807897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183609+0000","last_change":"2024-02-26T04:55:28.214233+0000","last_active":"2024-02-26T04:57:21.183609+0000","last_peered":"2024-02-26T04:57:21.183609+0000","last_clean":"2024-02-26T04:57:21.183609+0000","last_became_active":"2024-02-26T04:55:28.213927+0000","last_became_peered":"2024-02-26T04:55:28.213927+0000","last_unstale":"2024-02-26T04:57:21.183609+0000","last_undegraded":"2024-02-26T04:57:21.183609+0000","last_fullsized":"2024-02-26T04:57:21.183609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:31:07.967134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181680+0000","last_change":"2024-02-26T04:55:28.217610+0000","last_active":"2024-02-26T04:57:21.181680+0000","last_peered":"2024-02-26T04:57:21.181680+0000","last_clean":"2024-02-26T04:57:21.181680+0000","last_became_active":"2024-02-26T04:55:28.217263+0000","last_became_peered":"2024-02-26T04:55:28.217263+0000","last_unstale":"2024-02-26T04:57:21.181680+0000","last_undegraded":"2024-02-26T04:57:21.181680+0000","last_fullsized":"2024-02-26T04:57:21.181680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:28:18.831887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179617+0000","last_change":"2024-02-26T04:56:20.510859+0000","last_active":"2024-02-26T04:57:21.179617+0000","last_peered":"2024-02-26T04:57:21.179617+0000","last_clean":"2024-02-26T04:57:21.179617+0000","last_became_active":"2024-02-26T04:56:20.510420+0000","last_became_peered":"2024-02-26T04:56:20.510420+0000","last_unstale":"2024-02-26T04:57:21.179617+0000","last_undegraded":"2024-02-26T04:57:21.179617+0000","last_fullsized":"2024-02-26T04:57:21.179617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:02.961996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177656+0000","last_change":"2024-02-26T04:55:28.217247+0000","last_active":"2024-02-26T04:57:21.177656+0000","last_peered":"2024-02-26T04:57:21.177656+0000","last_clean":"2024-02-26T04:57:21.177656+0000","last_became_active":"2024-02-26T04:55:28.214691+0000","last_became_peered":"2024-02-26T04:55:28.214691+0000","last_unstale":"2024-02-26T04:57:21.177656+0000","last_undegraded":"2024-02-26T04:57:21.177656+0000","last_fullsized":"2024-02-26T04:57:21.177656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:27:45.564520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186211+0000","last_change":"2024-02-26T04:55:28.210491+0000","last_active":"2024-02-26T04:57:21.186211+0000","last_peered":"2024-02-26T04:57:21.186211+0000","last_clean":"2024-02-26T04:57:21.186211+0000","last_became_active":"2024-02-26T04:55:28.210144+0000","last_became_peered":"2024-02-26T04:55:28.210144+0000","last_unstale":"2024-02-26T04:57:21.186211+0000","last_undegraded":"2024-02-26T04:57:21.186211+0000","last_fullsized":"2024-02-26T04:57:21.186211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:11.214716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363822+0000","last_change":"2024-02-26T04:55:28.216037+0000","last_active":"2024-02-26T04:57:21.363822+0000","last_peered":"2024-02-26T04:57:21.363822+0000","last_clean":"2024-02-26T04:57:21.363822+0000","last_became_active":"2024-02-26T04:55:28.215712+0000","last_became_peered":"2024-02-26T04:55:28.215712+0000","last_unstale":"2024-02-26T04:57:21.363822+0000","last_undegraded":"2024-02-26T04:57:21.363822+0000","last_fullsized":"2024-02-26T04:57:21.363822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:38:55.437642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176653+0000","last_change":"2024-02-26T04:55:28.205469+0000","last_active":"2024-02-26T04:57:21.176653+0000","last_peered":"2024-02-26T04:57:21.176653+0000","last_clean":"2024-02-26T04:57:21.176653+0000","last_became_active":"2024-02-26T04:55:28.205137+0000","last_became_peered":"2024-02-26T04:55:28.205137+0000","last_unstale":"2024-02-26T04:57:21.176653+0000","last_undegraded":"2024-02-26T04:57:21.176653+0000","last_fullsized":"2024-02-26T04:57:21.176653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:50.425010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.182926+0000","last_change":"2024-02-26T04:55:28.213066+0000","last_active":"2024-02-26T04:57:21.182926+0000","last_peered":"2024-02-26T04:57:21.182926+0000","last_clean":"2024-02-26T04:57:21.182926+0000","last_became_active":"2024-02-26T04:55:28.212706+0000","last_became_peered":"2024-02-26T04:55:28.212706+0000","last_unstale":"2024-02-26T04:57:21.182926+0000","last_undegraded":"2024-02-26T04:57:21.182926+0000","last_fullsized":"2024-02-26T04:57:21.182926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:21:24.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281347,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67215,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":728,"num_read_kb":436,"num_write":166,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":582,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":201953,"data_compressed":4869,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":520717,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.43099999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68100000000000005}]},{"osd":1,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.51500000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.45900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.56799999999999995}]},{"osd":2,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.33600000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.504,"15min":0.5},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":4,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.67800000000000005,"15min":0.67500000000000004},"min":{"1min":0.443,"5min":0.442,"15min":0.442},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.48299999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77000000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.41499999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84799999999999998,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.623},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91400000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59099999999999997}]},{"osd":6,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.56000000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1559999999999999,"5min":1.175,"15min":1.175},"last":0.72399999999999998}]},{"osd":7,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.74399999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.75,"15min":0.748},"min":{"1min":0.59299999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.88400000000000001}]},{"osd":8,"last update":"Mon Feb 26 04:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72299999999999998,"15min":0.71799999999999997},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.73899999999999999,"15min":0.73699999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.90000000000000002}]},{"osd":9,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.79900000000000004,"15min":0.79300000000000004},"min":{"1min":0.68100000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0369999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.80400000000000005,"15min":0.79800000000000004},"min":{"1min":0.622,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0960000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.81699999999999995}]},{"osd":10,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.63500000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.63200000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.86599999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1163264,"data_stored":1171008,"data_compressed":63914,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.91800000000000004,"15min":0.98299999999999998},"min":{"1min":0.37,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.749,"5min":9.6609999999999996,"15min":9.6609999999999996},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.96599999999999997,"15min":1.0269999999999999},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90500000000000003,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":0.46000000000000002}]},{"osd":1,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.499},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70799999999999996,"5min":0.871,"15min":0.871},"last":0.53400000000000003}]},{"osd":2,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.435,"5min":0.309,"15min":0.309},"max":{"1min":0.73199999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81699999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69399999999999995}]},{"osd":3,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.69999999999999996,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":8.3840000000000003,"5min":8.3840000000000003,"15min":8.3840000000000003},"last":0.629},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.67500000000000004,"15min":0.63100000000000001},"min":{"1min":0.38600000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":8.4160000000000004,"5min":8.4160000000000004,"15min":8.4160000000000004},"last":0.67500000000000004}]},{"osd":4,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56100000000000005,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.77300000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56100000000000005,"15min":0.57299999999999995},"min":{"1min":0.35499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80300000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.56000000000000005}]},{"osd":5,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68200000000000005,"15min":0.68700000000000006},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.87,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.52900000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.88400000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.76200000000000001}]},{"osd":6,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":0.79900000000000004,"15min":0.753},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":9.7119999999999997,"5min":9.7119999999999997,"15min":9.7119999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.54400000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89800000000000002,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.77800000000000002}]},{"osd":7,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.72699999999999998}]},{"osd":8,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.751,"15min":0.76500000000000001},"min":{"1min":0.52900000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93500000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.79200000000000004,"15min":0.81399999999999995},"min":{"1min":0.52800000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94699999999999995,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.80500000000000005}]},{"osd":9,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":0.84299999999999997,"15min":0.80400000000000005},"min":{"1min":0.57099999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":8.3439999999999994,"5min":8.3439999999999994,"15min":8.3439999999999994},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81000000000000005,"15min":0.83299999999999996},"min":{"1min":0.55500000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97199999999999998,"5min":1.274,"15min":1.274},"last":0.73599999999999999}]},{"osd":11,"last update":"Mon Feb 26 04:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.63300000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91500000000000004,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78500000000000003,"15min":0.78800000000000003},"min":{"1min":0.58399999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88800000000000001,"5min":1.21,"15min":1.21},"last":0.82099999999999995}]}]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":524467,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.44400000000000001,"15min":0.47399999999999998},"min":{"1min":0.30299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.54100000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.46300000000000002,"15min":0.49299999999999999},"min":{"1min":0.28100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64900000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.32700000000000001}]},{"osd":1,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60399999999999998,"15min":0.627},"min":{"1min":0.40999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77000000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.433},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.64600000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79700000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.36799999999999999}]},{"osd":2,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.64600000000000002,"15min":0.66900000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73499999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.72999999999999998,"15min":0.79300000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77000000000000002,"5min":3.173,"15min":3.173},"last":0.42299999999999999}]},{"osd":3,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.49299999999999999,"15min":0.52100000000000002},"min":{"1min":0.27400000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.60099999999999998,"5min":0.753,"15min":0.753},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.498,"15min":0.51800000000000002},"min":{"1min":0.30599999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64000000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38300000000000001}]},{"osd":4,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.64500000000000002,"15min":0.68400000000000005},"min":{"1min":0.44400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77500000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63700000000000001,"15min":0.66500000000000004},"min":{"1min":0.36899999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.70199999999999996}]},{"osd":5,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.67600000000000005,"15min":0.70199999999999996},"min":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.68799999999999994,"15min":0.71699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.76600000000000001,"5min":1.133,"15min":1.133},"last":0.41299999999999998}]},{"osd":6,"last update":"Mon Feb 26 04:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.71199999999999997,"15min":0.754},"min":{"1min":0.46300000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83299999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.73499999999999999,"15min":0.77200000000000002},"min":{"1min":0.46100000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81899999999999995,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.45400000000000001}]},{"osd":8,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.84799999999999998,"15min":0.91900000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85699999999999998,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.501},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76400000000000001,"15min":0.80200000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.63300000000000001}]},{"osd":10,"last update":"Mon Feb 26 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.74199999999999999,"15min":0.75800000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.89200000000000002,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73999999999999999,"15min":0.752},"min":{"1min":0.58699999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.872,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.73499999999999999}]},{"osd":11,"last update":"Mon Feb 26 04:57:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.73999999999999999,"15min":0.73399999999999999},"min":{"1min":0.628,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.61099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.51100000000000001}]}]},{"osd":8,"up_from":59,"seq":253403070523,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":520291,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.49099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.84799999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.37},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.66600000000000004,"15min":0.57399999999999995},"min":{"1min":0.44900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.81699999999999995,"5min":12.359999999999999,"15min":12.359999999999999},"last":0.39800000000000002}]},{"osd":1,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78200000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.66000000000000003,"15min":0.626},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.87,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.502}]},{"osd":2,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.46800000000000003,"15min":0.44},"min":{"1min":0.42699999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.48699999999999999,"15min":0.44800000000000001},"min":{"1min":0.41099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73599999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55200000000000005}]},{"osd":3,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58699999999999997,"15min":0.56000000000000005},"min":{"1min":0.48499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.499,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76700000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.48299999999999998}]},{"osd":4,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64300000000000002,"15min":0.60899999999999999},"min":{"1min":0.45100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88100000000000001,"5min":1.2,"15min":1.2},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64400000000000002,"15min":0.621},"min":{"1min":0.51900000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89500000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.52400000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.59699999999999998,"15min":0.54900000000000004},"min":{"1min":0.55100000000000005,"5min":0.309,"15min":0.309},"max":{"1min":0.92500000000000004,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.59399999999999997,"15min":0.55200000000000005},"min":{"1min":0.47099999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.91300000000000003,"5min":1.27,"15min":1.27},"last":0.69199999999999995}]},{"osd":6,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63900000000000001,"15min":0.61499999999999999},"min":{"1min":0.53900000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.84299999999999997,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.84299999999999997,"15min":0.73099999999999998},"min":{"1min":0.66700000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97499999999999998,"5min":12.384,"15min":12.384},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.752,"15min":0.71399999999999997},"min":{"1min":0.72999999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.73999999999999999,"15min":0.68799999999999994},"min":{"1min":0.63800000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.97399999999999998,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.73299999999999998}]},{"osd":9,"last update":"Mon Feb 26 04:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.76600000000000001,"15min":0.71899999999999997},"min":{"1min":0.65100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.996,"5min":1.073,"15min":1.073},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.746,"15min":0.69299999999999995},"min":{"1min":0.65500000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94299999999999995,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.65500000000000003}]},{"osd":10,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.78000000000000003,"15min":0.748},"min":{"1min":0.67900000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0229999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.80700000000000005,"15min":0.77500000000000002},"min":{"1min":0.71499999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.84099999999999997}]},{"osd":11,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.72999999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0660000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.879},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.85299999999999998},"min":{"1min":0.66100000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.995,"5min":1.238,"15min":1.238},"last":0.82299999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1081344,"data_stored":1115535,"data_compressed":62291,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67000000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.40799999999999997}]},{"osd":1,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42399999999999999,"15min":0.41799999999999998},"min":{"1min":0.27900000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45700000000000002,"15min":0.443},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.58999999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.55000000000000004}]},{"osd":2,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82299999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84599999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72599999999999998}]},{"osd":3,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.49299999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79600000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81599999999999995,"5min":0.877,"15min":0.877},"last":0.69999999999999996}]},{"osd":4,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.30499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.753,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.58699999999999997}]},{"osd":5,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.46500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86499999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66500000000000004,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.79200000000000004}]},{"osd":6,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.46200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.56200000000000006,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85499999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.83799999999999997}]},{"osd":8,"last update":"Mon Feb 26 04:57:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.53700000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93000000000000005,"5min":1.044,"15min":1.044},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72099999999999997,"15min":0.71399999999999997},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.89900000000000002,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.89700000000000002}]},{"osd":9,"last update":"Mon Feb 26 04:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.78000000000000003,"15min":0.74299999999999999},"min":{"1min":0.57699999999999996,"5min":0.504,"15min":0.504},"max":{"1min":4.7160000000000002,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.71899999999999997},"min":{"1min":0.59899999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.93100000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.93100000000000005}]},{"osd":10,"last update":"Mon Feb 26 04:57:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.747,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.48899999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.995,"5min":1.119,"15min":1.119},"last":0.86899999999999999}]},{"osd":11,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79600000000000004,"15min":0.79400000000000004},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95399999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.65300000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96799999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.88700000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":515547,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42799999999999999,"15min":0.39200000000000002},"min":{"1min":0.29299999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.434},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.433}]},{"osd":1,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64200000000000002,"15min":0.62},"min":{"1min":0.497,"5min":0.376,"15min":0.376},"max":{"1min":0.878,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.498},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.621,"15min":0.57899999999999996},"min":{"1min":0.40699999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85899999999999999,"5min":1.032,"15min":1.032},"last":0.59999999999999998}]},{"osd":2,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.627,"15min":0.58099999999999996},"min":{"1min":0.51800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.95399999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.751,"15min":1.0209999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94299999999999995,"5min":14.884,"15min":14.884},"last":0.57099999999999995}]},{"osd":3,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.48099999999999998,"15min":0.45600000000000002},"min":{"1min":0.33200000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71099999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54200000000000004,"15min":0.51800000000000002},"min":{"1min":0.38200000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.83199999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55300000000000005}]},{"osd":4,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.63700000000000001},"min":{"1min":0.441,"5min":0.38,"15min":0.38},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.79400000000000004,"15min":1.0720000000000001},"min":{"1min":0.45600000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.97699999999999998,"5min":14.898,"15min":14.898},"last":0.53500000000000003}]},{"osd":5,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68100000000000005,"15min":0.64700000000000002},"min":{"1min":0.50800000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90100000000000002,"5min":1.288,"15min":1.288},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70899999999999996,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0009999999999999,"5min":1.151,"15min":1.151},"last":0.61599999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76500000000000001,"15min":0.74199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71399999999999997,"15min":0.69199999999999995},"min":{"1min":0.501,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.044,"5min":1.075,"15min":1.075},"last":0.63600000000000001}]},{"osd":8,"last update":"Mon Feb 26 04:57:26 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":0.90200000000000002,"15min":0.81599999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":12.880000000000001,"5min":12.880000000000001,"15min":12.880000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.4570000000000001,"5min":0.90400000000000003,"15min":0.81200000000000006},"min":{"1min":0.58499999999999996,"5min":0.432,"15min":0.432},"max":{"1min":12.863,"5min":12.863,"15min":12.863},"last":0.69899999999999995}]},{"osd":9,"last update":"Mon Feb 26 04:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.745,"15min":0.79100000000000004},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96299999999999997,"5min":1.111,"15min":1.111},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.78500000000000003},"min":{"1min":0.437,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.008,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.72499999999999998}]},{"osd":10,"last update":"Mon Feb 26 04:57:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.84799999999999998,"15min":0.86899999999999999},"min":{"1min":0.622,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1200000000000001,"5min":1.133,"15min":1.133},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.58999999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0700000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.74299999999999999}]},{"osd":11,"last update":"Mon Feb 26 04:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81100000000000005,"15min":0.80900000000000005},"min":{"1min":0.68799999999999994,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.099,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83099999999999996,"15min":0.83399999999999996},"min":{"1min":0.67600000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1399999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.755}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":579062,"data_compressed":28131,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55400000000000005,"15min":0.502},"min":{"1min":0.36099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.625,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.749},{"interface":"front","average":{"1min":0.496,"5min":0.54500000000000004,"15min":0.505},"min":{"1min":0.35799999999999998,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.70499999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61099999999999999}]},{"osd":2,"last update":"Mon Feb 26 04:56:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.60599999999999998,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59299999999999997,"15min":0.54100000000000004},"min":{"1min":0.48199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83899999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53500000000000003}]},{"osd":3,"last update":"Mon Feb 26 04:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.55200000000000005},"min":{"1min":0.41199999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69999999999999996,"5min":0.997,"15min":0.997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63800000000000001,"15min":0.57899999999999996},"min":{"1min":0.45300000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73199999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64400000000000002}]},{"osd":4,"last update":"Mon Feb 26 04:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.42799999999999999},"min":{"1min":0.34100000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.48699999999999999,"15min":0.435},"min":{"1min":0.36699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.443}]},{"osd":5,"last update":"Mon Feb 26 04:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64400000000000002,"15min":0.57199999999999995},"min":{"1min":0.44900000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85599999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63,"15min":0.56100000000000005},"min":{"1min":0.42799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97499999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.72699999999999998}]},{"osd":6,"last update":"Mon Feb 26 04:57:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.49099999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66500000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.627},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.71699999999999997,"15min":0.66600000000000004},"min":{"1min":0.60499999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.79800000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.70799999999999996}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.55200000000000005},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88300000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75600000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68400000000000005}]},{"osd":8,"last update":"Mon Feb 26 04:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.748,"15min":0.71899999999999997},"min":{"1min":0.53600000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.90800000000000003,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.74199999999999999,"15min":0.71799999999999997},"min":{"1min":0.58399999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":1.04,"15min":1.04},"last":0.82499999999999996}]},{"osd":9,"last update":"Mon Feb 26 04:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1080000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.81100000000000005}]},{"osd":10,"last update":"Mon Feb 26 04:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.79400000000000004,"15min":0.79500000000000004},"min":{"1min":0.57099999999999995,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96499999999999997,"5min":1.095,"15min":1.095},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92300000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.76200000000000001}]},{"osd":11,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.89100000000000001,"15min":0.90300000000000002},"min":{"1min":0.625,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.93899999999999995,"5min":1.141,"15min":1.141},"last":0.875},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.83999999999999997,"15min":0.84899999999999998},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.79000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":516935,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 04:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.51300000000000001},"min":{"1min":0.38400000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83099999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.88700000000000001,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Mon Feb 26 04:56:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56899999999999995,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.33500000000000002,"15min":0.28999999999999998},"max":{"1min":0.81899999999999995,"5min":1.006,"15min":1.006},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60899999999999999,"15min":0.55700000000000005},"min":{"1min":0.45000000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.83799999999999997}]},{"osd":3,"last update":"Mon Feb 26 04:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.45700000000000002,"15min":0.46300000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.57799999999999996,"5min":1.208,"15min":1.208},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Feb 26 04:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65900000000000003,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.98399999999999999,"5min":1.073,"15min":1.073},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63100000000000001,"15min":0.56299999999999994},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.65200000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60999999999999999,"15min":0.56200000000000006},"min":{"1min":0.42399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85199999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66600000000000004,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.89200000000000002}]},{"osd":6,"last update":"Mon Feb 26 04:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52500000000000002,"15min":0.51600000000000001},"min":{"1min":0.34100000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.754,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.54400000000000004},"min":{"1min":0.38,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80400000000000005,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Feb 26 04:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.54600000000000004,"5min":0.374,"15min":0.374},"max":{"1min":0.87,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.749,"15min":0.751},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93600000000000005,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.69599999999999995}]},{"osd":8,"last update":"Mon Feb 26 04:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.44900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96799999999999997,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73199999999999998,"15min":0.75},"min":{"1min":0.48299999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96599999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.78800000000000003}]},{"osd":9,"last update":"Mon Feb 26 04:56:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.46500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.80100000000000005},"min":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.51800000000000002}]},{"osd":10,"last update":"Mon Feb 26 04:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.83099999999999996,"15min":0.84099999999999997},"min":{"1min":0.61599999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0609999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.61799999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.036,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.73699999999999999}]},{"osd":11,"last update":"Mon Feb 26 04:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.65100000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.96899999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.75},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78400000000000003,"15min":0.78300000000000003},"min":{"1min":0.64800000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0349999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.82199999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":580412,"data_compressed":28131,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.53900000000000003,"15min":0.47299999999999998},"min":{"1min":0.41599999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.96399999999999997,"5min":1.002,"15min":1.002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53000000000000003,"15min":0.47399999999999998},"min":{"1min":0.378,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.46700000000000003}]},{"osd":1,"last update":"Mon Feb 26 04:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.50600000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.57899999999999996,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.71199999999999997}]},{"osd":3,"last update":"Mon Feb 26 04:57:27 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.623,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.32000000000000001},"max":{"1min":0.93799999999999994,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.64300000000000002,"15min":0.58499999999999996},"min":{"1min":0.49299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94599999999999995,"5min":1.004,"15min":1.004},"last":0.63600000000000001}]},{"osd":4,"last update":"Mon Feb 26 04:57:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.75600000000000001,"15min":0.77000000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.85799999999999998,"5min":1.002,"15min":1.002},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.875,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68400000000000005}]},{"osd":5,"last update":"Mon Feb 26 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.35199999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.48299999999999998,"15min":0.45100000000000001},"min":{"1min":0.35599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.71099999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56799999999999995}]},{"osd":6,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63700000000000001,"15min":0.56899999999999995},"min":{"1min":0.51100000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.877,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.64000000000000001,"15min":0.57699999999999996},"min":{"1min":0.46899999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91400000000000003,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.499}]},{"osd":7,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75900000000000001,"15min":0.77000000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91400000000000003,"5min":1.038,"15min":1.038},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.78700000000000003,"15min":0.80300000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":1.014,"15min":1.014},"last":0.83599999999999997}]},{"osd":8,"last update":"Mon Feb 26 04:57:27 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.871,"5min":1.115,"15min":1.115},"last":0.746},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.35099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.88700000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.72799999999999998}]},{"osd":9,"last update":"Mon Feb 26 04:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.68899999999999995,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.73699999999999999,"15min":0.71899999999999997},"min":{"1min":0.48699999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.042,"5min":1.056,"15min":1.056},"last":0.88800000000000001}]},{"osd":10,"last update":"Mon Feb 26 04:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.70099999999999996},"min":{"1min":0.55600000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72999999999999998,"15min":0.69699999999999995},"min":{"1min":0.48199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96499999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80600000000000005}]},{"osd":11,"last update":"Mon Feb 26 04:57:27 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.82499999999999996,"15min":0.83099999999999996},"min":{"1min":0.53400000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.008,"5min":1.109,"15min":1.109},"last":0.871},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.80700000000000005,"15min":0.81200000000000006},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.85099999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":515741,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.34000000000000002},"min":{"1min":0.309,"5min":0.21199999999999999,"15min":0.17899999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.315},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.33800000000000002},"min":{"1min":0.28100000000000003,"5min":0.22700000000000001,"15min":0.20699999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.28999999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.52700000000000002},"min":{"1min":0.38,"5min":0.36599999999999999,"15min":0.35299999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57099999999999995,"15min":0.53100000000000003},"min":{"1min":0.40100000000000002,"5min":0.34499999999999997,"15min":0.32600000000000001},"max":{"1min":0.76200000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.61599999999999999}]},{"osd":2,"last update":"Mon Feb 26 04:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.42799999999999999,"5min":0.36699999999999999,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60199999999999998,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.32700000000000001},"max":{"1min":0.76300000000000001,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.59299999999999997}]},{"osd":4,"last update":"Mon Feb 26 04:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.48499999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82699999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64900000000000002,"15min":0.623},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.56100000000000005}]},{"osd":5,"last update":"Mon Feb 26 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.56899999999999995},"min":{"1min":0.40699999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.69499999999999995}]},{"osd":6,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50800000000000001,"15min":0.47999999999999998},"min":{"1min":0.33600000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.93899999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54100000000000004,"15min":0.51400000000000001},"min":{"1min":0.40300000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85599999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.36499999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66600000000000004,"15min":0.629},"min":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95999999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.629}]},{"osd":8,"last update":"Mon Feb 26 04:55:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.874,"5min":0.995,"15min":0.995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.878,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63100000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90500000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.44700000000000001}]},{"osd":10,"last update":"Mon Feb 26 04:57:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78000000000000003,"15min":0.79100000000000004},"min":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.038,"5min":1.232,"15min":1.232},"last":0.65700000000000003}]},{"osd":11,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.57899999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.751,"15min":0.75},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.67700000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":520197,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:54:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.51100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79500000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42899999999999999}]},{"osd":1,"last update":"Mon Feb 26 04:57:42 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.436,"15min":0.378},"min":{"1min":0.33600000000000002,"5min":0.26300000000000001,"15min":0.24299999999999999},"max":{"1min":0.58399999999999996,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.44},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.438,"15min":0.39200000000000002},"min":{"1min":0.36399999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.54200000000000004,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.38700000000000001}]},{"osd":2,"last update":"Mon Feb 26 04:57:42 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58399999999999996,"15min":0.53400000000000003},"min":{"1min":0.442,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82199999999999995,"5min":1.657,"15min":1.657},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.60399999999999998,"15min":0.55200000000000005},"min":{"1min":0.496,"5min":0.40899999999999997,"15min":0.38},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.66000000000000003}]},{"osd":3,"last update":"Mon Feb 26 04:57:42 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61899999999999999,"15min":0.56200000000000006},"min":{"1min":0.41599999999999998,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.70499999999999996,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60499999999999998,"15min":0.54700000000000004},"min":{"1min":0.46300000000000002,"5min":0.36699999999999999,"15min":0.35599999999999998},"max":{"1min":0.81599999999999995,"5min":2.133,"15min":2.133},"last":0.74099999999999999}]},{"osd":5,"last update":"Mon Feb 26 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59299999999999997,"15min":0.55400000000000005},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89000000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63,"15min":0.56399999999999995},"min":{"1min":0.41799999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82199999999999995,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.78000000000000003}]},{"osd":6,"last update":"Mon Feb 26 04:57:40 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64000000000000001,"15min":0.58799999999999997},"min":{"1min":0.441,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75700000000000001,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.65200000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84499999999999997,"5min":4.1639999999999997,"15min":4.1639999999999997},"last":0.85399999999999998}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.55300000000000005},"min":{"1min":0.34200000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.55500000000000005},"min":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":4.2300000000000004,"15min":4.2300000000000004},"last":0.50600000000000001}]},{"osd":8,"last update":"Mon Feb 26 04:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71299999999999997,"15min":0.68600000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92700000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68500000000000005,"15min":0.64500000000000002},"min":{"1min":0.54400000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93200000000000005,"5min":1.907,"15min":1.907},"last":0.63200000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:57:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.82899999999999996,"15min":0.749},"min":{"1min":0.55800000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.84199999999999997,"5min":10.051,"15min":10.051},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.82899999999999996,"15min":0.76500000000000001},"min":{"1min":0.58099999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94499999999999995,"5min":6.3360000000000003,"15min":6.3360000000000003},"last":0.94499999999999995}]},{"osd":10,"last update":"Mon Feb 26 04:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.78200000000000003,"15min":0.77800000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94799999999999995,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.88},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95499999999999996,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.58599999999999997}]},{"osd":11,"last update":"Mon Feb 26 04:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.97099999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82099999999999995,"15min":0.82899999999999996},"min":{"1min":0.59099999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.95899999999999996,"5min":1.946,"15min":1.946},"last":0.71699999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":1111397,"data_compressed":62291,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51300000000000001,"15min":0.49099999999999999},"min":{"1min":0.39600000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.32900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59599999999999997}]},{"osd":1,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77400000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58599999999999997,"15min":0.53900000000000003},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85799999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.45700000000000002}]},{"osd":2,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.41299999999999998,"15min":0.36899999999999999},"min":{"1min":0.30099999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.72199999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.42099999999999999,"15min":0.38700000000000001},"min":{"1min":0.32800000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.34699999999999998}]},{"osd":3,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.54800000000000004,"15min":0.52000000000000002},"min":{"1min":0.41399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.56899999999999995,"15min":0.51300000000000001},"min":{"1min":0.40100000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.8010000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":0.70699999999999996}]},{"osd":4,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63900000000000001,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83999999999999997,"5min":0.871,"15min":0.871},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58999999999999997,"15min":0.56000000000000005},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.82399999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Feb 26 04:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63500000000000001,"15min":0.59699999999999998},"min":{"1min":0.52300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91800000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63200000000000001,"15min":0.60999999999999999},"min":{"1min":0.51300000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80000000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.67600000000000005}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67400000000000004,"15min":0.65700000000000003},"min":{"1min":0.46500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89600000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53600000000000003}]},{"osd":8,"last update":"Mon Feb 26 04:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61099999999999999,"15min":0.59599999999999997},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.36399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.498}]},{"osd":9,"last update":"Mon Feb 26 04:56:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.67600000000000005,"15min":0.67000000000000004},"min":{"1min":0.49199999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.70699999999999996,"15min":0.68500000000000005},"min":{"1min":0.50700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.72999999999999998}]},{"osd":10,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.48299999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.62},{"interface":"front","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.90900000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":63605,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64955,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65183,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T04:57:56.120 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T04:57:56.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.256+0000 7fa4e351e640 1 -- 172.21.15.60:0/3513195734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4dc100a20 msgr2=0x7fa4dc100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.256+0000 7fa4e351e640 1 --2- 172.21.15.60:0/3513195734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4dc100a20 0x7fa4dc100ea0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009900 tx=0x7fa4d002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:56.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.256+0000 7fa4e351e640 1 -- 172.21.15.60:0/3513195734 shutdown_connections 2024-02-26T04:57:56.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.256+0000 7fa4e351e640 1 --2- 172.21.15.60:0/3513195734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4dc1013e0 0x7fa4dc107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.256+0000 7fa4e351e640 1 --2- 172.21.15.60:0/3513195734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4dc100a20 0x7fa4dc100ea0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.256+0000 7fa4e351e640 1 --2- 172.21.15.60:0/3513195734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4dc0ff830 0x7fa4dc0ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.256+0000 7fa4e351e640 1 -- 172.21.15.60:0/3513195734 >> 172.21.15.60:0/3513195734 conn(0x7fa4dc0fada0 msgr2=0x7fa4dc0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:56.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.256+0000 7fa4e351e640 1 -- 172.21.15.60:0/3513195734 shutdown_connections 2024-02-26T04:57:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.256+0000 7fa4e351e640 1 -- 172.21.15.60:0/3513195734 wait complete. 2024-02-26T04:57:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e351e640 1 Processor -- start 2024-02-26T04:57:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e351e640 1 -- start start 2024-02-26T04:57:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e351e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4dc0ff830 0x7fa4dc197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e351e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4dc100a20 0x7fa4dc1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e351e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4dc1013e0 0x7fa4dc19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e351e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc19ed80 con 0x7fa4dc0ff830 2024-02-26T04:57:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e351e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc19eef0 con 0x7fa4dc1013e0 2024-02-26T04:57:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e351e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc19f0b0 con 0x7fa4dc100a20 2024-02-26T04:57:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e12ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4dc0ff830 0x7fa4dc197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e12ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4dc0ff830 0x7fa4dc197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44202/0 (socket says 172.21.15.60:44202) 2024-02-26T04:57:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e12ba640 1 -- 172.21.15.60:0/454467043 learned_addr learned my addr 172.21.15.60:0/454467043 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e1abb640 1 --2- 172.21.15.60:0/454467043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4dc1013e0 0x7fa4dc19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e0ab9640 1 --2- 172.21.15.60:0/454467043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4dc100a20 0x7fa4dc1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:56.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e12ba640 1 -- 172.21.15.60:0/454467043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4dc100a20 msgr2=0x7fa4dc1977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e12ba640 1 --2- 172.21.15.60:0/454467043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4dc100a20 0x7fa4dc1977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e12ba640 1 -- 172.21.15.60:0/454467043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4dc1013e0 msgr2=0x7fa4dc19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.257+0000 7fa4e12ba640 1 --2- 172.21.15.60:0/454467043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4dc1013e0 0x7fa4dc19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.258+0000 7fa4e12ba640 1 -- 172.21.15.60:0/454467043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d0009560 con 0x7fa4dc0ff830 2024-02-26T04:57:56.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.258+0000 7fa4e12ba640 1 --2- 172.21.15.60:0/454467043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4dc0ff830 0x7fa4dc197290 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400d770 tx=0x7fa4d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:56.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.258+0000 7fa4da7fc640 1 -- 172.21.15.60:0/454467043 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4009c30 con 0x7fa4dc0ff830 2024-02-26T04:57:56.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.258+0000 7fa4da7fc640 1 -- 172.21.15.60:0/454467043 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4d400de80 con 0x7fa4dc0ff830 2024-02-26T04:57:56.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.258+0000 7fa4e351e640 1 -- 172.21.15.60:0/454467043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4dc19f390 con 0x7fa4dc0ff830 2024-02-26T04:57:56.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.258+0000 7fa4da7fc640 1 -- 172.21.15.60:0/454467043 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4010460 con 0x7fa4dc0ff830 2024-02-26T04:57:56.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.258+0000 7fa4e351e640 1 -- 172.21.15.60:0/454467043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4dc19f8a0 con 0x7fa4dc0ff830 2024-02-26T04:57:56.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.259+0000 7fa4e351e640 1 -- 172.21.15.60:0/454467043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4dc10b210 con 0x7fa4dc0ff830 2024-02-26T04:57:56.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.260+0000 7fa4da7fc640 1 -- 172.21.15.60:0/454467043 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fa4d4003f60 con 0x7fa4dc0ff830 2024-02-26T04:57:56.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.260+0000 7fa4da7fc640 1 --2- 172.21.15.60:0/454467043 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa4c40a20c0 0x7fa4c40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:56.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.261+0000 7fa4e0ab9640 1 --2- 172.21.15.60:0/454467043 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa4c40a20c0 0x7fa4c40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:56.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.261+0000 7fa4da7fc640 1 -- 172.21.15.60:0/454467043 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fa4d40cdf20 con 0x7fa4dc0ff830 2024-02-26T04:57:56.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.261+0000 7fa4e0ab9640 1 --2- 172.21.15.60:0/454467043 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa4c40a20c0 0x7fa4c40a4580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc1987a0 tx=0x7fa4d0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:56.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.264+0000 7fa4da7fc640 1 -- 172.21.15.60:0/454467043 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa4d4096a50 con 0x7fa4dc0ff830 2024-02-26T04:57:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.443+0000 7fa4e351e640 1 -- 172.21.15.60:0/454467043 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa4dc0fcaf0 con 0x7fa4c40a20c0 2024-02-26T04:57:56.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.456+0000 7fa4da7fc640 1 -- 172.21.15.60:0/454467043 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427917 (secure 0 0 0) 0x7fa4dc0fcaf0 con 0x7fa4c40a20c0 2024-02-26T04:57:56.463 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:56.465 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-02-26T04:57:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.459+0000 7fa4e351e640 1 -- 172.21.15.60:0/454467043 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa4c40a20c0 msgr2=0x7fa4c40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.459+0000 7fa4e351e640 1 --2- 172.21.15.60:0/454467043 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa4c40a20c0 0x7fa4c40a4580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc1987a0 tx=0x7fa4d0000bc0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.459+0000 7fa4e351e640 1 -- 172.21.15.60:0/454467043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4dc0ff830 msgr2=0x7fa4dc197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.459+0000 7fa4e351e640 1 --2- 172.21.15.60:0/454467043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4dc0ff830 0x7fa4dc197290 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400d770 tx=0x7fa4d400dc30 comp rx=0 tx=0).stop 2024-02-26T04:57:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.459+0000 7fa4e351e640 1 -- 172.21.15.60:0/454467043 shutdown_connections 2024-02-26T04:57:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.459+0000 7fa4e351e640 1 --2- 172.21.15.60:0/454467043 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa4c40a20c0 0x7fa4c40a4580 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.459+0000 7fa4e351e640 1 --2- 172.21.15.60:0/454467043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4dc1013e0 0x7fa4dc19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.459+0000 7fa4e351e640 1 --2- 172.21.15.60:0/454467043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4dc100a20 0x7fa4dc1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.459+0000 7fa4e351e640 1 --2- 172.21.15.60:0/454467043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4dc0ff830 0x7fa4dc197290 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.459+0000 7fa4e351e640 1 -- 172.21.15.60:0/454467043 >> 172.21.15.60:0/454467043 conn(0x7fa4dc0fada0 msgr2=0x7fa4dc0fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.460+0000 7fa4e351e640 1 -- 172.21.15.60:0/454467043 shutdown_connections 2024-02-26T04:57:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.460+0000 7fa4e351e640 1 -- 172.21.15.60:0/454467043 wait complete. 2024-02-26T04:57:56.472 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:56 smithi060 ceph-mon[32382]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T04:57:56.472 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:56 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/889553695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:56.472 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:56 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4002182226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:56.472 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:56 smithi060 ceph-mon[32382]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:56.473 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-02-26T04:57:56.345435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663806,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":881,"num_read_kb":562,"num_write":355,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181318,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386628,"kb_used_data":12452,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555196,"statfs":{"total":1151940427776,"available":1151544520704,"internally_reserved":0,"allocated":12750848,"data_stored":8208968,"data_compressed":430314,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5989,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":47,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003013"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184951+0000","last_change":"2024-02-26T04:55:28.208832+0000","last_active":"2024-02-26T04:57:21.184951+0000","last_peered":"2024-02-26T04:57:21.184951+0000","last_clean":"2024-02-26T04:57:21.184951+0000","last_became_active":"2024-02-26T04:55:28.208454+0000","last_became_peered":"2024-02-26T04:55:28.208454+0000","last_unstale":"2024-02-26T04:57:21.184951+0000","last_undegraded":"2024-02-26T04:57:21.184951+0000","last_fullsized":"2024-02-26T04:57:21.184951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:55:54.132384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391600+0000","last_change":"2024-02-26T04:55:28.203876+0000","last_active":"2024-02-26T04:57:21.391600+0000","last_peered":"2024-02-26T04:57:21.391600+0000","last_clean":"2024-02-26T04:57:21.391600+0000","last_became_active":"2024-02-26T04:55:28.203399+0000","last_became_peered":"2024-02-26T04:55:28.203399+0000","last_unstale":"2024-02-26T04:57:21.391600+0000","last_undegraded":"2024-02-26T04:57:21.391600+0000","last_fullsized":"2024-02-26T04:57:21.391600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:31:40.927929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391340+0000","last_change":"2024-02-26T04:55:28.211388+0000","last_active":"2024-02-26T04:57:21.391340+0000","last_peered":"2024-02-26T04:57:21.391340+0000","last_clean":"2024-02-26T04:57:21.391340+0000","last_became_active":"2024-02-26T04:55:28.209007+0000","last_became_peered":"2024-02-26T04:55:28.209007+0000","last_unstale":"2024-02-26T04:57:21.391340+0000","last_undegraded":"2024-02-26T04:57:21.391340+0000","last_fullsized":"2024-02-26T04:57:21.391340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:37:01.197589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177443+0000","last_change":"2024-02-26T04:55:28.216378+0000","last_active":"2024-02-26T04:57:21.177443+0000","last_peered":"2024-02-26T04:57:21.177443+0000","last_clean":"2024-02-26T04:57:21.177443+0000","last_became_active":"2024-02-26T04:55:28.216074+0000","last_became_peered":"2024-02-26T04:55:28.216074+0000","last_unstale":"2024-02-26T04:57:21.177443+0000","last_undegraded":"2024-02-26T04:57:21.177443+0000","last_fullsized":"2024-02-26T04:57:21.177443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:15:20.460699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180626+0000","last_change":"2024-02-26T04:55:28.213026+0000","last_active":"2024-02-26T04:57:21.180626+0000","last_peered":"2024-02-26T04:57:21.180626+0000","last_clean":"2024-02-26T04:57:21.180626+0000","last_became_active":"2024-02-26T04:55:28.212706+0000","last_became_peered":"2024-02-26T04:55:28.212706+0000","last_unstale":"2024-02-26T04:57:21.180626+0000","last_undegraded":"2024-02-26T04:57:21.180626+0000","last_fullsized":"2024-02-26T04:57:21.180626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:57:35.961956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.245681+0000","last_change":"2024-02-26T04:55:28.202123+0000","last_active":"2024-02-26T04:57:21.245681+0000","last_peered":"2024-02-26T04:57:21.245681+0000","last_clean":"2024-02-26T04:57:21.245681+0000","last_became_active":"2024-02-26T04:55:28.201733+0000","last_became_peered":"2024-02-26T04:55:28.201733+0000","last_unstale":"2024-02-26T04:57:21.245681+0000","last_undegraded":"2024-02-26T04:57:21.245681+0000","last_fullsized":"2024-02-26T04:57:21.245681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:43:11.300006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180147+0000","last_change":"2024-02-26T04:56:20.508346+0000","last_active":"2024-02-26T04:57:21.180147+0000","last_peered":"2024-02-26T04:57:21.180147+0000","last_clean":"2024-02-26T04:57:21.180147+0000","last_became_active":"2024-02-26T04:56:20.507941+0000","last_became_peered":"2024-02-26T04:56:20.507941+0000","last_unstale":"2024-02-26T04:57:21.180147+0000","last_undegraded":"2024-02-26T04:57:21.180147+0000","last_fullsized":"2024-02-26T04:57:21.180147+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:35:27.425477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.185909+0000","last_change":"2024-02-26T04:55:28.214631+0000","last_active":"2024-02-26T04:57:21.185909+0000","last_peered":"2024-02-26T04:57:21.185909+0000","last_clean":"2024-02-26T04:57:21.185909+0000","last_became_active":"2024-02-26T04:55:28.214290+0000","last_became_peered":"2024-02-26T04:55:28.214290+0000","last_unstale":"2024-02-26T04:57:21.185909+0000","last_undegraded":"2024-02-26T04:57:21.185909+0000","last_fullsized":"2024-02-26T04:57:21.185909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:30:11.494833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179697+0000","last_change":"2024-02-26T04:56:20.508783+0000","last_active":"2024-02-26T04:57:21.179697+0000","last_peered":"2024-02-26T04:57:21.179697+0000","last_clean":"2024-02-26T04:57:21.179697+0000","last_became_active":"2024-02-26T04:56:20.508372+0000","last_became_peered":"2024-02-26T04:56:20.508372+0000","last_unstale":"2024-02-26T04:57:21.179697+0000","last_undegraded":"2024-02-26T04:57:21.179697+0000","last_fullsized":"2024-02-26T04:57:21.179697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:32.415022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.246168+0000","last_change":"2024-02-26T04:55:28.204176+0000","last_active":"2024-02-26T04:57:21.246168+0000","last_peered":"2024-02-26T04:57:21.246168+0000","last_clean":"2024-02-26T04:57:21.246168+0000","last_became_active":"2024-02-26T04:55:28.203821+0000","last_became_peered":"2024-02-26T04:55:28.203821+0000","last_unstale":"2024-02-26T04:57:21.246168+0000","last_undegraded":"2024-02-26T04:57:21.246168+0000","last_fullsized":"2024-02-26T04:57:21.246168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:24:03.132650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180742+0000","last_change":"2024-02-26T04:55:28.215958+0000","last_active":"2024-02-26T04:57:21.180742+0000","last_peered":"2024-02-26T04:57:21.180742+0000","last_clean":"2024-02-26T04:57:21.180742+0000","last_became_active":"2024-02-26T04:55:28.215594+0000","last_became_peered":"2024-02-26T04:55:28.215594+0000","last_unstale":"2024-02-26T04:57:21.180742+0000","last_undegraded":"2024-02-26T04:57:21.180742+0000","last_fullsized":"2024-02-26T04:57:21.180742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:45:47.915379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177442+0000","last_change":"2024-02-26T04:55:28.218392+0000","last_active":"2024-02-26T04:57:21.177442+0000","last_peered":"2024-02-26T04:57:21.177442+0000","last_clean":"2024-02-26T04:57:21.177442+0000","last_became_active":"2024-02-26T04:55:28.217933+0000","last_became_peered":"2024-02-26T04:55:28.217933+0000","last_unstale":"2024-02-26T04:57:21.177442+0000","last_undegraded":"2024-02-26T04:57:21.177442+0000","last_fullsized":"2024-02-26T04:57:21.177442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:30:38.880649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176503+0000","last_change":"2024-02-26T04:55:28.197099+0000","last_active":"2024-02-26T04:57:21.176503+0000","last_peered":"2024-02-26T04:57:21.176503+0000","last_clean":"2024-02-26T04:57:21.176503+0000","last_became_active":"2024-02-26T04:55:28.196755+0000","last_became_peered":"2024-02-26T04:55:28.196755+0000","last_unstale":"2024-02-26T04:57:21.176503+0000","last_undegraded":"2024-02-26T04:57:21.176503+0000","last_fullsized":"2024-02-26T04:57:21.176503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:37:31.685845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391224+0000","last_change":"2024-02-26T04:55:28.212064+0000","last_active":"2024-02-26T04:57:21.391224+0000","last_peered":"2024-02-26T04:57:21.391224+0000","last_clean":"2024-02-26T04:57:21.391224+0000","last_became_active":"2024-02-26T04:55:28.208742+0000","last_became_peered":"2024-02-26T04:55:28.208742+0000","last_unstale":"2024-02-26T04:57:21.391224+0000","last_undegraded":"2024-02-26T04:57:21.391224+0000","last_fullsized":"2024-02-26T04:57:21.391224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:38:19.742468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391162+0000","last_change":"2024-02-26T04:56:20.511014+0000","last_active":"2024-02-26T04:57:21.391162+0000","last_peered":"2024-02-26T04:57:21.391162+0000","last_clean":"2024-02-26T04:57:21.391162+0000","last_became_active":"2024-02-26T04:56:20.510463+0000","last_became_peered":"2024-02-26T04:56:20.510463+0000","last_unstale":"2024-02-26T04:57:21.391162+0000","last_undegraded":"2024-02-26T04:57:21.391162+0000","last_fullsized":"2024-02-26T04:57:21.391162+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:44:59.574588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363448+0000","last_change":"2024-02-26T04:55:28.222810+0000","last_active":"2024-02-26T04:57:21.363448+0000","last_peered":"2024-02-26T04:57:21.363448+0000","last_clean":"2024-02-26T04:57:21.363448+0000","last_became_active":"2024-02-26T04:55:28.222341+0000","last_became_peered":"2024-02-26T04:55:28.222341+0000","last_unstale":"2024-02-26T04:57:21.363448+0000","last_undegraded":"2024-02-26T04:57:21.363448+0000","last_fullsized":"2024-02-26T04:57:21.363448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:59:21.627517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184837+0000","last_change":"2024-02-26T04:55:28.208377+0000","last_active":"2024-02-26T04:57:21.184837+0000","last_peered":"2024-02-26T04:57:21.184837+0000","last_clean":"2024-02-26T04:57:21.184837+0000","last_became_active":"2024-02-26T04:55:28.207932+0000","last_became_peered":"2024-02-26T04:55:28.207932+0000","last_unstale":"2024-02-26T04:57:21.184837+0000","last_undegraded":"2024-02-26T04:57:21.184837+0000","last_fullsized":"2024-02-26T04:57:21.184837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:15:39.586191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179812+0000","last_change":"2024-02-26T04:55:28.201105+0000","last_active":"2024-02-26T04:57:21.179812+0000","last_peered":"2024-02-26T04:57:21.179812+0000","last_clean":"2024-02-26T04:57:21.179812+0000","last_became_active":"2024-02-26T04:55:28.200796+0000","last_became_peered":"2024-02-26T04:55:28.200796+0000","last_unstale":"2024-02-26T04:57:21.179812+0000","last_undegraded":"2024-02-26T04:57:21.179812+0000","last_fullsized":"2024-02-26T04:57:21.179812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:53:43.713055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181623+0000","last_change":"2024-02-26T04:55:28.215725+0000","last_active":"2024-02-26T04:57:21.181623+0000","last_peered":"2024-02-26T04:57:21.181623+0000","last_clean":"2024-02-26T04:57:21.181623+0000","last_became_active":"2024-02-26T04:55:28.215341+0000","last_became_peered":"2024-02-26T04:55:28.215341+0000","last_unstale":"2024-02-26T04:57:21.181623+0000","last_undegraded":"2024-02-26T04:57:21.181623+0000","last_fullsized":"2024-02-26T04:57:21.181623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:26:26.960474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180646+0000","last_change":"2024-02-26T04:55:32.233303+0000","last_active":"2024-02-26T04:57:21.180646+0000","last_peered":"2024-02-26T04:57:21.180646+0000","last_clean":"2024-02-26T04:57:21.180646+0000","last_became_active":"2024-02-26T04:55:32.232901+0000","last_became_peered":"2024-02-26T04:55:32.232901+0000","last_unstale":"2024-02-26T04:57:21.180646+0000","last_undegraded":"2024-02-26T04:57:21.180646+0000","last_fullsized":"2024-02-26T04:57:21.180646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:37:49.659668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.194950+0000","last_change":"2024-02-26T04:57:21.194629+0000","last_active":"2024-02-26T04:57:21.194950+0000","last_peered":"2024-02-26T04:57:21.194950+0000","last_clean":"2024-02-26T04:57:21.194950+0000","last_became_active":"2024-02-26T04:57:21.186661+0000","last_became_peered":"2024-02-26T04:57:21.186661+0000","last_unstale":"2024-02-26T04:57:21.194950+0000","last_undegraded":"2024-02-26T04:57:21.194950+0000","last_fullsized":"2024-02-26T04:57:21.194950+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:08:47.904102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179621+0000","last_change":"2024-02-26T04:56:20.511029+0000","last_active":"2024-02-26T04:57:21.179621+0000","last_peered":"2024-02-26T04:57:21.179621+0000","last_clean":"2024-02-26T04:57:21.179621+0000","last_became_active":"2024-02-26T04:56:20.510590+0000","last_became_peered":"2024-02-26T04:56:20.510590+0000","last_unstale":"2024-02-26T04:57:21.179621+0000","last_undegraded":"2024-02-26T04:57:21.179621+0000","last_fullsized":"2024-02-26T04:57:21.179621+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:01:25.641444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176799+0000","last_change":"2024-02-26T04:55:32.233142+0000","last_active":"2024-02-26T04:57:21.176799+0000","last_peered":"2024-02-26T04:57:21.176799+0000","last_clean":"2024-02-26T04:57:21.176799+0000","last_became_active":"2024-02-26T04:55:32.232722+0000","last_became_peered":"2024-02-26T04:55:32.232722+0000","last_unstale":"2024-02-26T04:57:21.176799+0000","last_undegraded":"2024-02-26T04:57:21.176799+0000","last_fullsized":"2024-02-26T04:57:21.176799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:30:20.906104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186620+0000","last_change":"2024-02-26T04:55:27.203420+0000","last_active":"2024-02-26T04:57:21.186620+0000","last_peered":"2024-02-26T04:57:21.186620+0000","last_clean":"2024-02-26T04:57:21.186620+0000","last_became_active":"2024-02-26T04:55:27.202968+0000","last_became_peered":"2024-02-26T04:55:27.202968+0000","last_unstale":"2024-02-26T04:57:21.186620+0000","last_undegraded":"2024-02-26T04:57:21.186620+0000","last_fullsized":"2024-02-26T04:57:21.186620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:44.496528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391314+0000","last_change":"2024-02-26T04:55:28.217422+0000","last_active":"2024-02-26T04:57:21.391314+0000","last_peered":"2024-02-26T04:57:21.391314+0000","last_clean":"2024-02-26T04:57:21.391314+0000","last_became_active":"2024-02-26T04:55:28.217103+0000","last_became_peered":"2024-02-26T04:55:28.217103+0000","last_unstale":"2024-02-26T04:57:21.391314+0000","last_undegraded":"2024-02-26T04:57:21.391314+0000","last_fullsized":"2024-02-26T04:57:21.391314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:08:37.075467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176904+0000","last_change":"2024-02-26T04:55:32.233322+0000","last_active":"2024-02-26T04:57:21.176904+0000","last_peered":"2024-02-26T04:57:21.176904+0000","last_clean":"2024-02-26T04:57:21.176904+0000","last_became_active":"2024-02-26T04:55:32.232679+0000","last_became_peered":"2024-02-26T04:55:32.232679+0000","last_unstale":"2024-02-26T04:57:21.176904+0000","last_undegraded":"2024-02-26T04:57:21.176904+0000","last_fullsized":"2024-02-26T04:57:21.176904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:10.775924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.192327+0000","last_change":"2024-02-26T04:57:21.192327+0000","last_active":"2024-02-26T04:57:21.192327+0000","last_peered":"2024-02-26T04:57:21.192327+0000","last_clean":"2024-02-26T04:57:21.192327+0000","last_became_active":"2024-02-26T04:57:21.186299+0000","last_became_peered":"2024-02-26T04:57:21.186299+0000","last_unstale":"2024-02-26T04:57:21.192327+0000","last_undegraded":"2024-02-26T04:57:21.192327+0000","last_fullsized":"2024-02-26T04:57:21.192327+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:15.566202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363709+0000","last_change":"2024-02-26T04:55:28.210669+0000","last_active":"2024-02-26T04:57:21.363709+0000","last_peered":"2024-02-26T04:57:21.363709+0000","last_clean":"2024-02-26T04:57:21.363709+0000","last_became_active":"2024-02-26T04:55:28.210322+0000","last_became_peered":"2024-02-26T04:55:28.210322+0000","last_unstale":"2024-02-26T04:57:21.363709+0000","last_undegraded":"2024-02-26T04:57:21.363709+0000","last_fullsized":"2024-02-26T04:57:21.363709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:57:29.121762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176799+0000","last_change":"2024-02-26T04:55:32.229077+0000","last_active":"2024-02-26T04:57:21.176799+0000","last_peered":"2024-02-26T04:57:21.176799+0000","last_clean":"2024-02-26T04:57:21.176799+0000","last_became_active":"2024-02-26T04:55:32.228670+0000","last_became_peered":"2024-02-26T04:55:32.228670+0000","last_unstale":"2024-02-26T04:57:21.176799+0000","last_undegraded":"2024-02-26T04:57:21.176799+0000","last_fullsized":"2024-02-26T04:57:21.176799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:35:20.843351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181671+0000","last_change":"2024-02-26T04:55:27.206088+0000","last_active":"2024-02-26T04:57:21.181671+0000","last_peered":"2024-02-26T04:57:21.181671+0000","last_clean":"2024-02-26T04:57:21.181671+0000","last_became_active":"2024-02-26T04:55:27.205696+0000","last_became_peered":"2024-02-26T04:55:27.205696+0000","last_unstale":"2024-02-26T04:57:21.181671+0000","last_undegraded":"2024-02-26T04:57:21.181671+0000","last_fullsized":"2024-02-26T04:57:21.181671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:14:03.552251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178333+0000","last_change":"2024-02-26T04:56:20.508805+0000","last_active":"2024-02-26T04:57:21.178333+0000","last_peered":"2024-02-26T04:57:21.178333+0000","last_clean":"2024-02-26T04:57:21.178333+0000","last_became_active":"2024-02-26T04:56:20.507796+0000","last_became_peered":"2024-02-26T04:56:20.507796+0000","last_unstale":"2024-02-26T04:57:21.178333+0000","last_undegraded":"2024-02-26T04:57:21.178333+0000","last_fullsized":"2024-02-26T04:57:21.178333+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:25.453111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.187012+0000","last_change":"2024-02-26T04:57:21.187012+0000","last_active":"2024-02-26T04:57:21.187012+0000","last_peered":"2024-02-26T04:57:21.187012+0000","last_clean":"2024-02-26T04:57:21.187012+0000","last_became_active":"2024-02-26T04:57:21.186512+0000","last_became_peered":"2024-02-26T04:57:21.186512+0000","last_unstale":"2024-02-26T04:57:21.187012+0000","last_undegraded":"2024-02-26T04:57:21.187012+0000","last_fullsized":"2024-02-26T04:57:21.187012+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:36:14.768552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:54.322839+0000","last_change":"2024-02-26T04:55:27.201261+0000","last_active":"2024-02-26T04:57:54.322839+0000","last_peered":"2024-02-26T04:57:54.322839+0000","last_clean":"2024-02-26T04:57:54.322839+0000","last_became_active":"2024-02-26T04:55:27.200892+0000","last_became_peered":"2024-02-26T04:55:27.200892+0000","last_unstale":"2024-02-26T04:57:54.322839+0000","last_undegraded":"2024-02-26T04:57:54.322839+0000","last_fullsized":"2024-02-26T04:57:54.322839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:34:32.774143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181394+0000","last_change":"2024-02-26T04:55:28.200445+0000","last_active":"2024-02-26T04:57:21.181394+0000","last_peered":"2024-02-26T04:57:21.181394+0000","last_clean":"2024-02-26T04:57:21.181394+0000","last_became_active":"2024-02-26T04:55:28.199959+0000","last_became_peered":"2024-02-26T04:55:28.199959+0000","last_unstale":"2024-02-26T04:57:21.181394+0000","last_undegraded":"2024-02-26T04:57:21.181394+0000","last_fullsized":"2024-02-26T04:57:21.181394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:36:27.397644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181032+0000","last_change":"2024-02-26T04:55:32.232619+0000","last_active":"2024-02-26T04:57:21.181032+0000","last_peered":"2024-02-26T04:57:21.181032+0000","last_clean":"2024-02-26T04:57:21.181032+0000","last_became_active":"2024-02-26T04:55:32.232112+0000","last_became_peered":"2024-02-26T04:55:32.232112+0000","last_unstale":"2024-02-26T04:57:21.181032+0000","last_undegraded":"2024-02-26T04:57:21.181032+0000","last_fullsized":"2024-02-26T04:57:21.181032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:19.741393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.245662+0000","last_change":"2024-02-26T04:55:27.197376+0000","last_active":"2024-02-26T04:57:21.245662+0000","last_peered":"2024-02-26T04:57:21.245662+0000","last_clean":"2024-02-26T04:57:21.245662+0000","last_became_active":"2024-02-26T04:55:27.195258+0000","last_became_peered":"2024-02-26T04:55:27.195258+0000","last_unstale":"2024-02-26T04:57:21.245662+0000","last_undegraded":"2024-02-26T04:57:21.245662+0000","last_fullsized":"2024-02-26T04:57:21.245662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:41:51.284525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178375+0000","last_change":"2024-02-26T04:55:28.214337+0000","last_active":"2024-02-26T04:57:21.178375+0000","last_peered":"2024-02-26T04:57:21.178375+0000","last_clean":"2024-02-26T04:57:21.178375+0000","last_became_active":"2024-02-26T04:55:28.213966+0000","last_became_peered":"2024-02-26T04:55:28.213966+0000","last_unstale":"2024-02-26T04:57:21.178375+0000","last_undegraded":"2024-02-26T04:57:21.178375+0000","last_fullsized":"2024-02-26T04:57:21.178375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:42.974084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184714+0000","last_change":"2024-02-26T04:55:32.235579+0000","last_active":"2024-02-26T04:57:21.184714+0000","last_peered":"2024-02-26T04:57:21.184714+0000","last_clean":"2024-02-26T04:57:21.184714+0000","last_became_active":"2024-02-26T04:55:32.235241+0000","last_became_peered":"2024-02-26T04:55:32.235241+0000","last_unstale":"2024-02-26T04:57:21.184714+0000","last_undegraded":"2024-02-26T04:57:21.184714+0000","last_fullsized":"2024-02-26T04:57:21.184714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:06.319874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176279+0000","last_change":"2024-02-26T04:55:27.197689+0000","last_active":"2024-02-26T04:57:21.176279+0000","last_peered":"2024-02-26T04:57:21.176279+0000","last_clean":"2024-02-26T04:57:21.176279+0000","last_became_active":"2024-02-26T04:55:27.197285+0000","last_became_peered":"2024-02-26T04:55:27.197285+0000","last_unstale":"2024-02-26T04:57:21.176279+0000","last_undegraded":"2024-02-26T04:57:21.176279+0000","last_fullsized":"2024-02-26T04:57:21.176279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:45:36.688902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.185917+0000","last_change":"2024-02-26T04:55:28.201902+0000","last_active":"2024-02-26T04:57:21.185917+0000","last_peered":"2024-02-26T04:57:21.185917+0000","last_clean":"2024-02-26T04:57:21.185917+0000","last_became_active":"2024-02-26T04:55:28.201604+0000","last_became_peered":"2024-02-26T04:55:28.201604+0000","last_unstale":"2024-02-26T04:57:21.185917+0000","last_undegraded":"2024-02-26T04:57:21.185917+0000","last_fullsized":"2024-02-26T04:57:21.185917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:48:57.711555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391379+0000","last_change":"2024-02-26T04:55:32.230153+0000","last_active":"2024-02-26T04:57:21.391379+0000","last_peered":"2024-02-26T04:57:21.391379+0000","last_clean":"2024-02-26T04:57:21.391379+0000","last_became_active":"2024-02-26T04:55:32.229695+0000","last_became_peered":"2024-02-26T04:55:32.229695+0000","last_unstale":"2024-02-26T04:57:21.391379+0000","last_undegraded":"2024-02-26T04:57:21.391379+0000","last_fullsized":"2024-02-26T04:57:21.391379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:15:39.575727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177757+0000","last_change":"2024-02-26T04:55:27.200943+0000","last_active":"2024-02-26T04:57:21.177757+0000","last_peered":"2024-02-26T04:57:21.177757+0000","last_clean":"2024-02-26T04:57:21.177757+0000","last_became_active":"2024-02-26T04:55:27.200565+0000","last_became_peered":"2024-02-26T04:55:27.200565+0000","last_unstale":"2024-02-26T04:57:21.177757+0000","last_undegraded":"2024-02-26T04:57:21.177757+0000","last_fullsized":"2024-02-26T04:57:21.177757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:54:19.670631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179352+0000","last_change":"2024-02-26T04:55:28.212232+0000","last_active":"2024-02-26T04:57:21.179352+0000","last_peered":"2024-02-26T04:57:21.179352+0000","last_clean":"2024-02-26T04:57:21.179352+0000","last_became_active":"2024-02-26T04:55:28.211875+0000","last_became_peered":"2024-02-26T04:55:28.211875+0000","last_unstale":"2024-02-26T04:57:21.179352+0000","last_undegraded":"2024-02-26T04:57:21.179352+0000","last_fullsized":"2024-02-26T04:57:21.179352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:55.379271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186082+0000","last_change":"2024-02-26T04:55:32.234961+0000","last_active":"2024-02-26T04:57:21.186082+0000","last_peered":"2024-02-26T04:57:21.186082+0000","last_clean":"2024-02-26T04:57:21.186082+0000","last_became_active":"2024-02-26T04:55:32.234455+0000","last_became_peered":"2024-02-26T04:55:32.234455+0000","last_unstale":"2024-02-26T04:57:21.186082+0000","last_undegraded":"2024-02-26T04:57:21.186082+0000","last_fullsized":"2024-02-26T04:57:21.186082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:00:44.322341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":156,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:54.322208+0000","last_change":"2024-02-26T04:55:27.199036+0000","last_active":"2024-02-26T04:57:54.322208+0000","last_peered":"2024-02-26T04:57:54.322208+0000","last_clean":"2024-02-26T04:57:54.322208+0000","last_became_active":"2024-02-26T04:55:27.198694+0000","last_became_peered":"2024-02-26T04:55:27.198694+0000","last_unstale":"2024-02-26T04:57:54.322208+0000","last_undegraded":"2024-02-26T04:57:54.322208+0000","last_fullsized":"2024-02-26T04:57:54.322208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:31:41.935123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181847+0000","last_change":"2024-02-26T04:56:20.208105+0000","last_active":"2024-02-26T04:57:21.181847+0000","last_peered":"2024-02-26T04:57:21.181847+0000","last_clean":"2024-02-26T04:57:21.181847+0000","last_became_active":"2024-02-26T04:56:20.207691+0000","last_became_peered":"2024-02-26T04:56:20.207691+0000","last_unstale":"2024-02-26T04:57:21.181847+0000","last_undegraded":"2024-02-26T04:57:21.181847+0000","last_fullsized":"2024-02-26T04:57:21.181847+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:49:49.019089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,9],"acting":[8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179345+0000","last_change":"2024-02-26T04:55:32.228033+0000","last_active":"2024-02-26T04:57:21.179345+0000","last_peered":"2024-02-26T04:57:21.179345+0000","last_clean":"2024-02-26T04:57:21.179345+0000","last_became_active":"2024-02-26T04:55:32.224273+0000","last_became_peered":"2024-02-26T04:55:32.224273+0000","last_unstale":"2024-02-26T04:57:21.179345+0000","last_undegraded":"2024-02-26T04:57:21.179345+0000","last_fullsized":"2024-02-26T04:57:21.179345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:25:13.618421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180910+0000","last_change":"2024-02-26T04:55:27.202833+0000","last_active":"2024-02-26T04:57:21.180910+0000","last_peered":"2024-02-26T04:57:21.180910+0000","last_clean":"2024-02-26T04:57:21.180910+0000","last_became_active":"2024-02-26T04:55:27.202167+0000","last_became_peered":"2024-02-26T04:55:27.202167+0000","last_unstale":"2024-02-26T04:57:21.180910+0000","last_undegraded":"2024-02-26T04:57:21.180910+0000","last_fullsized":"2024-02-26T04:57:21.180910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:55:46.605763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391476+0000","last_change":"2024-02-26T04:56:20.207930+0000","last_active":"2024-02-26T04:57:21.391476+0000","last_peered":"2024-02-26T04:57:21.391476+0000","last_clean":"2024-02-26T04:57:21.391476+0000","last_became_active":"2024-02-26T04:56:20.207586+0000","last_became_peered":"2024-02-26T04:56:20.207586+0000","last_unstale":"2024-02-26T04:57:21.391476+0000","last_undegraded":"2024-02-26T04:57:21.391476+0000","last_fullsized":"2024-02-26T04:57:21.391476+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:37:57.141645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.365832+0000","last_change":"2024-02-26T04:57:21.365354+0000","last_active":"2024-02-26T04:57:21.365832+0000","last_peered":"2024-02-26T04:57:21.365832+0000","last_clean":"2024-02-26T04:57:21.365832+0000","last_became_active":"2024-02-26T04:57:21.364768+0000","last_became_peered":"2024-02-26T04:57:21.364768+0000","last_unstale":"2024-02-26T04:57:21.365832+0000","last_undegraded":"2024-02-26T04:57:21.365832+0000","last_fullsized":"2024-02-26T04:57:21.365832+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:33:05.881044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178420+0000","last_change":"2024-02-26T04:55:27.200389+0000","last_active":"2024-02-26T04:57:21.178420+0000","last_peered":"2024-02-26T04:57:21.178420+0000","last_clean":"2024-02-26T04:57:21.178420+0000","last_became_active":"2024-02-26T04:55:27.200019+0000","last_became_peered":"2024-02-26T04:55:27.200019+0000","last_unstale":"2024-02-26T04:57:21.178420+0000","last_undegraded":"2024-02-26T04:57:21.178420+0000","last_fullsized":"2024-02-26T04:57:21.178420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:59:11.820988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363892+0000","last_change":"2024-02-26T04:55:28.220309+0000","last_active":"2024-02-26T04:57:21.363892+0000","last_peered":"2024-02-26T04:57:21.363892+0000","last_clean":"2024-02-26T04:57:21.363892+0000","last_became_active":"2024-02-26T04:55:28.219987+0000","last_became_peered":"2024-02-26T04:55:28.219987+0000","last_unstale":"2024-02-26T04:57:21.363892+0000","last_undegraded":"2024-02-26T04:57:21.363892+0000","last_fullsized":"2024-02-26T04:57:21.363892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:39:34.713802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179368+0000","last_change":"2024-02-26T04:55:32.236817+0000","last_active":"2024-02-26T04:57:21.179368+0000","last_peered":"2024-02-26T04:57:21.179368+0000","last_clean":"2024-02-26T04:57:21.179368+0000","last_became_active":"2024-02-26T04:55:32.235818+0000","last_became_peered":"2024-02-26T04:55:32.235818+0000","last_unstale":"2024-02-26T04:57:21.179368+0000","last_undegraded":"2024-02-26T04:57:21.179368+0000","last_fullsized":"2024-02-26T04:57:21.179368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:56:54.121684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'39","reported_seq":323,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:54.343952+0000","last_change":"2024-02-26T04:55:27.201582+0000","last_active":"2024-02-26T04:57:54.343952+0000","last_peered":"2024-02-26T04:57:54.343952+0000","last_clean":"2024-02-26T04:57:54.343952+0000","last_became_active":"2024-02-26T04:55:27.201011+0000","last_became_peered":"2024-02-26T04:55:27.201011+0000","last_unstale":"2024-02-26T04:57:54.343952+0000","last_undegraded":"2024-02-26T04:57:54.343952+0000","last_fullsized":"2024-02-26T04:57:54.343952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:07:01.888924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69286,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":252,"num_read_kb":194,"num_write":40,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183993+0000","last_change":"2024-02-26T04:55:28.199313+0000","last_active":"2024-02-26T04:57:21.183993+0000","last_peered":"2024-02-26T04:57:21.183993+0000","last_clean":"2024-02-26T04:57:21.183993+0000","last_became_active":"2024-02-26T04:55:28.198973+0000","last_became_peered":"2024-02-26T04:55:28.198973+0000","last_unstale":"2024-02-26T04:57:21.183993+0000","last_undegraded":"2024-02-26T04:57:21.183993+0000","last_fullsized":"2024-02-26T04:57:21.183993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:33:49.005899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178664+0000","last_change":"2024-02-26T04:55:32.234529+0000","last_active":"2024-02-26T04:57:21.178664+0000","last_peered":"2024-02-26T04:57:21.178664+0000","last_clean":"2024-02-26T04:57:21.178664+0000","last_became_active":"2024-02-26T04:55:32.234105+0000","last_became_peered":"2024-02-26T04:55:32.234105+0000","last_unstale":"2024-02-26T04:57:21.178664+0000","last_undegraded":"2024-02-26T04:57:21.178664+0000","last_fullsized":"2024-02-26T04:57:21.178664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:17:40.909267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180944+0000","last_change":"2024-02-26T04:55:27.209178+0000","last_active":"2024-02-26T04:57:21.180944+0000","last_peered":"2024-02-26T04:57:21.180944+0000","last_clean":"2024-02-26T04:57:21.180944+0000","last_became_active":"2024-02-26T04:55:27.208834+0000","last_became_peered":"2024-02-26T04:55:27.208834+0000","last_unstale":"2024-02-26T04:57:21.180944+0000","last_undegraded":"2024-02-26T04:57:21.180944+0000","last_fullsized":"2024-02-26T04:57:21.180944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:28.187333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176297+0000","last_change":"2024-02-26T04:55:28.215208+0000","last_active":"2024-02-26T04:57:21.176297+0000","last_peered":"2024-02-26T04:57:21.176297+0000","last_clean":"2024-02-26T04:57:21.176297+0000","last_became_active":"2024-02-26T04:55:28.212135+0000","last_became_peered":"2024-02-26T04:55:28.212135+0000","last_unstale":"2024-02-26T04:57:21.176297+0000","last_undegraded":"2024-02-26T04:57:21.176297+0000","last_fullsized":"2024-02-26T04:57:21.176297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:51.269557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.182783+0000","last_change":"2024-02-26T04:55:32.235439+0000","last_active":"2024-02-26T04:57:21.182783+0000","last_peered":"2024-02-26T04:57:21.182783+0000","last_clean":"2024-02-26T04:57:21.182783+0000","last_became_active":"2024-02-26T04:55:32.234987+0000","last_became_peered":"2024-02-26T04:55:32.234987+0000","last_unstale":"2024-02-26T04:57:21.182783+0000","last_undegraded":"2024-02-26T04:57:21.182783+0000","last_fullsized":"2024-02-26T04:57:21.182783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:31:00.637846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177962+0000","last_change":"2024-02-26T04:55:27.205498+0000","last_active":"2024-02-26T04:57:21.177962+0000","last_peered":"2024-02-26T04:57:21.177962+0000","last_clean":"2024-02-26T04:57:21.177962+0000","last_became_active":"2024-02-26T04:55:27.205179+0000","last_became_peered":"2024-02-26T04:55:27.205179+0000","last_unstale":"2024-02-26T04:57:21.177962+0000","last_undegraded":"2024-02-26T04:57:21.177962+0000","last_fullsized":"2024-02-26T04:57:21.177962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:02:55.491896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180574+0000","last_change":"2024-02-26T04:55:28.201110+0000","last_active":"2024-02-26T04:57:21.180574+0000","last_peered":"2024-02-26T04:57:21.180574+0000","last_clean":"2024-02-26T04:57:21.180574+0000","last_became_active":"2024-02-26T04:55:28.200725+0000","last_became_peered":"2024-02-26T04:55:28.200725+0000","last_unstale":"2024-02-26T04:57:21.180574+0000","last_undegraded":"2024-02-26T04:57:21.180574+0000","last_fullsized":"2024-02-26T04:57:21.180574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:50.377925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.392866+0000","last_change":"2024-02-26T04:57:21.392457+0000","last_active":"2024-02-26T04:57:21.392866+0000","last_peered":"2024-02-26T04:57:21.392866+0000","last_clean":"2024-02-26T04:57:21.392866+0000","last_became_active":"2024-02-26T04:57:21.392057+0000","last_became_peered":"2024-02-26T04:57:21.392057+0000","last_unstale":"2024-02-26T04:57:21.392866+0000","last_undegraded":"2024-02-26T04:57:21.392866+0000","last_fullsized":"2024-02-26T04:57:21.392866+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:23:42.695300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"93'4","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:54.324261+0000","last_change":"2024-02-26T04:55:27.197338+0000","last_active":"2024-02-26T04:57:54.324261+0000","last_peered":"2024-02-26T04:57:54.324261+0000","last_clean":"2024-02-26T04:57:54.324261+0000","last_became_active":"2024-02-26T04:55:27.196898+0000","last_became_peered":"2024-02-26T04:55:27.196898+0000","last_unstale":"2024-02-26T04:57:54.324261+0000","last_undegraded":"2024-02-26T04:57:54.324261+0000","last_fullsized":"2024-02-26T04:57:54.324261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:27:09.635379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180780+0000","last_change":"2024-02-26T04:55:28.200665+0000","last_active":"2024-02-26T04:57:21.180780+0000","last_peered":"2024-02-26T04:57:21.180780+0000","last_clean":"2024-02-26T04:57:21.180780+0000","last_became_active":"2024-02-26T04:55:28.200329+0000","last_became_peered":"2024-02-26T04:55:28.200329+0000","last_unstale":"2024-02-26T04:57:21.180780+0000","last_undegraded":"2024-02-26T04:57:21.180780+0000","last_fullsized":"2024-02-26T04:57:21.180780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:27.090259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176631+0000","last_change":"2024-02-26T04:55:32.225861+0000","last_active":"2024-02-26T04:57:21.176631+0000","last_peered":"2024-02-26T04:57:21.176631+0000","last_clean":"2024-02-26T04:57:21.176631+0000","last_became_active":"2024-02-26T04:55:32.225187+0000","last_became_peered":"2024-02-26T04:55:32.225187+0000","last_unstale":"2024-02-26T04:57:21.176631+0000","last_undegraded":"2024-02-26T04:57:21.176631+0000","last_fullsized":"2024-02-26T04:57:21.176631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:14:09.077383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180544+0000","last_change":"2024-02-26T04:55:27.200321+0000","last_active":"2024-02-26T04:57:21.180544+0000","last_peered":"2024-02-26T04:57:21.180544+0000","last_clean":"2024-02-26T04:57:21.180544+0000","last_became_active":"2024-02-26T04:55:27.199978+0000","last_became_peered":"2024-02-26T04:55:27.199978+0000","last_unstale":"2024-02-26T04:57:21.180544+0000","last_undegraded":"2024-02-26T04:57:21.180544+0000","last_fullsized":"2024-02-26T04:57:21.180544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:08:20.381010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177958+0000","last_change":"2024-02-26T04:55:28.213940+0000","last_active":"2024-02-26T04:57:21.177958+0000","last_peered":"2024-02-26T04:57:21.177958+0000","last_clean":"2024-02-26T04:57:21.177958+0000","last_became_active":"2024-02-26T04:55:28.213590+0000","last_became_peered":"2024-02-26T04:55:28.213590+0000","last_unstale":"2024-02-26T04:57:21.177958+0000","last_undegraded":"2024-02-26T04:57:21.177958+0000","last_fullsized":"2024-02-26T04:57:21.177958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:29:29.151391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177711+0000","last_change":"2024-02-26T04:55:32.232669+0000","last_active":"2024-02-26T04:57:21.177711+0000","last_peered":"2024-02-26T04:57:21.177711+0000","last_clean":"2024-02-26T04:57:21.177711+0000","last_became_active":"2024-02-26T04:55:32.232288+0000","last_became_peered":"2024-02-26T04:55:32.232288+0000","last_unstale":"2024-02-26T04:57:21.177711+0000","last_undegraded":"2024-02-26T04:57:21.177711+0000","last_fullsized":"2024-02-26T04:57:21.177711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:31:42.436516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.182351+0000","last_change":"2024-02-26T04:55:27.205993+0000","last_active":"2024-02-26T04:57:21.182351+0000","last_peered":"2024-02-26T04:57:21.182351+0000","last_clean":"2024-02-26T04:57:21.182351+0000","last_became_active":"2024-02-26T04:55:27.205594+0000","last_became_peered":"2024-02-26T04:55:27.205594+0000","last_unstale":"2024-02-26T04:57:21.182351+0000","last_undegraded":"2024-02-26T04:57:21.182351+0000","last_fullsized":"2024-02-26T04:57:21.182351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:44:19.993925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184074+0000","last_change":"2024-02-26T04:55:28.203429+0000","last_active":"2024-02-26T04:57:21.184074+0000","last_peered":"2024-02-26T04:57:21.184074+0000","last_clean":"2024-02-26T04:57:21.184074+0000","last_became_active":"2024-02-26T04:55:28.203002+0000","last_became_peered":"2024-02-26T04:55:28.203002+0000","last_unstale":"2024-02-26T04:57:21.184074+0000","last_undegraded":"2024-02-26T04:57:21.184074+0000","last_fullsized":"2024-02-26T04:57:21.184074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:08:05.872241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181329+0000","last_change":"2024-02-26T04:55:32.239323+0000","last_active":"2024-02-26T04:57:21.181329+0000","last_peered":"2024-02-26T04:57:21.181329+0000","last_clean":"2024-02-26T04:57:21.181329+0000","last_became_active":"2024-02-26T04:55:32.238843+0000","last_became_peered":"2024-02-26T04:55:32.238843+0000","last_unstale":"2024-02-26T04:57:21.181329+0000","last_undegraded":"2024-02-26T04:57:21.181329+0000","last_fullsized":"2024-02-26T04:57:21.181329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:39:56.645429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179549+0000","last_change":"2024-02-26T04:55:27.203222+0000","last_active":"2024-02-26T04:57:21.179549+0000","last_peered":"2024-02-26T04:57:21.179549+0000","last_clean":"2024-02-26T04:57:21.179549+0000","last_became_active":"2024-02-26T04:55:27.202845+0000","last_became_peered":"2024-02-26T04:55:27.202845+0000","last_unstale":"2024-02-26T04:57:21.179549+0000","last_undegraded":"2024-02-26T04:57:21.179549+0000","last_fullsized":"2024-02-26T04:57:21.179549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:16:07.524260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.246028+0000","last_change":"2024-02-26T04:55:28.209122+0000","last_active":"2024-02-26T04:57:21.246028+0000","last_peered":"2024-02-26T04:57:21.246028+0000","last_clean":"2024-02-26T04:57:21.246028+0000","last_became_active":"2024-02-26T04:55:28.207598+0000","last_became_peered":"2024-02-26T04:55:28.207598+0000","last_unstale":"2024-02-26T04:57:21.246028+0000","last_undegraded":"2024-02-26T04:57:21.246028+0000","last_fullsized":"2024-02-26T04:57:21.246028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:02:10.763515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176657+0000","last_change":"2024-02-26T04:55:32.227333+0000","last_active":"2024-02-26T04:57:21.176657+0000","last_peered":"2024-02-26T04:57:21.176657+0000","last_clean":"2024-02-26T04:57:21.176657+0000","last_became_active":"2024-02-26T04:55:32.226956+0000","last_became_peered":"2024-02-26T04:55:32.226956+0000","last_unstale":"2024-02-26T04:57:21.176657+0000","last_undegraded":"2024-02-26T04:57:21.176657+0000","last_fullsized":"2024-02-26T04:57:21.176657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:51:40.710242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180411+0000","last_change":"2024-02-26T04:55:27.198781+0000","last_active":"2024-02-26T04:57:21.180411+0000","last_peered":"2024-02-26T04:57:21.180411+0000","last_clean":"2024-02-26T04:57:21.180411+0000","last_became_active":"2024-02-26T04:55:27.198288+0000","last_became_peered":"2024-02-26T04:55:27.198288+0000","last_unstale":"2024-02-26T04:57:21.180411+0000","last_undegraded":"2024-02-26T04:57:21.180411+0000","last_fullsized":"2024-02-26T04:57:21.180411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:52:18.665816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.245833+0000","last_change":"2024-02-26T04:55:28.213044+0000","last_active":"2024-02-26T04:57:21.245833+0000","last_peered":"2024-02-26T04:57:21.245833+0000","last_clean":"2024-02-26T04:57:21.245833+0000","last_became_active":"2024-02-26T04:55:28.212712+0000","last_became_peered":"2024-02-26T04:55:28.212712+0000","last_unstale":"2024-02-26T04:57:21.245833+0000","last_undegraded":"2024-02-26T04:57:21.245833+0000","last_fullsized":"2024-02-26T04:57:21.245833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:02:09.155551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176495+0000","last_change":"2024-02-26T04:55:32.231867+0000","last_active":"2024-02-26T04:57:21.176495+0000","last_peered":"2024-02-26T04:57:21.176495+0000","last_clean":"2024-02-26T04:57:21.176495+0000","last_became_active":"2024-02-26T04:55:32.231450+0000","last_became_peered":"2024-02-26T04:55:32.231450+0000","last_unstale":"2024-02-26T04:57:21.176495+0000","last_undegraded":"2024-02-26T04:57:21.176495+0000","last_fullsized":"2024-02-26T04:57:21.176495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:12:14.367427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177833+0000","last_change":"2024-02-26T04:55:28.207746+0000","last_active":"2024-02-26T04:57:21.177833+0000","last_peered":"2024-02-26T04:57:21.177833+0000","last_clean":"2024-02-26T04:57:21.177833+0000","last_became_active":"2024-02-26T04:55:28.207438+0000","last_became_peered":"2024-02-26T04:55:28.207438+0000","last_unstale":"2024-02-26T04:57:21.177833+0000","last_undegraded":"2024-02-26T04:57:21.177833+0000","last_fullsized":"2024-02-26T04:57:21.177833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:09.409635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":156,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:51.454112+0000","last_change":"2024-02-26T04:55:27.192233+0000","last_active":"2024-02-26T04:57:51.454112+0000","last_peered":"2024-02-26T04:57:51.454112+0000","last_clean":"2024-02-26T04:57:51.454112+0000","last_became_active":"2024-02-26T04:55:27.191868+0000","last_became_peered":"2024-02-26T04:55:27.191868+0000","last_unstale":"2024-02-26T04:57:51.454112+0000","last_undegraded":"2024-02-26T04:57:51.454112+0000","last_fullsized":"2024-02-26T04:57:51.454112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:37:58.430504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181068+0000","last_change":"2024-02-26T04:55:32.226577+0000","last_active":"2024-02-26T04:57:21.181068+0000","last_peered":"2024-02-26T04:57:21.181068+0000","last_clean":"2024-02-26T04:57:21.181068+0000","last_became_active":"2024-02-26T04:55:32.226148+0000","last_became_peered":"2024-02-26T04:55:32.226148+0000","last_unstale":"2024-02-26T04:57:21.181068+0000","last_undegraded":"2024-02-26T04:57:21.181068+0000","last_fullsized":"2024-02-26T04:57:21.181068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:34:53.099211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186167+0000","last_change":"2024-02-26T04:55:28.217865+0000","last_active":"2024-02-26T04:57:21.186167+0000","last_peered":"2024-02-26T04:57:21.186167+0000","last_clean":"2024-02-26T04:57:21.186167+0000","last_became_active":"2024-02-26T04:55:28.217075+0000","last_became_peered":"2024-02-26T04:55:28.217075+0000","last_unstale":"2024-02-26T04:57:21.186167+0000","last_undegraded":"2024-02-26T04:57:21.186167+0000","last_fullsized":"2024-02-26T04:57:21.186167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:30:38.720694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177120+0000","last_change":"2024-02-26T04:55:27.204300+0000","last_active":"2024-02-26T04:57:21.177120+0000","last_peered":"2024-02-26T04:57:21.177120+0000","last_clean":"2024-02-26T04:57:21.177120+0000","last_became_active":"2024-02-26T04:55:27.203933+0000","last_became_peered":"2024-02-26T04:55:27.203933+0000","last_unstale":"2024-02-26T04:57:21.177120+0000","last_undegraded":"2024-02-26T04:57:21.177120+0000","last_fullsized":"2024-02-26T04:57:21.177120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:19:36.247209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180188+0000","last_change":"2024-02-26T04:54:13.145088+0000","last_active":"2024-02-26T04:57:21.180188+0000","last_peered":"2024-02-26T04:57:21.180188+0000","last_clean":"2024-02-26T04:57:21.180188+0000","last_became_active":"2024-02-26T04:54:13.144657+0000","last_became_peered":"2024-02-26T04:54:13.144657+0000","last_unstale":"2024-02-26T04:57:21.180188+0000","last_undegraded":"2024-02-26T04:57:21.180188+0000","last_fullsized":"2024-02-26T04:57:21.180188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:50:45.246330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:50:45.246330+0000","last_clean_scrub_stamp":"2024-02-26T04:50:45.246330+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:00:50.842020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178258+0000","last_change":"2024-02-26T04:55:32.240124+0000","last_active":"2024-02-26T04:57:21.178258+0000","last_peered":"2024-02-26T04:57:21.178258+0000","last_clean":"2024-02-26T04:57:21.178258+0000","last_became_active":"2024-02-26T04:55:32.239624+0000","last_became_peered":"2024-02-26T04:55:32.239624+0000","last_unstale":"2024-02-26T04:57:21.178258+0000","last_undegraded":"2024-02-26T04:57:21.178258+0000","last_fullsized":"2024-02-26T04:57:21.178258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:40:33.770917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176056+0000","last_change":"2024-02-26T04:55:28.204631+0000","last_active":"2024-02-26T04:57:21.176056+0000","last_peered":"2024-02-26T04:57:21.176056+0000","last_clean":"2024-02-26T04:57:21.176056+0000","last_became_active":"2024-02-26T04:55:28.204274+0000","last_became_peered":"2024-02-26T04:55:28.204274+0000","last_unstale":"2024-02-26T04:57:21.176056+0000","last_undegraded":"2024-02-26T04:57:21.176056+0000","last_fullsized":"2024-02-26T04:57:21.176056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:18:18.336015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.187620+0000","last_change":"2024-02-26T04:57:21.187202+0000","last_active":"2024-02-26T04:57:21.187620+0000","last_peered":"2024-02-26T04:57:21.187620+0000","last_clean":"2024-02-26T04:57:21.187620+0000","last_became_active":"2024-02-26T04:57:21.183709+0000","last_became_peered":"2024-02-26T04:57:21.183709+0000","last_unstale":"2024-02-26T04:57:21.187620+0000","last_undegraded":"2024-02-26T04:57:21.187620+0000","last_fullsized":"2024-02-26T04:57:21.187620+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:05.733443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183832+0000","last_change":"2024-02-26T04:55:32.228900+0000","last_active":"2024-02-26T04:57:21.183832+0000","last_peered":"2024-02-26T04:57:21.183832+0000","last_clean":"2024-02-26T04:57:21.183832+0000","last_became_active":"2024-02-26T04:55:32.228262+0000","last_became_peered":"2024-02-26T04:55:32.228262+0000","last_unstale":"2024-02-26T04:57:21.183832+0000","last_undegraded":"2024-02-26T04:57:21.183832+0000","last_fullsized":"2024-02-26T04:57:21.183832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:33.069756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183858+0000","last_change":"2024-02-26T04:55:27.205264+0000","last_active":"2024-02-26T04:57:21.183858+0000","last_peered":"2024-02-26T04:57:21.183858+0000","last_clean":"2024-02-26T04:57:21.183858+0000","last_became_active":"2024-02-26T04:55:27.204882+0000","last_became_peered":"2024-02-26T04:55:27.204882+0000","last_unstale":"2024-02-26T04:57:21.183858+0000","last_undegraded":"2024-02-26T04:57:21.183858+0000","last_fullsized":"2024-02-26T04:57:21.183858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:14:19.497725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176445+0000","last_change":"2024-02-26T04:55:28.200264+0000","last_active":"2024-02-26T04:57:21.176445+0000","last_peered":"2024-02-26T04:57:21.176445+0000","last_clean":"2024-02-26T04:57:21.176445+0000","last_became_active":"2024-02-26T04:55:28.199913+0000","last_became_peered":"2024-02-26T04:55:28.199913+0000","last_unstale":"2024-02-26T04:57:21.176445+0000","last_undegraded":"2024-02-26T04:57:21.176445+0000","last_fullsized":"2024-02-26T04:57:21.176445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:54:51.529149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.185839+0000","last_change":"2024-02-26T04:55:32.233362+0000","last_active":"2024-02-26T04:57:21.185839+0000","last_peered":"2024-02-26T04:57:21.185839+0000","last_clean":"2024-02-26T04:57:21.185839+0000","last_became_active":"2024-02-26T04:55:32.232907+0000","last_became_peered":"2024-02-26T04:55:32.232907+0000","last_unstale":"2024-02-26T04:57:21.185839+0000","last_undegraded":"2024-02-26T04:57:21.185839+0000","last_fullsized":"2024-02-26T04:57:21.185839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:49:11.889863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'10","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:40.284901+0000","last_change":"2024-02-26T04:55:27.207371+0000","last_active":"2024-02-26T04:57:40.284901+0000","last_peered":"2024-02-26T04:57:40.284901+0000","last_clean":"2024-02-26T04:57:40.284901+0000","last_became_active":"2024-02-26T04:55:27.207047+0000","last_became_peered":"2024-02-26T04:55:27.207047+0000","last_unstale":"2024-02-26T04:57:40.284901+0000","last_undegraded":"2024-02-26T04:57:40.284901+0000","last_fullsized":"2024-02-26T04:57:40.284901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:29.575633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186642+0000","last_change":"2024-02-26T04:55:28.207132+0000","last_active":"2024-02-26T04:57:21.186642+0000","last_peered":"2024-02-26T04:57:21.186642+0000","last_clean":"2024-02-26T04:57:21.186642+0000","last_became_active":"2024-02-26T04:55:28.206376+0000","last_became_peered":"2024-02-26T04:55:28.206376+0000","last_unstale":"2024-02-26T04:57:21.186642+0000","last_undegraded":"2024-02-26T04:57:21.186642+0000","last_fullsized":"2024-02-26T04:57:21.186642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:17:53.802386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180403+0000","last_change":"2024-02-26T04:55:32.226381+0000","last_active":"2024-02-26T04:57:21.180403+0000","last_peered":"2024-02-26T04:57:21.180403+0000","last_clean":"2024-02-26T04:57:21.180403+0000","last_became_active":"2024-02-26T04:55:32.225940+0000","last_became_peered":"2024-02-26T04:55:32.225940+0000","last_unstale":"2024-02-26T04:57:21.180403+0000","last_undegraded":"2024-02-26T04:57:21.180403+0000","last_fullsized":"2024-02-26T04:57:21.180403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:51:20.190514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177325+0000","last_change":"2024-02-26T04:55:27.200246+0000","last_active":"2024-02-26T04:57:21.177325+0000","last_peered":"2024-02-26T04:57:21.177325+0000","last_clean":"2024-02-26T04:57:21.177325+0000","last_became_active":"2024-02-26T04:55:27.199906+0000","last_became_peered":"2024-02-26T04:55:27.199906+0000","last_unstale":"2024-02-26T04:57:21.177325+0000","last_undegraded":"2024-02-26T04:57:21.177325+0000","last_fullsized":"2024-02-26T04:57:21.177325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:29.037230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391466+0000","last_change":"2024-02-26T04:56:20.204222+0000","last_active":"2024-02-26T04:57:21.391466+0000","last_peered":"2024-02-26T04:57:21.391466+0000","last_clean":"2024-02-26T04:57:21.391466+0000","last_became_active":"2024-02-26T04:56:20.203726+0000","last_became_peered":"2024-02-26T04:56:20.203726+0000","last_unstale":"2024-02-26T04:57:21.391466+0000","last_undegraded":"2024-02-26T04:57:21.391466+0000","last_fullsized":"2024-02-26T04:57:21.391466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:37:59.371734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177174+0000","last_change":"2024-02-26T04:55:32.227765+0000","last_active":"2024-02-26T04:57:21.177174+0000","last_peered":"2024-02-26T04:57:21.177174+0000","last_clean":"2024-02-26T04:57:21.177174+0000","last_became_active":"2024-02-26T04:55:32.227346+0000","last_became_peered":"2024-02-26T04:55:32.227346+0000","last_unstale":"2024-02-26T04:57:21.177174+0000","last_undegraded":"2024-02-26T04:57:21.177174+0000","last_fullsized":"2024-02-26T04:57:21.177174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:05:12.634958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363549+0000","last_change":"2024-02-26T04:55:27.199979+0000","last_active":"2024-02-26T04:57:21.363549+0000","last_peered":"2024-02-26T04:57:21.363549+0000","last_clean":"2024-02-26T04:57:21.363549+0000","last_became_active":"2024-02-26T04:55:27.199618+0000","last_became_peered":"2024-02-26T04:55:27.199618+0000","last_unstale":"2024-02-26T04:57:21.363549+0000","last_undegraded":"2024-02-26T04:57:21.363549+0000","last_fullsized":"2024-02-26T04:57:21.363549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:20:59.649473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180646+0000","last_change":"2024-02-26T04:55:28.202253+0000","last_active":"2024-02-26T04:57:21.180646+0000","last_peered":"2024-02-26T04:57:21.180646+0000","last_clean":"2024-02-26T04:57:21.180646+0000","last_became_active":"2024-02-26T04:55:28.199825+0000","last_became_peered":"2024-02-26T04:55:28.199825+0000","last_unstale":"2024-02-26T04:57:21.180646+0000","last_undegraded":"2024-02-26T04:57:21.180646+0000","last_fullsized":"2024-02-26T04:57:21.180646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:21:59.148995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.188691+0000","last_change":"2024-02-26T04:57:21.188067+0000","last_active":"2024-02-26T04:57:21.188691+0000","last_peered":"2024-02-26T04:57:21.188691+0000","last_clean":"2024-02-26T04:57:21.188691+0000","last_became_active":"2024-02-26T04:57:21.187498+0000","last_became_peered":"2024-02-26T04:57:21.187498+0000","last_unstale":"2024-02-26T04:57:21.188691+0000","last_undegraded":"2024-02-26T04:57:21.188691+0000","last_fullsized":"2024-02-26T04:57:21.188691+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:00:12.060520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186148+0000","last_change":"2024-02-26T04:55:27.206529+0000","last_active":"2024-02-26T04:57:21.186148+0000","last_peered":"2024-02-26T04:57:21.186148+0000","last_clean":"2024-02-26T04:57:21.186148+0000","last_became_active":"2024-02-26T04:55:27.206203+0000","last_became_peered":"2024-02-26T04:55:27.206203+0000","last_unstale":"2024-02-26T04:57:21.186148+0000","last_undegraded":"2024-02-26T04:57:21.186148+0000","last_fullsized":"2024-02-26T04:57:21.186148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:24.491832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184260+0000","last_change":"2024-02-26T04:55:28.205915+0000","last_active":"2024-02-26T04:57:21.184260+0000","last_peered":"2024-02-26T04:57:21.184260+0000","last_clean":"2024-02-26T04:57:21.184260+0000","last_became_active":"2024-02-26T04:55:28.205576+0000","last_became_peered":"2024-02-26T04:55:28.205576+0000","last_unstale":"2024-02-26T04:57:21.184260+0000","last_undegraded":"2024-02-26T04:57:21.184260+0000","last_fullsized":"2024-02-26T04:57:21.184260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:18:54.639778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177053+0000","last_change":"2024-02-26T04:55:32.226253+0000","last_active":"2024-02-26T04:57:21.177053+0000","last_peered":"2024-02-26T04:57:21.177053+0000","last_clean":"2024-02-26T04:57:21.177053+0000","last_became_active":"2024-02-26T04:55:32.225815+0000","last_became_peered":"2024-02-26T04:55:32.225815+0000","last_unstale":"2024-02-26T04:57:21.177053+0000","last_undegraded":"2024-02-26T04:57:21.177053+0000","last_fullsized":"2024-02-26T04:57:21.177053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:22.764756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.196831+0000","last_change":"2024-02-26T04:57:21.196391+0000","last_active":"2024-02-26T04:57:21.196831+0000","last_peered":"2024-02-26T04:57:21.196831+0000","last_clean":"2024-02-26T04:57:21.196831+0000","last_became_active":"2024-02-26T04:57:21.187476+0000","last_became_peered":"2024-02-26T04:57:21.187476+0000","last_unstale":"2024-02-26T04:57:21.196831+0000","last_undegraded":"2024-02-26T04:57:21.196831+0000","last_fullsized":"2024-02-26T04:57:21.196831+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:04:51.219911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186401+0000","last_change":"2024-02-26T04:55:28.213065+0000","last_active":"2024-02-26T04:57:21.186401+0000","last_peered":"2024-02-26T04:57:21.186401+0000","last_clean":"2024-02-26T04:57:21.186401+0000","last_became_active":"2024-02-26T04:55:28.211112+0000","last_became_peered":"2024-02-26T04:55:28.211112+0000","last_unstale":"2024-02-26T04:57:21.186401+0000","last_undegraded":"2024-02-26T04:57:21.186401+0000","last_fullsized":"2024-02-26T04:57:21.186401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:41:10.070183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177186+0000","last_change":"2024-02-26T04:55:32.229838+0000","last_active":"2024-02-26T04:57:21.177186+0000","last_peered":"2024-02-26T04:57:21.177186+0000","last_clean":"2024-02-26T04:57:21.177186+0000","last_became_active":"2024-02-26T04:55:32.228696+0000","last_became_peered":"2024-02-26T04:55:32.228696+0000","last_unstale":"2024-02-26T04:57:21.177186+0000","last_undegraded":"2024-02-26T04:57:21.177186+0000","last_fullsized":"2024-02-26T04:57:21.177186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:59:42.359301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180801+0000","last_change":"2024-02-26T04:55:27.205688+0000","last_active":"2024-02-26T04:57:21.180801+0000","last_peered":"2024-02-26T04:57:21.180801+0000","last_clean":"2024-02-26T04:57:21.180801+0000","last_became_active":"2024-02-26T04:55:27.205306+0000","last_became_peered":"2024-02-26T04:55:27.205306+0000","last_unstale":"2024-02-26T04:57:21.180801+0000","last_undegraded":"2024-02-26T04:57:21.180801+0000","last_fullsized":"2024-02-26T04:57:21.180801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:23:50.203565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180575+0000","last_change":"2024-02-26T04:55:28.210073+0000","last_active":"2024-02-26T04:57:21.180575+0000","last_peered":"2024-02-26T04:57:21.180575+0000","last_clean":"2024-02-26T04:57:21.180575+0000","last_became_active":"2024-02-26T04:55:28.208475+0000","last_became_peered":"2024-02-26T04:55:28.208475+0000","last_unstale":"2024-02-26T04:57:21.180575+0000","last_undegraded":"2024-02-26T04:57:21.180575+0000","last_fullsized":"2024-02-26T04:57:21.180575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:06:49.513916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363553+0000","last_change":"2024-02-26T04:55:32.228708+0000","last_active":"2024-02-26T04:57:21.363553+0000","last_peered":"2024-02-26T04:57:21.363553+0000","last_clean":"2024-02-26T04:57:21.363553+0000","last_became_active":"2024-02-26T04:55:32.228304+0000","last_became_peered":"2024-02-26T04:55:32.228304+0000","last_unstale":"2024-02-26T04:57:21.363553+0000","last_undegraded":"2024-02-26T04:57:21.363553+0000","last_fullsized":"2024-02-26T04:57:21.363553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:53:02.390712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181305+0000","last_change":"2024-02-26T04:55:27.199761+0000","last_active":"2024-02-26T04:57:21.181305+0000","last_peered":"2024-02-26T04:57:21.181305+0000","last_clean":"2024-02-26T04:57:21.181305+0000","last_became_active":"2024-02-26T04:55:27.199113+0000","last_became_peered":"2024-02-26T04:55:27.199113+0000","last_unstale":"2024-02-26T04:57:21.181305+0000","last_undegraded":"2024-02-26T04:57:21.181305+0000","last_fullsized":"2024-02-26T04:57:21.181305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:14:10.631886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180844+0000","last_change":"2024-02-26T04:55:28.211484+0000","last_active":"2024-02-26T04:57:21.180844+0000","last_peered":"2024-02-26T04:57:21.180844+0000","last_clean":"2024-02-26T04:57:21.180844+0000","last_became_active":"2024-02-26T04:55:28.211131+0000","last_became_peered":"2024-02-26T04:55:28.211131+0000","last_unstale":"2024-02-26T04:57:21.180844+0000","last_undegraded":"2024-02-26T04:57:21.180844+0000","last_fullsized":"2024-02-26T04:57:21.180844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:54:09.393847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391396+0000","last_change":"2024-02-26T04:55:32.227493+0000","last_active":"2024-02-26T04:57:21.391396+0000","last_peered":"2024-02-26T04:57:21.391396+0000","last_clean":"2024-02-26T04:57:21.391396+0000","last_became_active":"2024-02-26T04:55:32.227106+0000","last_became_peered":"2024-02-26T04:55:32.227106+0000","last_unstale":"2024-02-26T04:57:21.391396+0000","last_undegraded":"2024-02-26T04:57:21.391396+0000","last_fullsized":"2024-02-26T04:57:21.391396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:33.040966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178567+0000","last_change":"2024-02-26T04:55:27.205331+0000","last_active":"2024-02-26T04:57:21.178567+0000","last_peered":"2024-02-26T04:57:21.178567+0000","last_clean":"2024-02-26T04:57:21.178567+0000","last_became_active":"2024-02-26T04:55:27.204998+0000","last_became_peered":"2024-02-26T04:55:27.204998+0000","last_unstale":"2024-02-26T04:57:21.178567+0000","last_undegraded":"2024-02-26T04:57:21.178567+0000","last_fullsized":"2024-02-26T04:57:21.178567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:10:03.671016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180697+0000","last_change":"2024-02-26T04:56:20.508854+0000","last_active":"2024-02-26T04:57:21.180697+0000","last_peered":"2024-02-26T04:57:21.180697+0000","last_clean":"2024-02-26T04:57:21.180697+0000","last_became_active":"2024-02-26T04:56:20.508396+0000","last_became_peered":"2024-02-26T04:56:20.508396+0000","last_unstale":"2024-02-26T04:57:21.180697+0000","last_undegraded":"2024-02-26T04:57:21.180697+0000","last_fullsized":"2024-02-26T04:57:21.180697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:47:33.249035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177528+0000","last_change":"2024-02-26T04:55:32.232222+0000","last_active":"2024-02-26T04:57:21.177528+0000","last_peered":"2024-02-26T04:57:21.177528+0000","last_clean":"2024-02-26T04:57:21.177528+0000","last_became_active":"2024-02-26T04:55:32.231836+0000","last_became_peered":"2024-02-26T04:55:32.231836+0000","last_unstale":"2024-02-26T04:57:21.177528+0000","last_undegraded":"2024-02-26T04:57:21.177528+0000","last_fullsized":"2024-02-26T04:57:21.177528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:18:00.766266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176570+0000","last_change":"2024-02-26T04:55:27.200385+0000","last_active":"2024-02-26T04:57:21.176570+0000","last_peered":"2024-02-26T04:57:21.176570+0000","last_clean":"2024-02-26T04:57:21.176570+0000","last_became_active":"2024-02-26T04:55:27.200046+0000","last_became_peered":"2024-02-26T04:55:27.200046+0000","last_unstale":"2024-02-26T04:57:21.176570+0000","last_undegraded":"2024-02-26T04:57:21.176570+0000","last_fullsized":"2024-02-26T04:57:21.176570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:01:58.112457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181168+0000","last_change":"2024-02-26T04:55:28.220702+0000","last_active":"2024-02-26T04:57:21.181168+0000","last_peered":"2024-02-26T04:57:21.181168+0000","last_clean":"2024-02-26T04:57:21.181168+0000","last_became_active":"2024-02-26T04:55:28.220386+0000","last_became_peered":"2024-02-26T04:55:28.220386+0000","last_unstale":"2024-02-26T04:57:21.181168+0000","last_undegraded":"2024-02-26T04:57:21.181168+0000","last_fullsized":"2024-02-26T04:57:21.181168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:40:37.774831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183841+0000","last_change":"2024-02-26T04:55:28.206409+0000","last_active":"2024-02-26T04:57:21.183841+0000","last_peered":"2024-02-26T04:57:21.183841+0000","last_clean":"2024-02-26T04:57:21.183841+0000","last_became_active":"2024-02-26T04:55:28.206046+0000","last_became_peered":"2024-02-26T04:55:28.206046+0000","last_unstale":"2024-02-26T04:57:21.183841+0000","last_undegraded":"2024-02-26T04:57:21.183841+0000","last_fullsized":"2024-02-26T04:57:21.183841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:35:43.277376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181921+0000","last_change":"2024-02-26T04:55:28.216146+0000","last_active":"2024-02-26T04:57:21.181921+0000","last_peered":"2024-02-26T04:57:21.181921+0000","last_clean":"2024-02-26T04:57:21.181921+0000","last_became_active":"2024-02-26T04:55:28.215800+0000","last_became_peered":"2024-02-26T04:55:28.215800+0000","last_unstale":"2024-02-26T04:57:21.181921+0000","last_undegraded":"2024-02-26T04:57:21.181921+0000","last_fullsized":"2024-02-26T04:57:21.181921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:18:47.902694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180868+0000","last_change":"2024-02-26T04:55:28.221587+0000","last_active":"2024-02-26T04:57:21.180868+0000","last_peered":"2024-02-26T04:57:21.180868+0000","last_clean":"2024-02-26T04:57:21.180868+0000","last_became_active":"2024-02-26T04:55:28.220990+0000","last_became_peered":"2024-02-26T04:55:28.220990+0000","last_unstale":"2024-02-26T04:57:21.180868+0000","last_undegraded":"2024-02-26T04:57:21.180868+0000","last_fullsized":"2024-02-26T04:57:21.180868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:24:13.302081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391622+0000","last_change":"2024-02-26T04:55:28.220586+0000","last_active":"2024-02-26T04:57:21.391622+0000","last_peered":"2024-02-26T04:57:21.391622+0000","last_clean":"2024-02-26T04:57:21.391622+0000","last_became_active":"2024-02-26T04:55:28.219585+0000","last_became_peered":"2024-02-26T04:55:28.219585+0000","last_unstale":"2024-02-26T04:57:21.391622+0000","last_undegraded":"2024-02-26T04:57:21.391622+0000","last_fullsized":"2024-02-26T04:57:21.391622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:41.786142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176530+0000","last_change":"2024-02-26T04:55:28.204599+0000","last_active":"2024-02-26T04:57:21.176530+0000","last_peered":"2024-02-26T04:57:21.176530+0000","last_clean":"2024-02-26T04:57:21.176530+0000","last_became_active":"2024-02-26T04:55:28.204252+0000","last_became_peered":"2024-02-26T04:55:28.204252+0000","last_unstale":"2024-02-26T04:57:21.176530+0000","last_undegraded":"2024-02-26T04:57:21.176530+0000","last_fullsized":"2024-02-26T04:57:21.176530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:30:19.807897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183609+0000","last_change":"2024-02-26T04:55:28.214233+0000","last_active":"2024-02-26T04:57:21.183609+0000","last_peered":"2024-02-26T04:57:21.183609+0000","last_clean":"2024-02-26T04:57:21.183609+0000","last_became_active":"2024-02-26T04:55:28.213927+0000","last_became_peered":"2024-02-26T04:55:28.213927+0000","last_unstale":"2024-02-26T04:57:21.183609+0000","last_undegraded":"2024-02-26T04:57:21.183609+0000","last_fullsized":"2024-02-26T04:57:21.183609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:31:07.967134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181680+0000","last_change":"2024-02-26T04:55:28.217610+0000","last_active":"2024-02-26T04:57:21.181680+0000","last_peered":"2024-02-26T04:57:21.181680+0000","last_clean":"2024-02-26T04:57:21.181680+0000","last_became_active":"2024-02-26T04:55:28.217263+0000","last_became_peered":"2024-02-26T04:55:28.217263+0000","last_unstale":"2024-02-26T04:57:21.181680+0000","last_undegraded":"2024-02-26T04:57:21.181680+0000","last_fullsized":"2024-02-26T04:57:21.181680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:28:18.831887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179617+0000","last_change":"2024-02-26T04:56:20.510859+0000","last_active":"2024-02-26T04:57:21.179617+0000","last_peered":"2024-02-26T04:57:21.179617+0000","last_clean":"2024-02-26T04:57:21.179617+0000","last_became_active":"2024-02-26T04:56:20.510420+0000","last_became_peered":"2024-02-26T04:56:20.510420+0000","last_unstale":"2024-02-26T04:57:21.179617+0000","last_undegraded":"2024-02-26T04:57:21.179617+0000","last_fullsized":"2024-02-26T04:57:21.179617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:02.961996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177656+0000","last_change":"2024-02-26T04:55:28.217247+0000","last_active":"2024-02-26T04:57:21.177656+0000","last_peered":"2024-02-26T04:57:21.177656+0000","last_clean":"2024-02-26T04:57:21.177656+0000","last_became_active":"2024-02-26T04:55:28.214691+0000","last_became_peered":"2024-02-26T04:55:28.214691+0000","last_unstale":"2024-02-26T04:57:21.177656+0000","last_undegraded":"2024-02-26T04:57:21.177656+0000","last_fullsized":"2024-02-26T04:57:21.177656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:27:45.564520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186211+0000","last_change":"2024-02-26T04:55:28.210491+0000","last_active":"2024-02-26T04:57:21.186211+0000","last_peered":"2024-02-26T04:57:21.186211+0000","last_clean":"2024-02-26T04:57:21.186211+0000","last_became_active":"2024-02-26T04:55:28.210144+0000","last_became_peered":"2024-02-26T04:55:28.210144+0000","last_unstale":"2024-02-26T04:57:21.186211+0000","last_undegraded":"2024-02-26T04:57:21.186211+0000","last_fullsized":"2024-02-26T04:57:21.186211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:11.214716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363822+0000","last_change":"2024-02-26T04:55:28.216037+0000","last_active":"2024-02-26T04:57:21.363822+0000","last_peered":"2024-02-26T04:57:21.363822+0000","last_clean":"2024-02-26T04:57:21.363822+0000","last_became_active":"2024-02-26T04:55:28.215712+0000","last_became_peered":"2024-02-26T04:55:28.215712+0000","last_unstale":"2024-02-26T04:57:21.363822+0000","last_undegraded":"2024-02-26T04:57:21.363822+0000","last_fullsized":"2024-02-26T04:57:21.363822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:38:55.437642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176653+0000","last_change":"2024-02-26T04:55:28.205469+0000","last_active":"2024-02-26T04:57:21.176653+0000","last_peered":"2024-02-26T04:57:21.176653+0000","last_clean":"2024-02-26T04:57:21.176653+0000","last_became_active":"2024-02-26T04:55:28.205137+0000","last_became_peered":"2024-02-26T04:55:28.205137+0000","last_unstale":"2024-02-26T04:57:21.176653+0000","last_undegraded":"2024-02-26T04:57:21.176653+0000","last_fullsized":"2024-02-26T04:57:21.176653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:50.425010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.182926+0000","last_change":"2024-02-26T04:55:28.213066+0000","last_active":"2024-02-26T04:57:21.182926+0000","last_peered":"2024-02-26T04:57:21.182926+0000","last_clean":"2024-02-26T04:57:21.182926+0000","last_became_active":"2024-02-26T04:55:28.212706+0000","last_became_peered":"2024-02-26T04:55:28.212706+0000","last_unstale":"2024-02-26T04:57:21.182926+0000","last_undegraded":"2024-02-26T04:57:21.182926+0000","last_fullsized":"2024-02-26T04:57:21.182926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:21:24.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281347,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73204,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":755,"num_read_kb":453,"num_write":172,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":582,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":219612,"data_compressed":4869,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":520717,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.43099999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68100000000000005}]},{"osd":1,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.51500000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.45900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.56799999999999995}]},{"osd":2,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.33600000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.504,"15min":0.5},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":4,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.67800000000000005,"15min":0.67500000000000004},"min":{"1min":0.443,"5min":0.442,"15min":0.442},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.48299999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77000000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.41499999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84799999999999998,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.623},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91400000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59099999999999997}]},{"osd":6,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.56000000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1559999999999999,"5min":1.175,"15min":1.175},"last":0.72399999999999998}]},{"osd":7,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.74399999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.75,"15min":0.748},"min":{"1min":0.59299999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.88400000000000001}]},{"osd":8,"last update":"Mon Feb 26 04:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72299999999999998,"15min":0.71799999999999997},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.73899999999999999,"15min":0.73699999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.90000000000000002}]},{"osd":9,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.79900000000000004,"15min":0.79300000000000004},"min":{"1min":0.68100000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0369999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.80400000000000005,"15min":0.79800000000000004},"min":{"1min":0.622,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0960000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.81699999999999995}]},{"osd":10,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.63500000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.63200000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.86599999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":1176997,"data_compressed":63914,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.91800000000000004,"15min":0.98299999999999998},"min":{"1min":0.37,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.749,"5min":9.6609999999999996,"15min":9.6609999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.96599999999999997,"15min":1.0269999999999999},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90500000000000003,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":0.41899999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.499},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70799999999999996,"5min":0.871,"15min":0.871},"last":0.44700000000000001}]},{"osd":2,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.435,"5min":0.309,"15min":0.309},"max":{"1min":0.73199999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81699999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58899999999999997}]},{"osd":3,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.69999999999999996,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":8.3840000000000003,"5min":8.3840000000000003,"15min":8.3840000000000003},"last":0.501},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.67500000000000004,"15min":0.63100000000000001},"min":{"1min":0.38600000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":8.4160000000000004,"5min":8.4160000000000004,"15min":8.4160000000000004},"last":0.63200000000000001}]},{"osd":4,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56100000000000005,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.77300000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56100000000000005,"15min":0.57299999999999995},"min":{"1min":0.35499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80300000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.75}]},{"osd":5,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68200000000000005,"15min":0.68700000000000006},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.87,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.52900000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.88400000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61799999999999999}]},{"osd":6,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":0.79900000000000004,"15min":0.753},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":9.7119999999999997,"5min":9.7119999999999997,"15min":9.7119999999999997},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.54400000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89800000000000002,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.72699999999999998}]},{"osd":8,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.751,"15min":0.76500000000000001},"min":{"1min":0.52900000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93500000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.79200000000000004,"15min":0.81399999999999995},"min":{"1min":0.52800000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94699999999999995,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.80000000000000004}]},{"osd":9,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":0.84299999999999997,"15min":0.80400000000000005},"min":{"1min":0.57099999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":8.3439999999999994,"5min":8.3439999999999994,"15min":8.3439999999999994},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81000000000000005,"15min":0.83299999999999996},"min":{"1min":0.55500000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97199999999999998,"5min":1.274,"15min":1.274},"last":0.83099999999999996}]},{"osd":11,"last update":"Mon Feb 26 04:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.63300000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91500000000000004,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78500000000000003,"15min":0.78800000000000003},"min":{"1min":0.58399999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88800000000000001,"5min":1.21,"15min":1.21},"last":0.76900000000000002}]}]},{"osd":9,"up_from":67,"seq":287762808886,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":524467,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42099999999999999,"15min":0.46700000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.51400000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.44600000000000001,"15min":0.48799999999999999},"min":{"1min":0.32600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65800000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.34899999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59099999999999997,"15min":0.623},"min":{"1min":0.433,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.59699999999999998,"15min":0.63800000000000001},"min":{"1min":0.36799999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77400000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63,"15min":0.66400000000000003},"min":{"1min":0.41599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.86399999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.68300000000000005,"15min":0.77700000000000002},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78200000000000003,"5min":3.173,"15min":3.173},"last":0.56000000000000005}]},{"osd":3,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47799999999999998,"15min":0.51600000000000001},"min":{"1min":0.31,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.442},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.48099999999999998,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.747,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.37}]},{"osd":4,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.67600000000000005},"min":{"1min":0.39500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78300000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.629,"15min":0.66200000000000003},"min":{"1min":0.45900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84099999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.41499999999999998}]},{"osd":5,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":0.79200000000000004,"15min":0.74099999999999999},"min":{"1min":0.46700000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":11.49,"5min":11.49,"15min":11.49},"last":0.49199999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":0.79000000000000004,"15min":0.751},"min":{"1min":0.41299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":11.470000000000001,"5min":11.470000000000001,"15min":11.470000000000001},"last":0.57599999999999996}]},{"osd":6,"last update":"Mon Feb 26 04:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69399999999999995,"15min":0.748},"min":{"1min":0.51800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92100000000000004,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.70199999999999996,"15min":0.76100000000000001},"min":{"1min":0.441,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83599999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.50900000000000001}]},{"osd":8,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.79600000000000004,"15min":0.90200000000000002},"min":{"1min":0.501,"5min":0.434,"15min":0.434},"max":{"1min":0.90200000000000002,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.79600000000000004},"min":{"1min":0.52300000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96299999999999997,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.58899999999999997}]},{"osd":10,"last update":"Mon Feb 26 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.74199999999999999,"15min":0.75800000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.89200000000000002,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73999999999999999,"15min":0.752},"min":{"1min":0.58699999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.872,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.54100000000000004}]},{"osd":11,"last update":"Mon Feb 26 04:57:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.73999999999999999,"15min":0.73399999999999999},"min":{"1min":0.628,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.61099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.61199999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070523,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":520291,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.49099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.84799999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.37},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.66600000000000004,"15min":0.57399999999999995},"min":{"1min":0.44900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.81699999999999995,"5min":12.359999999999999,"15min":12.359999999999999},"last":0.39800000000000002}]},{"osd":1,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78200000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.66000000000000003,"15min":0.626},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.87,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.502}]},{"osd":2,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.46800000000000003,"15min":0.44},"min":{"1min":0.42699999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.48699999999999999,"15min":0.44800000000000001},"min":{"1min":0.41099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73599999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55200000000000005}]},{"osd":3,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58699999999999997,"15min":0.56000000000000005},"min":{"1min":0.48499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.499,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76700000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.48299999999999998}]},{"osd":4,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64300000000000002,"15min":0.60899999999999999},"min":{"1min":0.45100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88100000000000001,"5min":1.2,"15min":1.2},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64400000000000002,"15min":0.621},"min":{"1min":0.51900000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89500000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.52400000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.59699999999999998,"15min":0.54900000000000004},"min":{"1min":0.55100000000000005,"5min":0.309,"15min":0.309},"max":{"1min":0.92500000000000004,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.59399999999999997,"15min":0.55200000000000005},"min":{"1min":0.47099999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.91300000000000003,"5min":1.27,"15min":1.27},"last":0.69199999999999995}]},{"osd":6,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63900000000000001,"15min":0.61499999999999999},"min":{"1min":0.53900000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.84299999999999997,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.84299999999999997,"15min":0.73099999999999998},"min":{"1min":0.66700000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97499999999999998,"5min":12.384,"15min":12.384},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.752,"15min":0.71399999999999997},"min":{"1min":0.72999999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.73999999999999999,"15min":0.68799999999999994},"min":{"1min":0.63800000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.97399999999999998,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.73299999999999998}]},{"osd":9,"last update":"Mon Feb 26 04:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.76600000000000001,"15min":0.71899999999999997},"min":{"1min":0.65100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.996,"5min":1.073,"15min":1.073},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.746,"15min":0.69299999999999995},"min":{"1min":0.65500000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94299999999999995,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.65500000000000003}]},{"osd":10,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.78000000000000003,"15min":0.748},"min":{"1min":0.67900000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0229999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.80700000000000005,"15min":0.77500000000000002},"min":{"1min":0.71499999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.84099999999999997}]},{"osd":11,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.72999999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0660000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.879},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.85299999999999998},"min":{"1min":0.66100000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.995,"5min":1.238,"15min":1.238},"last":0.82299999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1081344,"data_stored":1115535,"data_compressed":62291,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67000000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.40799999999999997}]},{"osd":1,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42399999999999999,"15min":0.41799999999999998},"min":{"1min":0.27900000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45700000000000002,"15min":0.443},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.58999999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.55000000000000004}]},{"osd":2,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82299999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84599999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72599999999999998}]},{"osd":3,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.49299999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79600000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81599999999999995,"5min":0.877,"15min":0.877},"last":0.69999999999999996}]},{"osd":4,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.30499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.753,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.58699999999999997}]},{"osd":5,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.46500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86499999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66500000000000004,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.79200000000000004}]},{"osd":6,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.46200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.56200000000000006,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85499999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.83799999999999997}]},{"osd":8,"last update":"Mon Feb 26 04:57:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.53700000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93000000000000005,"5min":1.044,"15min":1.044},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72099999999999997,"15min":0.71399999999999997},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.89900000000000002,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.89700000000000002}]},{"osd":9,"last update":"Mon Feb 26 04:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.78000000000000003,"15min":0.74299999999999999},"min":{"1min":0.57699999999999996,"5min":0.504,"15min":0.504},"max":{"1min":4.7160000000000002,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.71899999999999997},"min":{"1min":0.59899999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.93100000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.93100000000000005}]},{"osd":10,"last update":"Mon Feb 26 04:57:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.747,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.48899999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.995,"5min":1.119,"15min":1.119},"last":0.86899999999999999}]},{"osd":11,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79600000000000004,"15min":0.79400000000000004},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95399999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.65300000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96799999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.88700000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":515547,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42799999999999999,"15min":0.39200000000000002},"min":{"1min":0.29299999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.434},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.433}]},{"osd":1,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64200000000000002,"15min":0.62},"min":{"1min":0.497,"5min":0.376,"15min":0.376},"max":{"1min":0.878,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.498},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.621,"15min":0.57899999999999996},"min":{"1min":0.40699999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85899999999999999,"5min":1.032,"15min":1.032},"last":0.59999999999999998}]},{"osd":2,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.627,"15min":0.58099999999999996},"min":{"1min":0.51800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.95399999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.751,"15min":1.0209999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94299999999999995,"5min":14.884,"15min":14.884},"last":0.57099999999999995}]},{"osd":3,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.48099999999999998,"15min":0.45600000000000002},"min":{"1min":0.33200000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71099999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54200000000000004,"15min":0.51800000000000002},"min":{"1min":0.38200000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.83199999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55300000000000005}]},{"osd":4,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.63700000000000001},"min":{"1min":0.441,"5min":0.38,"15min":0.38},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.79400000000000004,"15min":1.0720000000000001},"min":{"1min":0.45600000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.97699999999999998,"5min":14.898,"15min":14.898},"last":0.53500000000000003}]},{"osd":5,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68100000000000005,"15min":0.64700000000000002},"min":{"1min":0.50800000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90100000000000002,"5min":1.288,"15min":1.288},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70899999999999996,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0009999999999999,"5min":1.151,"15min":1.151},"last":0.61599999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76500000000000001,"15min":0.74199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71399999999999997,"15min":0.69199999999999995},"min":{"1min":0.501,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.044,"5min":1.075,"15min":1.075},"last":0.63600000000000001}]},{"osd":8,"last update":"Mon Feb 26 04:57:26 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":0.90200000000000002,"15min":0.81599999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":12.880000000000001,"5min":12.880000000000001,"15min":12.880000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.4570000000000001,"5min":0.90400000000000003,"15min":0.81200000000000006},"min":{"1min":0.58499999999999996,"5min":0.432,"15min":0.432},"max":{"1min":12.863,"5min":12.863,"15min":12.863},"last":0.69899999999999995}]},{"osd":9,"last update":"Mon Feb 26 04:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.745,"15min":0.79100000000000004},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96299999999999997,"5min":1.111,"15min":1.111},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.78500000000000003},"min":{"1min":0.437,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.008,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.72499999999999998}]},{"osd":10,"last update":"Mon Feb 26 04:57:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.84799999999999998,"15min":0.86899999999999999},"min":{"1min":0.622,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1200000000000001,"5min":1.133,"15min":1.133},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.58999999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0700000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.74299999999999999}]},{"osd":11,"last update":"Mon Feb 26 04:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81100000000000005,"15min":0.80900000000000005},"min":{"1min":0.68799999999999994,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.099,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83099999999999996,"15min":0.83399999999999996},"min":{"1min":0.67600000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1399999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.755}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":584743,"data_compressed":28131,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55400000000000005,"15min":0.502},"min":{"1min":0.36099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.625,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.54500000000000004,"15min":0.505},"min":{"1min":0.35799999999999998,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.70499999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.34899999999999998}]},{"osd":2,"last update":"Mon Feb 26 04:57:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.64400000000000002,"15min":0.55900000000000005},"min":{"1min":0.45900000000000002,"5min":0.371,"15min":0.35199999999999998},"max":{"1min":0.91600000000000004,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.55100000000000005},"min":{"1min":0.52100000000000002,"5min":0.35699999999999998,"15min":0.35599999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.56699999999999995}]},{"osd":3,"last update":"Mon Feb 26 04:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.55200000000000005},"min":{"1min":0.41199999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69999999999999996,"5min":0.997,"15min":0.997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63800000000000001,"15min":0.57899999999999996},"min":{"1min":0.45300000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73199999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.51600000000000001}]},{"osd":4,"last update":"Mon Feb 26 04:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.42799999999999999},"min":{"1min":0.34100000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.48699999999999999,"15min":0.435},"min":{"1min":0.36699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46600000000000003}]},{"osd":5,"last update":"Mon Feb 26 04:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64400000000000002,"15min":0.57199999999999995},"min":{"1min":0.44900000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85599999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63,"15min":0.56100000000000005},"min":{"1min":0.42799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97499999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.75700000000000001}]},{"osd":6,"last update":"Mon Feb 26 04:57:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.49099999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66500000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.71699999999999997,"15min":0.66600000000000004},"min":{"1min":0.60499999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.79800000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.55200000000000005},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88300000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75600000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54400000000000004}]},{"osd":8,"last update":"Mon Feb 26 04:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.748,"15min":0.71899999999999997},"min":{"1min":0.53600000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.90800000000000003,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.74199999999999999,"15min":0.71799999999999997},"min":{"1min":0.58399999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":1.04,"15min":1.04},"last":0.67900000000000005}]},{"osd":9,"last update":"Mon Feb 26 04:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1080000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.81100000000000005}]},{"osd":10,"last update":"Mon Feb 26 04:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.79400000000000004,"15min":0.79500000000000004},"min":{"1min":0.57099999999999995,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96499999999999997,"5min":1.095,"15min":1.095},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92300000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.72099999999999997}]},{"osd":11,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.89100000000000001,"15min":0.90300000000000002},"min":{"1min":0.625,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.93899999999999995,"5min":1.141,"15min":1.141},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.83999999999999997,"15min":0.84899999999999998},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.79800000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":516935,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 04:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.51300000000000001},"min":{"1min":0.38400000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83099999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.88700000000000001,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Mon Feb 26 04:56:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56899999999999995,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.33500000000000002,"15min":0.28999999999999998},"max":{"1min":0.81899999999999995,"5min":1.006,"15min":1.006},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60899999999999999,"15min":0.55700000000000005},"min":{"1min":0.45000000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.83799999999999997}]},{"osd":3,"last update":"Mon Feb 26 04:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.45700000000000002,"15min":0.46300000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.57799999999999996,"5min":1.208,"15min":1.208},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Feb 26 04:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65900000000000003,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.98399999999999999,"5min":1.073,"15min":1.073},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63100000000000001,"15min":0.56299999999999994},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.65200000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60999999999999999,"15min":0.56200000000000006},"min":{"1min":0.42399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85199999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66600000000000004,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.89200000000000002}]},{"osd":6,"last update":"Mon Feb 26 04:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52500000000000002,"15min":0.51600000000000001},"min":{"1min":0.34100000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.754,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.54400000000000004},"min":{"1min":0.38,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80400000000000005,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Feb 26 04:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.54600000000000004,"5min":0.374,"15min":0.374},"max":{"1min":0.87,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.749,"15min":0.751},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93600000000000005,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.69599999999999995}]},{"osd":8,"last update":"Mon Feb 26 04:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.44900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96799999999999997,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73199999999999998,"15min":0.75},"min":{"1min":0.48299999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96599999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.78800000000000003}]},{"osd":9,"last update":"Mon Feb 26 04:56:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.46500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.80100000000000005},"min":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.51800000000000002}]},{"osd":10,"last update":"Mon Feb 26 04:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.83099999999999996,"15min":0.84099999999999997},"min":{"1min":0.61599999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0609999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.61799999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.036,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.73699999999999999}]},{"osd":11,"last update":"Mon Feb 26 04:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.65100000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.96899999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.75},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78400000000000003,"15min":0.78300000000000003},"min":{"1min":0.64800000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0349999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.82199999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":586093,"data_compressed":28131,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.53900000000000003,"15min":0.47299999999999998},"min":{"1min":0.41599999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.96399999999999997,"5min":1.002,"15min":1.002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53000000000000003,"15min":0.47399999999999998},"min":{"1min":0.378,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64700000000000002}]},{"osd":1,"last update":"Mon Feb 26 04:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.50600000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.57899999999999996,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.75900000000000001}]},{"osd":3,"last update":"Mon Feb 26 04:57:27 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.623,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.32000000000000001},"max":{"1min":0.93799999999999994,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.873},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.64300000000000002,"15min":0.58499999999999996},"min":{"1min":0.49299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94599999999999995,"5min":1.004,"15min":1.004},"last":0.94399999999999995}]},{"osd":4,"last update":"Mon Feb 26 04:57:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.75600000000000001,"15min":0.77000000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.85799999999999998,"5min":1.002,"15min":1.002},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.875,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.0940000000000001}]},{"osd":5,"last update":"Mon Feb 26 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.35199999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.48299999999999998,"15min":0.45100000000000001},"min":{"1min":0.35599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.71099999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.52400000000000002}]},{"osd":6,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63700000000000001,"15min":0.56899999999999995},"min":{"1min":0.51100000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.877,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.64000000000000001,"15min":0.57699999999999996},"min":{"1min":0.46899999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91400000000000003,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75900000000000001,"15min":0.77000000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91400000000000003,"5min":1.038,"15min":1.038},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.78700000000000003,"15min":0.80300000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":1.014,"15min":1.014},"last":1.044}]},{"osd":8,"last update":"Mon Feb 26 04:57:27 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.871,"5min":1.115,"15min":1.115},"last":0.621},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.35099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.88700000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.78200000000000003}]},{"osd":9,"last update":"Mon Feb 26 04:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70599999999999996,"15min":0.67600000000000005},"min":{"1min":0.52900000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.95899999999999996,"5min":1.018,"15min":1.018},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.748,"15min":0.72199999999999998},"min":{"1min":0.54100000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.96599999999999997,"5min":1.056,"15min":1.056},"last":0.82699999999999996}]},{"osd":10,"last update":"Mon Feb 26 04:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.70099999999999996},"min":{"1min":0.55600000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":1.127},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72999999999999998,"15min":0.69699999999999995},"min":{"1min":0.48199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96499999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.92700000000000005}]},{"osd":11,"last update":"Mon Feb 26 04:57:27 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.82499999999999996,"15min":0.83099999999999996},"min":{"1min":0.53400000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.008,"5min":1.109,"15min":1.109},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.80700000000000005,"15min":0.81200000000000006},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.89700000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":515741,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.34000000000000002},"min":{"1min":0.309,"5min":0.21199999999999999,"15min":0.17899999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.315},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.33800000000000002},"min":{"1min":0.28100000000000003,"5min":0.22700000000000001,"15min":0.20699999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.28999999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.52700000000000002},"min":{"1min":0.38,"5min":0.36599999999999999,"15min":0.35299999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57099999999999995,"15min":0.53100000000000003},"min":{"1min":0.40100000000000002,"5min":0.34499999999999997,"15min":0.32600000000000001},"max":{"1min":0.76200000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.61599999999999999}]},{"osd":2,"last update":"Mon Feb 26 04:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.42799999999999999,"5min":0.36699999999999999,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60199999999999998,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.32700000000000001},"max":{"1min":0.76300000000000001,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.59299999999999997}]},{"osd":4,"last update":"Mon Feb 26 04:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.48499999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82699999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64900000000000002,"15min":0.623},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.56100000000000005}]},{"osd":5,"last update":"Mon Feb 26 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.56899999999999995},"min":{"1min":0.40699999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.69499999999999995}]},{"osd":6,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50800000000000001,"15min":0.47999999999999998},"min":{"1min":0.33600000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.93899999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54100000000000004,"15min":0.51400000000000001},"min":{"1min":0.40300000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85599999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.36499999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66600000000000004,"15min":0.629},"min":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95999999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.629}]},{"osd":8,"last update":"Mon Feb 26 04:55:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.874,"5min":0.995,"15min":0.995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.878,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63100000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90500000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.44700000000000001}]},{"osd":10,"last update":"Mon Feb 26 04:57:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78000000000000003,"15min":0.79100000000000004},"min":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.038,"5min":1.232,"15min":1.232},"last":0.65700000000000003}]},{"osd":11,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.57899999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.751,"15min":0.75},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.67700000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":520505,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:54:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.51100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79500000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42899999999999999}]},{"osd":1,"last update":"Mon Feb 26 04:57:42 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.436,"15min":0.378},"min":{"1min":0.33600000000000002,"5min":0.26300000000000001,"15min":0.24299999999999999},"max":{"1min":0.58399999999999996,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.438,"15min":0.39200000000000002},"min":{"1min":0.36399999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.54200000000000004,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.621}]},{"osd":2,"last update":"Mon Feb 26 04:57:42 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58399999999999996,"15min":0.53400000000000003},"min":{"1min":0.442,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82199999999999995,"5min":1.657,"15min":1.657},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.60399999999999998,"15min":0.55200000000000005},"min":{"1min":0.496,"5min":0.40899999999999997,"15min":0.38},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.73399999999999999}]},{"osd":3,"last update":"Mon Feb 26 04:57:42 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61899999999999999,"15min":0.56200000000000006},"min":{"1min":0.41599999999999998,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.70499999999999996,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60499999999999998,"15min":0.54700000000000004},"min":{"1min":0.46300000000000002,"5min":0.36699999999999999,"15min":0.35599999999999998},"max":{"1min":0.81599999999999995,"5min":2.133,"15min":2.133},"last":0.59099999999999997}]},{"osd":5,"last update":"Mon Feb 26 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59299999999999997,"15min":0.55400000000000005},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89000000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.748},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63,"15min":0.56399999999999995},"min":{"1min":0.41799999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82199999999999995,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.84599999999999997}]},{"osd":6,"last update":"Mon Feb 26 04:57:40 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64000000000000001,"15min":0.58799999999999997},"min":{"1min":0.441,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75700000000000001,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.65200000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84499999999999997,"5min":4.1639999999999997,"15min":4.1639999999999997},"last":0.52100000000000002}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.55300000000000005},"min":{"1min":0.34200000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.55500000000000005},"min":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":4.2300000000000004,"15min":4.2300000000000004},"last":0.66400000000000003}]},{"osd":8,"last update":"Mon Feb 26 04:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71299999999999997,"15min":0.68600000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92700000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68500000000000005,"15min":0.64500000000000002},"min":{"1min":0.54400000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93200000000000005,"5min":1.907,"15min":1.907},"last":0.71899999999999997}]},{"osd":9,"last update":"Mon Feb 26 04:57:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.82899999999999996,"15min":0.749},"min":{"1min":0.55800000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.84199999999999997,"5min":10.051,"15min":10.051},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.82899999999999996,"15min":0.76500000000000001},"min":{"1min":0.58099999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94499999999999995,"5min":6.3360000000000003,"15min":6.3360000000000003},"last":0.76400000000000001}]},{"osd":10,"last update":"Mon Feb 26 04:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.78200000000000003,"15min":0.77800000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94799999999999995,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95499999999999996,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.70299999999999996}]},{"osd":11,"last update":"Mon Feb 26 04:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.97099999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82099999999999995,"15min":0.82899999999999996},"min":{"1min":0.59099999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.95899999999999996,"5min":1.946,"15min":1.946},"last":0.79900000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757322,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":1111397,"data_compressed":62291,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51300000000000001,"15min":0.49099999999999999},"min":{"1min":0.39600000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.32900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.60099999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77400000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58599999999999997,"15min":0.53900000000000003},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85799999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61499999999999999}]},{"osd":2,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.41299999999999998,"15min":0.36899999999999999},"min":{"1min":0.30099999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.72199999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.42099999999999999,"15min":0.38700000000000001},"min":{"1min":0.32800000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.54800000000000004,"15min":0.52000000000000002},"min":{"1min":0.41399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.56899999999999995,"15min":0.51300000000000001},"min":{"1min":0.40100000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.8010000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":0.48899999999999999}]},{"osd":4,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63900000000000001,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83999999999999997,"5min":0.871,"15min":0.871},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58999999999999997,"15min":0.56000000000000005},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.82399999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65400000000000003}]},{"osd":6,"last update":"Mon Feb 26 04:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63500000000000001,"15min":0.59699999999999998},"min":{"1min":0.52300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91800000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63200000000000001,"15min":0.60999999999999999},"min":{"1min":0.51300000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80000000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.752}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67400000000000004,"15min":0.65700000000000003},"min":{"1min":0.46500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89600000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70899999999999996}]},{"osd":8,"last update":"Mon Feb 26 04:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61099999999999999,"15min":0.59599999999999997},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.36399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.68600000000000005}]},{"osd":9,"last update":"Mon Feb 26 04:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69299999999999995,"15min":0.67600000000000005},"min":{"1min":0.59499999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.73199999999999998,"15min":0.69399999999999995},"min":{"1min":0.71899999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95899999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.82699999999999996}]},{"osd":10,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.48299999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.90900000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.81399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":69286,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70636,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71172,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T04:57:56.516 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-02-26T04:57:56.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T04:57:56.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.618+0000 7f4beb00f640 1 -- 172.21.15.60:0/2788491948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be40ff830 msgr2=0x7f4be40ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.618+0000 7f4beb00f640 1 --2- 172.21.15.60:0/2788491948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be40ff830 0x7f4be40ffc50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009900 tx=0x7f4bdc02bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.618+0000 7f4beb00f640 1 -- 172.21.15.60:0/2788491948 shutdown_connections 2024-02-26T04:57:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.618+0000 7f4beb00f640 1 --2- 172.21.15.60:0/2788491948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be41013e0 0x7f4be4107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.618+0000 7f4beb00f640 1 --2- 172.21.15.60:0/2788491948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4be4100a20 0x7f4be4100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.618+0000 7f4beb00f640 1 --2- 172.21.15.60:0/2788491948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be40ff830 0x7f4be40ffc50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.618+0000 7f4beb00f640 1 -- 172.21.15.60:0/2788491948 >> 172.21.15.60:0/2788491948 conn(0x7f4be40fada0 msgr2=0x7f4be40fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.618+0000 7f4beb00f640 1 -- 172.21.15.60:0/2788491948 shutdown_connections 2024-02-26T04:57:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.618+0000 7f4beb00f640 1 -- 172.21.15.60:0/2788491948 wait complete. 2024-02-26T04:57:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.618+0000 7f4beb00f640 1 Processor -- start 2024-02-26T04:57:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.618+0000 7f4beb00f640 1 -- start start 2024-02-26T04:57:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4beb00f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be40ff830 0x7f4be40fce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4beb00f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be4100a20 0x7f4be419bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4beb00f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4be41013e0 0x7f4be419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4beb00f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be419eca0 con 0x7f4be40ff830 2024-02-26T04:57:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4beb00f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be419ee10 con 0x7f4be4100a20 2024-02-26T04:57:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4beb00f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be419efd0 con 0x7f4be41013e0 2024-02-26T04:57:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be8dab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be40ff830 0x7f4be40fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be8dab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be40ff830 0x7f4be40fce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44224/0 (socket says 172.21.15.60:44224) 2024-02-26T04:57:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be8dab640 1 -- 172.21.15.60:0/2685949320 learned_addr learned my addr 172.21.15.60:0/2685949320 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be3fff640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be4100a20 0x7f4be419bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be95ac640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4be41013e0 0x7f4be419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be8dab640 1 -- 172.21.15.60:0/2685949320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4be41013e0 msgr2=0x7f4be419e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be8dab640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4be41013e0 0x7f4be419e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be8dab640 1 -- 172.21.15.60:0/2685949320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be4100a20 msgr2=0x7f4be419bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be8dab640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be4100a20 0x7f4be419bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be8dab640 1 -- 172.21.15.60:0/2685949320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc009560 con 0x7f4be40ff830 2024-02-26T04:57:56.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be3fff640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be4100a20 0x7f4be419bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:56.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be8dab640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be40ff830 0x7f4be40fce30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc005ab0 tx=0x7f4bdc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:56.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be1ffb640 1 -- 172.21.15.60:0/2685949320 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc004aa0 con 0x7f4be40ff830 2024-02-26T04:57:56.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4beb00f640 1 -- 172.21.15.60:0/2685949320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be419f250 con 0x7f4be40ff830 2024-02-26T04:57:56.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be1ffb640 1 -- 172.21.15.60:0/2685949320 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bdc0052d0 con 0x7f4be40ff830 2024-02-26T04:57:56.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.619+0000 7f4be1ffb640 1 -- 172.21.15.60:0/2685949320 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc003e80 con 0x7f4be40ff830 2024-02-26T04:57:56.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.620+0000 7f4beb00f640 1 -- 172.21.15.60:0/2685949320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be419f7b0 con 0x7f4be40ff830 2024-02-26T04:57:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.621+0000 7f4beb00f640 1 -- 172.21.15.60:0/2685949320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be4106f20 con 0x7f4be40ff830 2024-02-26T04:57:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.622+0000 7f4be1ffb640 1 -- 172.21.15.60:0/2685949320 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f4bdc005000 con 0x7f4be40ff830 2024-02-26T04:57:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.622+0000 7f4be1ffb640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4bd00a20c0 0x7f4bd00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.622+0000 7f4be1ffb640 1 -- 172.21.15.60:0/2685949320 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f4bdc0f0c70 con 0x7f4be40ff830 2024-02-26T04:57:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.622+0000 7f4be3fff640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4bd00a20c0 0x7f4bd00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.623+0000 7f4be3fff640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4bd00a20c0 0x7f4bd00a4580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4be40fcba0 tx=0x7f4bd4007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:56.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.626+0000 7f4be1ffb640 1 -- 172.21.15.60:0/2685949320 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4bdc0b9800 con 0x7f4be40ff830 2024-02-26T04:57:56.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:56 smithi156 ceph-mon[35805]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T04:57:56.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:56 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/889553695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:56.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:56 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4002182226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:56.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:56 smithi156 ceph-mon[35805]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:56.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:56 smithi191 ceph-mon[36352]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T04:57:56.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:56 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/889553695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:56.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:56 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4002182226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:56.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:56 smithi191 ceph-mon[36352]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:56.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.804+0000 7f4beb00f640 1 -- 172.21.15.60:0/2685949320 --> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4be402ce10 con 0x7f4bd00a20c0 2024-02-26T04:57:56.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.817+0000 7f4be1ffb640 1 -- 172.21.15.60:0/2685949320 <== mgr.14152 v2:172.21.15.60:6800/3458567419 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427917 (secure 0 0 0) 0x7f4be402ce10 con 0x7f4bd00a20c0 2024-02-26T04:57:56.824 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:56.826 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-02-26T04:57:56.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.821+0000 7f4beb00f640 1 -- 172.21.15.60:0/2685949320 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4bd00a20c0 msgr2=0x7f4bd00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.821+0000 7f4beb00f640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4bd00a20c0 0x7f4bd00a4580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4be40fcba0 tx=0x7f4bd4007480 comp rx=0 tx=0).stop 2024-02-26T04:57:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.821+0000 7f4beb00f640 1 -- 172.21.15.60:0/2685949320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be40ff830 msgr2=0x7f4be40fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.821+0000 7f4beb00f640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be40ff830 0x7f4be40fce30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc005ab0 tx=0x7f4bdc004270 comp rx=0 tx=0).stop 2024-02-26T04:57:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.821+0000 7f4beb00f640 1 -- 172.21.15.60:0/2685949320 shutdown_connections 2024-02-26T04:57:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.821+0000 7f4beb00f640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f4bd00a20c0 0x7f4bd00a4580 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.821+0000 7f4beb00f640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4be41013e0 0x7f4be419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.821+0000 7f4beb00f640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4be4100a20 0x7f4be419bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.821+0000 7f4beb00f640 1 --2- 172.21.15.60:0/2685949320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be40ff830 0x7f4be40fce30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.821+0000 7f4beb00f640 1 -- 172.21.15.60:0/2685949320 >> 172.21.15.60:0/2685949320 conn(0x7f4be40fada0 msgr2=0x7f4be41040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.821+0000 7f4beb00f640 1 -- 172.21.15.60:0/2685949320 shutdown_connections 2024-02-26T04:57:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.821+0000 7f4beb00f640 1 -- 172.21.15.60:0/2685949320 wait complete. 2024-02-26T04:57:56.834 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-02-26T04:57:56.345435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663806,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":881,"num_read_kb":562,"num_write":355,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181318,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386628,"kb_used_data":12452,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555196,"statfs":{"total":1151940427776,"available":1151544520704,"internally_reserved":0,"allocated":12750848,"data_stored":8208968,"data_compressed":430314,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5989,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":47,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003013"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184951+0000","last_change":"2024-02-26T04:55:28.208832+0000","last_active":"2024-02-26T04:57:21.184951+0000","last_peered":"2024-02-26T04:57:21.184951+0000","last_clean":"2024-02-26T04:57:21.184951+0000","last_became_active":"2024-02-26T04:55:28.208454+0000","last_became_peered":"2024-02-26T04:55:28.208454+0000","last_unstale":"2024-02-26T04:57:21.184951+0000","last_undegraded":"2024-02-26T04:57:21.184951+0000","last_fullsized":"2024-02-26T04:57:21.184951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:55:54.132384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391600+0000","last_change":"2024-02-26T04:55:28.203876+0000","last_active":"2024-02-26T04:57:21.391600+0000","last_peered":"2024-02-26T04:57:21.391600+0000","last_clean":"2024-02-26T04:57:21.391600+0000","last_became_active":"2024-02-26T04:55:28.203399+0000","last_became_peered":"2024-02-26T04:55:28.203399+0000","last_unstale":"2024-02-26T04:57:21.391600+0000","last_undegraded":"2024-02-26T04:57:21.391600+0000","last_fullsized":"2024-02-26T04:57:21.391600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:31:40.927929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391340+0000","last_change":"2024-02-26T04:55:28.211388+0000","last_active":"2024-02-26T04:57:21.391340+0000","last_peered":"2024-02-26T04:57:21.391340+0000","last_clean":"2024-02-26T04:57:21.391340+0000","last_became_active":"2024-02-26T04:55:28.209007+0000","last_became_peered":"2024-02-26T04:55:28.209007+0000","last_unstale":"2024-02-26T04:57:21.391340+0000","last_undegraded":"2024-02-26T04:57:21.391340+0000","last_fullsized":"2024-02-26T04:57:21.391340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:37:01.197589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177443+0000","last_change":"2024-02-26T04:55:28.216378+0000","last_active":"2024-02-26T04:57:21.177443+0000","last_peered":"2024-02-26T04:57:21.177443+0000","last_clean":"2024-02-26T04:57:21.177443+0000","last_became_active":"2024-02-26T04:55:28.216074+0000","last_became_peered":"2024-02-26T04:55:28.216074+0000","last_unstale":"2024-02-26T04:57:21.177443+0000","last_undegraded":"2024-02-26T04:57:21.177443+0000","last_fullsized":"2024-02-26T04:57:21.177443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:15:20.460699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180626+0000","last_change":"2024-02-26T04:55:28.213026+0000","last_active":"2024-02-26T04:57:21.180626+0000","last_peered":"2024-02-26T04:57:21.180626+0000","last_clean":"2024-02-26T04:57:21.180626+0000","last_became_active":"2024-02-26T04:55:28.212706+0000","last_became_peered":"2024-02-26T04:55:28.212706+0000","last_unstale":"2024-02-26T04:57:21.180626+0000","last_undegraded":"2024-02-26T04:57:21.180626+0000","last_fullsized":"2024-02-26T04:57:21.180626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:57:35.961956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.245681+0000","last_change":"2024-02-26T04:55:28.202123+0000","last_active":"2024-02-26T04:57:21.245681+0000","last_peered":"2024-02-26T04:57:21.245681+0000","last_clean":"2024-02-26T04:57:21.245681+0000","last_became_active":"2024-02-26T04:55:28.201733+0000","last_became_peered":"2024-02-26T04:55:28.201733+0000","last_unstale":"2024-02-26T04:57:21.245681+0000","last_undegraded":"2024-02-26T04:57:21.245681+0000","last_fullsized":"2024-02-26T04:57:21.245681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:43:11.300006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180147+0000","last_change":"2024-02-26T04:56:20.508346+0000","last_active":"2024-02-26T04:57:21.180147+0000","last_peered":"2024-02-26T04:57:21.180147+0000","last_clean":"2024-02-26T04:57:21.180147+0000","last_became_active":"2024-02-26T04:56:20.507941+0000","last_became_peered":"2024-02-26T04:56:20.507941+0000","last_unstale":"2024-02-26T04:57:21.180147+0000","last_undegraded":"2024-02-26T04:57:21.180147+0000","last_fullsized":"2024-02-26T04:57:21.180147+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:35:27.425477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.185909+0000","last_change":"2024-02-26T04:55:28.214631+0000","last_active":"2024-02-26T04:57:21.185909+0000","last_peered":"2024-02-26T04:57:21.185909+0000","last_clean":"2024-02-26T04:57:21.185909+0000","last_became_active":"2024-02-26T04:55:28.214290+0000","last_became_peered":"2024-02-26T04:55:28.214290+0000","last_unstale":"2024-02-26T04:57:21.185909+0000","last_undegraded":"2024-02-26T04:57:21.185909+0000","last_fullsized":"2024-02-26T04:57:21.185909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:30:11.494833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179697+0000","last_change":"2024-02-26T04:56:20.508783+0000","last_active":"2024-02-26T04:57:21.179697+0000","last_peered":"2024-02-26T04:57:21.179697+0000","last_clean":"2024-02-26T04:57:21.179697+0000","last_became_active":"2024-02-26T04:56:20.508372+0000","last_became_peered":"2024-02-26T04:56:20.508372+0000","last_unstale":"2024-02-26T04:57:21.179697+0000","last_undegraded":"2024-02-26T04:57:21.179697+0000","last_fullsized":"2024-02-26T04:57:21.179697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:32.415022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.246168+0000","last_change":"2024-02-26T04:55:28.204176+0000","last_active":"2024-02-26T04:57:21.246168+0000","last_peered":"2024-02-26T04:57:21.246168+0000","last_clean":"2024-02-26T04:57:21.246168+0000","last_became_active":"2024-02-26T04:55:28.203821+0000","last_became_peered":"2024-02-26T04:55:28.203821+0000","last_unstale":"2024-02-26T04:57:21.246168+0000","last_undegraded":"2024-02-26T04:57:21.246168+0000","last_fullsized":"2024-02-26T04:57:21.246168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:24:03.132650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180742+0000","last_change":"2024-02-26T04:55:28.215958+0000","last_active":"2024-02-26T04:57:21.180742+0000","last_peered":"2024-02-26T04:57:21.180742+0000","last_clean":"2024-02-26T04:57:21.180742+0000","last_became_active":"2024-02-26T04:55:28.215594+0000","last_became_peered":"2024-02-26T04:55:28.215594+0000","last_unstale":"2024-02-26T04:57:21.180742+0000","last_undegraded":"2024-02-26T04:57:21.180742+0000","last_fullsized":"2024-02-26T04:57:21.180742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:45:47.915379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177442+0000","last_change":"2024-02-26T04:55:28.218392+0000","last_active":"2024-02-26T04:57:21.177442+0000","last_peered":"2024-02-26T04:57:21.177442+0000","last_clean":"2024-02-26T04:57:21.177442+0000","last_became_active":"2024-02-26T04:55:28.217933+0000","last_became_peered":"2024-02-26T04:55:28.217933+0000","last_unstale":"2024-02-26T04:57:21.177442+0000","last_undegraded":"2024-02-26T04:57:21.177442+0000","last_fullsized":"2024-02-26T04:57:21.177442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:30:38.880649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176503+0000","last_change":"2024-02-26T04:55:28.197099+0000","last_active":"2024-02-26T04:57:21.176503+0000","last_peered":"2024-02-26T04:57:21.176503+0000","last_clean":"2024-02-26T04:57:21.176503+0000","last_became_active":"2024-02-26T04:55:28.196755+0000","last_became_peered":"2024-02-26T04:55:28.196755+0000","last_unstale":"2024-02-26T04:57:21.176503+0000","last_undegraded":"2024-02-26T04:57:21.176503+0000","last_fullsized":"2024-02-26T04:57:21.176503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:37:31.685845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391224+0000","last_change":"2024-02-26T04:55:28.212064+0000","last_active":"2024-02-26T04:57:21.391224+0000","last_peered":"2024-02-26T04:57:21.391224+0000","last_clean":"2024-02-26T04:57:21.391224+0000","last_became_active":"2024-02-26T04:55:28.208742+0000","last_became_peered":"2024-02-26T04:55:28.208742+0000","last_unstale":"2024-02-26T04:57:21.391224+0000","last_undegraded":"2024-02-26T04:57:21.391224+0000","last_fullsized":"2024-02-26T04:57:21.391224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:38:19.742468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391162+0000","last_change":"2024-02-26T04:56:20.511014+0000","last_active":"2024-02-26T04:57:21.391162+0000","last_peered":"2024-02-26T04:57:21.391162+0000","last_clean":"2024-02-26T04:57:21.391162+0000","last_became_active":"2024-02-26T04:56:20.510463+0000","last_became_peered":"2024-02-26T04:56:20.510463+0000","last_unstale":"2024-02-26T04:57:21.391162+0000","last_undegraded":"2024-02-26T04:57:21.391162+0000","last_fullsized":"2024-02-26T04:57:21.391162+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:44:59.574588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363448+0000","last_change":"2024-02-26T04:55:28.222810+0000","last_active":"2024-02-26T04:57:21.363448+0000","last_peered":"2024-02-26T04:57:21.363448+0000","last_clean":"2024-02-26T04:57:21.363448+0000","last_became_active":"2024-02-26T04:55:28.222341+0000","last_became_peered":"2024-02-26T04:55:28.222341+0000","last_unstale":"2024-02-26T04:57:21.363448+0000","last_undegraded":"2024-02-26T04:57:21.363448+0000","last_fullsized":"2024-02-26T04:57:21.363448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:59:21.627517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184837+0000","last_change":"2024-02-26T04:55:28.208377+0000","last_active":"2024-02-26T04:57:21.184837+0000","last_peered":"2024-02-26T04:57:21.184837+0000","last_clean":"2024-02-26T04:57:21.184837+0000","last_became_active":"2024-02-26T04:55:28.207932+0000","last_became_peered":"2024-02-26T04:55:28.207932+0000","last_unstale":"2024-02-26T04:57:21.184837+0000","last_undegraded":"2024-02-26T04:57:21.184837+0000","last_fullsized":"2024-02-26T04:57:21.184837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:15:39.586191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179812+0000","last_change":"2024-02-26T04:55:28.201105+0000","last_active":"2024-02-26T04:57:21.179812+0000","last_peered":"2024-02-26T04:57:21.179812+0000","last_clean":"2024-02-26T04:57:21.179812+0000","last_became_active":"2024-02-26T04:55:28.200796+0000","last_became_peered":"2024-02-26T04:55:28.200796+0000","last_unstale":"2024-02-26T04:57:21.179812+0000","last_undegraded":"2024-02-26T04:57:21.179812+0000","last_fullsized":"2024-02-26T04:57:21.179812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:53:43.713055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181623+0000","last_change":"2024-02-26T04:55:28.215725+0000","last_active":"2024-02-26T04:57:21.181623+0000","last_peered":"2024-02-26T04:57:21.181623+0000","last_clean":"2024-02-26T04:57:21.181623+0000","last_became_active":"2024-02-26T04:55:28.215341+0000","last_became_peered":"2024-02-26T04:55:28.215341+0000","last_unstale":"2024-02-26T04:57:21.181623+0000","last_undegraded":"2024-02-26T04:57:21.181623+0000","last_fullsized":"2024-02-26T04:57:21.181623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:26:26.960474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180646+0000","last_change":"2024-02-26T04:55:32.233303+0000","last_active":"2024-02-26T04:57:21.180646+0000","last_peered":"2024-02-26T04:57:21.180646+0000","last_clean":"2024-02-26T04:57:21.180646+0000","last_became_active":"2024-02-26T04:55:32.232901+0000","last_became_peered":"2024-02-26T04:55:32.232901+0000","last_unstale":"2024-02-26T04:57:21.180646+0000","last_undegraded":"2024-02-26T04:57:21.180646+0000","last_fullsized":"2024-02-26T04:57:21.180646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:37:49.659668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.194950+0000","last_change":"2024-02-26T04:57:21.194629+0000","last_active":"2024-02-26T04:57:21.194950+0000","last_peered":"2024-02-26T04:57:21.194950+0000","last_clean":"2024-02-26T04:57:21.194950+0000","last_became_active":"2024-02-26T04:57:21.186661+0000","last_became_peered":"2024-02-26T04:57:21.186661+0000","last_unstale":"2024-02-26T04:57:21.194950+0000","last_undegraded":"2024-02-26T04:57:21.194950+0000","last_fullsized":"2024-02-26T04:57:21.194950+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:08:47.904102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179621+0000","last_change":"2024-02-26T04:56:20.511029+0000","last_active":"2024-02-26T04:57:21.179621+0000","last_peered":"2024-02-26T04:57:21.179621+0000","last_clean":"2024-02-26T04:57:21.179621+0000","last_became_active":"2024-02-26T04:56:20.510590+0000","last_became_peered":"2024-02-26T04:56:20.510590+0000","last_unstale":"2024-02-26T04:57:21.179621+0000","last_undegraded":"2024-02-26T04:57:21.179621+0000","last_fullsized":"2024-02-26T04:57:21.179621+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:01:25.641444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176799+0000","last_change":"2024-02-26T04:55:32.233142+0000","last_active":"2024-02-26T04:57:21.176799+0000","last_peered":"2024-02-26T04:57:21.176799+0000","last_clean":"2024-02-26T04:57:21.176799+0000","last_became_active":"2024-02-26T04:55:32.232722+0000","last_became_peered":"2024-02-26T04:55:32.232722+0000","last_unstale":"2024-02-26T04:57:21.176799+0000","last_undegraded":"2024-02-26T04:57:21.176799+0000","last_fullsized":"2024-02-26T04:57:21.176799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:30:20.906104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186620+0000","last_change":"2024-02-26T04:55:27.203420+0000","last_active":"2024-02-26T04:57:21.186620+0000","last_peered":"2024-02-26T04:57:21.186620+0000","last_clean":"2024-02-26T04:57:21.186620+0000","last_became_active":"2024-02-26T04:55:27.202968+0000","last_became_peered":"2024-02-26T04:55:27.202968+0000","last_unstale":"2024-02-26T04:57:21.186620+0000","last_undegraded":"2024-02-26T04:57:21.186620+0000","last_fullsized":"2024-02-26T04:57:21.186620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:44.496528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391314+0000","last_change":"2024-02-26T04:55:28.217422+0000","last_active":"2024-02-26T04:57:21.391314+0000","last_peered":"2024-02-26T04:57:21.391314+0000","last_clean":"2024-02-26T04:57:21.391314+0000","last_became_active":"2024-02-26T04:55:28.217103+0000","last_became_peered":"2024-02-26T04:55:28.217103+0000","last_unstale":"2024-02-26T04:57:21.391314+0000","last_undegraded":"2024-02-26T04:57:21.391314+0000","last_fullsized":"2024-02-26T04:57:21.391314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:08:37.075467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176904+0000","last_change":"2024-02-26T04:55:32.233322+0000","last_active":"2024-02-26T04:57:21.176904+0000","last_peered":"2024-02-26T04:57:21.176904+0000","last_clean":"2024-02-26T04:57:21.176904+0000","last_became_active":"2024-02-26T04:55:32.232679+0000","last_became_peered":"2024-02-26T04:55:32.232679+0000","last_unstale":"2024-02-26T04:57:21.176904+0000","last_undegraded":"2024-02-26T04:57:21.176904+0000","last_fullsized":"2024-02-26T04:57:21.176904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:10.775924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.192327+0000","last_change":"2024-02-26T04:57:21.192327+0000","last_active":"2024-02-26T04:57:21.192327+0000","last_peered":"2024-02-26T04:57:21.192327+0000","last_clean":"2024-02-26T04:57:21.192327+0000","last_became_active":"2024-02-26T04:57:21.186299+0000","last_became_peered":"2024-02-26T04:57:21.186299+0000","last_unstale":"2024-02-26T04:57:21.192327+0000","last_undegraded":"2024-02-26T04:57:21.192327+0000","last_fullsized":"2024-02-26T04:57:21.192327+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:15.566202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363709+0000","last_change":"2024-02-26T04:55:28.210669+0000","last_active":"2024-02-26T04:57:21.363709+0000","last_peered":"2024-02-26T04:57:21.363709+0000","last_clean":"2024-02-26T04:57:21.363709+0000","last_became_active":"2024-02-26T04:55:28.210322+0000","last_became_peered":"2024-02-26T04:55:28.210322+0000","last_unstale":"2024-02-26T04:57:21.363709+0000","last_undegraded":"2024-02-26T04:57:21.363709+0000","last_fullsized":"2024-02-26T04:57:21.363709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:57:29.121762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176799+0000","last_change":"2024-02-26T04:55:32.229077+0000","last_active":"2024-02-26T04:57:21.176799+0000","last_peered":"2024-02-26T04:57:21.176799+0000","last_clean":"2024-02-26T04:57:21.176799+0000","last_became_active":"2024-02-26T04:55:32.228670+0000","last_became_peered":"2024-02-26T04:55:32.228670+0000","last_unstale":"2024-02-26T04:57:21.176799+0000","last_undegraded":"2024-02-26T04:57:21.176799+0000","last_fullsized":"2024-02-26T04:57:21.176799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:35:20.843351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181671+0000","last_change":"2024-02-26T04:55:27.206088+0000","last_active":"2024-02-26T04:57:21.181671+0000","last_peered":"2024-02-26T04:57:21.181671+0000","last_clean":"2024-02-26T04:57:21.181671+0000","last_became_active":"2024-02-26T04:55:27.205696+0000","last_became_peered":"2024-02-26T04:55:27.205696+0000","last_unstale":"2024-02-26T04:57:21.181671+0000","last_undegraded":"2024-02-26T04:57:21.181671+0000","last_fullsized":"2024-02-26T04:57:21.181671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:14:03.552251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178333+0000","last_change":"2024-02-26T04:56:20.508805+0000","last_active":"2024-02-26T04:57:21.178333+0000","last_peered":"2024-02-26T04:57:21.178333+0000","last_clean":"2024-02-26T04:57:21.178333+0000","last_became_active":"2024-02-26T04:56:20.507796+0000","last_became_peered":"2024-02-26T04:56:20.507796+0000","last_unstale":"2024-02-26T04:57:21.178333+0000","last_undegraded":"2024-02-26T04:57:21.178333+0000","last_fullsized":"2024-02-26T04:57:21.178333+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:25.453111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.187012+0000","last_change":"2024-02-26T04:57:21.187012+0000","last_active":"2024-02-26T04:57:21.187012+0000","last_peered":"2024-02-26T04:57:21.187012+0000","last_clean":"2024-02-26T04:57:21.187012+0000","last_became_active":"2024-02-26T04:57:21.186512+0000","last_became_peered":"2024-02-26T04:57:21.186512+0000","last_unstale":"2024-02-26T04:57:21.187012+0000","last_undegraded":"2024-02-26T04:57:21.187012+0000","last_fullsized":"2024-02-26T04:57:21.187012+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:36:14.768552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:54.322839+0000","last_change":"2024-02-26T04:55:27.201261+0000","last_active":"2024-02-26T04:57:54.322839+0000","last_peered":"2024-02-26T04:57:54.322839+0000","last_clean":"2024-02-26T04:57:54.322839+0000","last_became_active":"2024-02-26T04:55:27.200892+0000","last_became_peered":"2024-02-26T04:55:27.200892+0000","last_unstale":"2024-02-26T04:57:54.322839+0000","last_undegraded":"2024-02-26T04:57:54.322839+0000","last_fullsized":"2024-02-26T04:57:54.322839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:34:32.774143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181394+0000","last_change":"2024-02-26T04:55:28.200445+0000","last_active":"2024-02-26T04:57:21.181394+0000","last_peered":"2024-02-26T04:57:21.181394+0000","last_clean":"2024-02-26T04:57:21.181394+0000","last_became_active":"2024-02-26T04:55:28.199959+0000","last_became_peered":"2024-02-26T04:55:28.199959+0000","last_unstale":"2024-02-26T04:57:21.181394+0000","last_undegraded":"2024-02-26T04:57:21.181394+0000","last_fullsized":"2024-02-26T04:57:21.181394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:36:27.397644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181032+0000","last_change":"2024-02-26T04:55:32.232619+0000","last_active":"2024-02-26T04:57:21.181032+0000","last_peered":"2024-02-26T04:57:21.181032+0000","last_clean":"2024-02-26T04:57:21.181032+0000","last_became_active":"2024-02-26T04:55:32.232112+0000","last_became_peered":"2024-02-26T04:55:32.232112+0000","last_unstale":"2024-02-26T04:57:21.181032+0000","last_undegraded":"2024-02-26T04:57:21.181032+0000","last_fullsized":"2024-02-26T04:57:21.181032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:19.741393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.245662+0000","last_change":"2024-02-26T04:55:27.197376+0000","last_active":"2024-02-26T04:57:21.245662+0000","last_peered":"2024-02-26T04:57:21.245662+0000","last_clean":"2024-02-26T04:57:21.245662+0000","last_became_active":"2024-02-26T04:55:27.195258+0000","last_became_peered":"2024-02-26T04:55:27.195258+0000","last_unstale":"2024-02-26T04:57:21.245662+0000","last_undegraded":"2024-02-26T04:57:21.245662+0000","last_fullsized":"2024-02-26T04:57:21.245662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:41:51.284525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178375+0000","last_change":"2024-02-26T04:55:28.214337+0000","last_active":"2024-02-26T04:57:21.178375+0000","last_peered":"2024-02-26T04:57:21.178375+0000","last_clean":"2024-02-26T04:57:21.178375+0000","last_became_active":"2024-02-26T04:55:28.213966+0000","last_became_peered":"2024-02-26T04:55:28.213966+0000","last_unstale":"2024-02-26T04:57:21.178375+0000","last_undegraded":"2024-02-26T04:57:21.178375+0000","last_fullsized":"2024-02-26T04:57:21.178375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:49:42.974084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184714+0000","last_change":"2024-02-26T04:55:32.235579+0000","last_active":"2024-02-26T04:57:21.184714+0000","last_peered":"2024-02-26T04:57:21.184714+0000","last_clean":"2024-02-26T04:57:21.184714+0000","last_became_active":"2024-02-26T04:55:32.235241+0000","last_became_peered":"2024-02-26T04:55:32.235241+0000","last_unstale":"2024-02-26T04:57:21.184714+0000","last_undegraded":"2024-02-26T04:57:21.184714+0000","last_fullsized":"2024-02-26T04:57:21.184714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:06.319874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176279+0000","last_change":"2024-02-26T04:55:27.197689+0000","last_active":"2024-02-26T04:57:21.176279+0000","last_peered":"2024-02-26T04:57:21.176279+0000","last_clean":"2024-02-26T04:57:21.176279+0000","last_became_active":"2024-02-26T04:55:27.197285+0000","last_became_peered":"2024-02-26T04:55:27.197285+0000","last_unstale":"2024-02-26T04:57:21.176279+0000","last_undegraded":"2024-02-26T04:57:21.176279+0000","last_fullsized":"2024-02-26T04:57:21.176279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:45:36.688902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.185917+0000","last_change":"2024-02-26T04:55:28.201902+0000","last_active":"2024-02-26T04:57:21.185917+0000","last_peered":"2024-02-26T04:57:21.185917+0000","last_clean":"2024-02-26T04:57:21.185917+0000","last_became_active":"2024-02-26T04:55:28.201604+0000","last_became_peered":"2024-02-26T04:55:28.201604+0000","last_unstale":"2024-02-26T04:57:21.185917+0000","last_undegraded":"2024-02-26T04:57:21.185917+0000","last_fullsized":"2024-02-26T04:57:21.185917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:48:57.711555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391379+0000","last_change":"2024-02-26T04:55:32.230153+0000","last_active":"2024-02-26T04:57:21.391379+0000","last_peered":"2024-02-26T04:57:21.391379+0000","last_clean":"2024-02-26T04:57:21.391379+0000","last_became_active":"2024-02-26T04:55:32.229695+0000","last_became_peered":"2024-02-26T04:55:32.229695+0000","last_unstale":"2024-02-26T04:57:21.391379+0000","last_undegraded":"2024-02-26T04:57:21.391379+0000","last_fullsized":"2024-02-26T04:57:21.391379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:15:39.575727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177757+0000","last_change":"2024-02-26T04:55:27.200943+0000","last_active":"2024-02-26T04:57:21.177757+0000","last_peered":"2024-02-26T04:57:21.177757+0000","last_clean":"2024-02-26T04:57:21.177757+0000","last_became_active":"2024-02-26T04:55:27.200565+0000","last_became_peered":"2024-02-26T04:55:27.200565+0000","last_unstale":"2024-02-26T04:57:21.177757+0000","last_undegraded":"2024-02-26T04:57:21.177757+0000","last_fullsized":"2024-02-26T04:57:21.177757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:54:19.670631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179352+0000","last_change":"2024-02-26T04:55:28.212232+0000","last_active":"2024-02-26T04:57:21.179352+0000","last_peered":"2024-02-26T04:57:21.179352+0000","last_clean":"2024-02-26T04:57:21.179352+0000","last_became_active":"2024-02-26T04:55:28.211875+0000","last_became_peered":"2024-02-26T04:55:28.211875+0000","last_unstale":"2024-02-26T04:57:21.179352+0000","last_undegraded":"2024-02-26T04:57:21.179352+0000","last_fullsized":"2024-02-26T04:57:21.179352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:55.379271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186082+0000","last_change":"2024-02-26T04:55:32.234961+0000","last_active":"2024-02-26T04:57:21.186082+0000","last_peered":"2024-02-26T04:57:21.186082+0000","last_clean":"2024-02-26T04:57:21.186082+0000","last_became_active":"2024-02-26T04:55:32.234455+0000","last_became_peered":"2024-02-26T04:55:32.234455+0000","last_unstale":"2024-02-26T04:57:21.186082+0000","last_undegraded":"2024-02-26T04:57:21.186082+0000","last_fullsized":"2024-02-26T04:57:21.186082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:00:44.322341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":156,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:54.322208+0000","last_change":"2024-02-26T04:55:27.199036+0000","last_active":"2024-02-26T04:57:54.322208+0000","last_peered":"2024-02-26T04:57:54.322208+0000","last_clean":"2024-02-26T04:57:54.322208+0000","last_became_active":"2024-02-26T04:55:27.198694+0000","last_became_peered":"2024-02-26T04:55:27.198694+0000","last_unstale":"2024-02-26T04:57:54.322208+0000","last_undegraded":"2024-02-26T04:57:54.322208+0000","last_fullsized":"2024-02-26T04:57:54.322208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:31:41.935123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181847+0000","last_change":"2024-02-26T04:56:20.208105+0000","last_active":"2024-02-26T04:57:21.181847+0000","last_peered":"2024-02-26T04:57:21.181847+0000","last_clean":"2024-02-26T04:57:21.181847+0000","last_became_active":"2024-02-26T04:56:20.207691+0000","last_became_peered":"2024-02-26T04:56:20.207691+0000","last_unstale":"2024-02-26T04:57:21.181847+0000","last_undegraded":"2024-02-26T04:57:21.181847+0000","last_fullsized":"2024-02-26T04:57:21.181847+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:49:49.019089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,9],"acting":[8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179345+0000","last_change":"2024-02-26T04:55:32.228033+0000","last_active":"2024-02-26T04:57:21.179345+0000","last_peered":"2024-02-26T04:57:21.179345+0000","last_clean":"2024-02-26T04:57:21.179345+0000","last_became_active":"2024-02-26T04:55:32.224273+0000","last_became_peered":"2024-02-26T04:55:32.224273+0000","last_unstale":"2024-02-26T04:57:21.179345+0000","last_undegraded":"2024-02-26T04:57:21.179345+0000","last_fullsized":"2024-02-26T04:57:21.179345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:25:13.618421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180910+0000","last_change":"2024-02-26T04:55:27.202833+0000","last_active":"2024-02-26T04:57:21.180910+0000","last_peered":"2024-02-26T04:57:21.180910+0000","last_clean":"2024-02-26T04:57:21.180910+0000","last_became_active":"2024-02-26T04:55:27.202167+0000","last_became_peered":"2024-02-26T04:55:27.202167+0000","last_unstale":"2024-02-26T04:57:21.180910+0000","last_undegraded":"2024-02-26T04:57:21.180910+0000","last_fullsized":"2024-02-26T04:57:21.180910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:55:46.605763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391476+0000","last_change":"2024-02-26T04:56:20.207930+0000","last_active":"2024-02-26T04:57:21.391476+0000","last_peered":"2024-02-26T04:57:21.391476+0000","last_clean":"2024-02-26T04:57:21.391476+0000","last_became_active":"2024-02-26T04:56:20.207586+0000","last_became_peered":"2024-02-26T04:56:20.207586+0000","last_unstale":"2024-02-26T04:57:21.391476+0000","last_undegraded":"2024-02-26T04:57:21.391476+0000","last_fullsized":"2024-02-26T04:57:21.391476+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:37:57.141645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.365832+0000","last_change":"2024-02-26T04:57:21.365354+0000","last_active":"2024-02-26T04:57:21.365832+0000","last_peered":"2024-02-26T04:57:21.365832+0000","last_clean":"2024-02-26T04:57:21.365832+0000","last_became_active":"2024-02-26T04:57:21.364768+0000","last_became_peered":"2024-02-26T04:57:21.364768+0000","last_unstale":"2024-02-26T04:57:21.365832+0000","last_undegraded":"2024-02-26T04:57:21.365832+0000","last_fullsized":"2024-02-26T04:57:21.365832+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:33:05.881044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178420+0000","last_change":"2024-02-26T04:55:27.200389+0000","last_active":"2024-02-26T04:57:21.178420+0000","last_peered":"2024-02-26T04:57:21.178420+0000","last_clean":"2024-02-26T04:57:21.178420+0000","last_became_active":"2024-02-26T04:55:27.200019+0000","last_became_peered":"2024-02-26T04:55:27.200019+0000","last_unstale":"2024-02-26T04:57:21.178420+0000","last_undegraded":"2024-02-26T04:57:21.178420+0000","last_fullsized":"2024-02-26T04:57:21.178420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:59:11.820988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363892+0000","last_change":"2024-02-26T04:55:28.220309+0000","last_active":"2024-02-26T04:57:21.363892+0000","last_peered":"2024-02-26T04:57:21.363892+0000","last_clean":"2024-02-26T04:57:21.363892+0000","last_became_active":"2024-02-26T04:55:28.219987+0000","last_became_peered":"2024-02-26T04:55:28.219987+0000","last_unstale":"2024-02-26T04:57:21.363892+0000","last_undegraded":"2024-02-26T04:57:21.363892+0000","last_fullsized":"2024-02-26T04:57:21.363892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:39:34.713802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179368+0000","last_change":"2024-02-26T04:55:32.236817+0000","last_active":"2024-02-26T04:57:21.179368+0000","last_peered":"2024-02-26T04:57:21.179368+0000","last_clean":"2024-02-26T04:57:21.179368+0000","last_became_active":"2024-02-26T04:55:32.235818+0000","last_became_peered":"2024-02-26T04:55:32.235818+0000","last_unstale":"2024-02-26T04:57:21.179368+0000","last_undegraded":"2024-02-26T04:57:21.179368+0000","last_fullsized":"2024-02-26T04:57:21.179368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:56:54.121684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'39","reported_seq":323,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:54.343952+0000","last_change":"2024-02-26T04:55:27.201582+0000","last_active":"2024-02-26T04:57:54.343952+0000","last_peered":"2024-02-26T04:57:54.343952+0000","last_clean":"2024-02-26T04:57:54.343952+0000","last_became_active":"2024-02-26T04:55:27.201011+0000","last_became_peered":"2024-02-26T04:55:27.201011+0000","last_unstale":"2024-02-26T04:57:54.343952+0000","last_undegraded":"2024-02-26T04:57:54.343952+0000","last_fullsized":"2024-02-26T04:57:54.343952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:07:01.888924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69286,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":252,"num_read_kb":194,"num_write":40,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183993+0000","last_change":"2024-02-26T04:55:28.199313+0000","last_active":"2024-02-26T04:57:21.183993+0000","last_peered":"2024-02-26T04:57:21.183993+0000","last_clean":"2024-02-26T04:57:21.183993+0000","last_became_active":"2024-02-26T04:55:28.198973+0000","last_became_peered":"2024-02-26T04:55:28.198973+0000","last_unstale":"2024-02-26T04:57:21.183993+0000","last_undegraded":"2024-02-26T04:57:21.183993+0000","last_fullsized":"2024-02-26T04:57:21.183993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:33:49.005899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178664+0000","last_change":"2024-02-26T04:55:32.234529+0000","last_active":"2024-02-26T04:57:21.178664+0000","last_peered":"2024-02-26T04:57:21.178664+0000","last_clean":"2024-02-26T04:57:21.178664+0000","last_became_active":"2024-02-26T04:55:32.234105+0000","last_became_peered":"2024-02-26T04:55:32.234105+0000","last_unstale":"2024-02-26T04:57:21.178664+0000","last_undegraded":"2024-02-26T04:57:21.178664+0000","last_fullsized":"2024-02-26T04:57:21.178664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:17:40.909267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180944+0000","last_change":"2024-02-26T04:55:27.209178+0000","last_active":"2024-02-26T04:57:21.180944+0000","last_peered":"2024-02-26T04:57:21.180944+0000","last_clean":"2024-02-26T04:57:21.180944+0000","last_became_active":"2024-02-26T04:55:27.208834+0000","last_became_peered":"2024-02-26T04:55:27.208834+0000","last_unstale":"2024-02-26T04:57:21.180944+0000","last_undegraded":"2024-02-26T04:57:21.180944+0000","last_fullsized":"2024-02-26T04:57:21.180944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:33:28.187333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176297+0000","last_change":"2024-02-26T04:55:28.215208+0000","last_active":"2024-02-26T04:57:21.176297+0000","last_peered":"2024-02-26T04:57:21.176297+0000","last_clean":"2024-02-26T04:57:21.176297+0000","last_became_active":"2024-02-26T04:55:28.212135+0000","last_became_peered":"2024-02-26T04:55:28.212135+0000","last_unstale":"2024-02-26T04:57:21.176297+0000","last_undegraded":"2024-02-26T04:57:21.176297+0000","last_fullsized":"2024-02-26T04:57:21.176297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:51.269557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.182783+0000","last_change":"2024-02-26T04:55:32.235439+0000","last_active":"2024-02-26T04:57:21.182783+0000","last_peered":"2024-02-26T04:57:21.182783+0000","last_clean":"2024-02-26T04:57:21.182783+0000","last_became_active":"2024-02-26T04:55:32.234987+0000","last_became_peered":"2024-02-26T04:55:32.234987+0000","last_unstale":"2024-02-26T04:57:21.182783+0000","last_undegraded":"2024-02-26T04:57:21.182783+0000","last_fullsized":"2024-02-26T04:57:21.182783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:31:00.637846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177962+0000","last_change":"2024-02-26T04:55:27.205498+0000","last_active":"2024-02-26T04:57:21.177962+0000","last_peered":"2024-02-26T04:57:21.177962+0000","last_clean":"2024-02-26T04:57:21.177962+0000","last_became_active":"2024-02-26T04:55:27.205179+0000","last_became_peered":"2024-02-26T04:55:27.205179+0000","last_unstale":"2024-02-26T04:57:21.177962+0000","last_undegraded":"2024-02-26T04:57:21.177962+0000","last_fullsized":"2024-02-26T04:57:21.177962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:02:55.491896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180574+0000","last_change":"2024-02-26T04:55:28.201110+0000","last_active":"2024-02-26T04:57:21.180574+0000","last_peered":"2024-02-26T04:57:21.180574+0000","last_clean":"2024-02-26T04:57:21.180574+0000","last_became_active":"2024-02-26T04:55:28.200725+0000","last_became_peered":"2024-02-26T04:55:28.200725+0000","last_unstale":"2024-02-26T04:57:21.180574+0000","last_undegraded":"2024-02-26T04:57:21.180574+0000","last_fullsized":"2024-02-26T04:57:21.180574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:47:50.377925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.392866+0000","last_change":"2024-02-26T04:57:21.392457+0000","last_active":"2024-02-26T04:57:21.392866+0000","last_peered":"2024-02-26T04:57:21.392866+0000","last_clean":"2024-02-26T04:57:21.392866+0000","last_became_active":"2024-02-26T04:57:21.392057+0000","last_became_peered":"2024-02-26T04:57:21.392057+0000","last_unstale":"2024-02-26T04:57:21.392866+0000","last_undegraded":"2024-02-26T04:57:21.392866+0000","last_fullsized":"2024-02-26T04:57:21.392866+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:23:42.695300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"93'4","reported_seq":161,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:54.324261+0000","last_change":"2024-02-26T04:55:27.197338+0000","last_active":"2024-02-26T04:57:54.324261+0000","last_peered":"2024-02-26T04:57:54.324261+0000","last_clean":"2024-02-26T04:57:54.324261+0000","last_became_active":"2024-02-26T04:55:27.196898+0000","last_became_peered":"2024-02-26T04:55:27.196898+0000","last_unstale":"2024-02-26T04:57:54.324261+0000","last_undegraded":"2024-02-26T04:57:54.324261+0000","last_fullsized":"2024-02-26T04:57:54.324261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:27:09.635379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180780+0000","last_change":"2024-02-26T04:55:28.200665+0000","last_active":"2024-02-26T04:57:21.180780+0000","last_peered":"2024-02-26T04:57:21.180780+0000","last_clean":"2024-02-26T04:57:21.180780+0000","last_became_active":"2024-02-26T04:55:28.200329+0000","last_became_peered":"2024-02-26T04:55:28.200329+0000","last_unstale":"2024-02-26T04:57:21.180780+0000","last_undegraded":"2024-02-26T04:57:21.180780+0000","last_fullsized":"2024-02-26T04:57:21.180780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:37:27.090259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176631+0000","last_change":"2024-02-26T04:55:32.225861+0000","last_active":"2024-02-26T04:57:21.176631+0000","last_peered":"2024-02-26T04:57:21.176631+0000","last_clean":"2024-02-26T04:57:21.176631+0000","last_became_active":"2024-02-26T04:55:32.225187+0000","last_became_peered":"2024-02-26T04:55:32.225187+0000","last_unstale":"2024-02-26T04:57:21.176631+0000","last_undegraded":"2024-02-26T04:57:21.176631+0000","last_fullsized":"2024-02-26T04:57:21.176631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:14:09.077383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180544+0000","last_change":"2024-02-26T04:55:27.200321+0000","last_active":"2024-02-26T04:57:21.180544+0000","last_peered":"2024-02-26T04:57:21.180544+0000","last_clean":"2024-02-26T04:57:21.180544+0000","last_became_active":"2024-02-26T04:55:27.199978+0000","last_became_peered":"2024-02-26T04:55:27.199978+0000","last_unstale":"2024-02-26T04:57:21.180544+0000","last_undegraded":"2024-02-26T04:57:21.180544+0000","last_fullsized":"2024-02-26T04:57:21.180544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:08:20.381010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177958+0000","last_change":"2024-02-26T04:55:28.213940+0000","last_active":"2024-02-26T04:57:21.177958+0000","last_peered":"2024-02-26T04:57:21.177958+0000","last_clean":"2024-02-26T04:57:21.177958+0000","last_became_active":"2024-02-26T04:55:28.213590+0000","last_became_peered":"2024-02-26T04:55:28.213590+0000","last_unstale":"2024-02-26T04:57:21.177958+0000","last_undegraded":"2024-02-26T04:57:21.177958+0000","last_fullsized":"2024-02-26T04:57:21.177958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:29:29.151391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177711+0000","last_change":"2024-02-26T04:55:32.232669+0000","last_active":"2024-02-26T04:57:21.177711+0000","last_peered":"2024-02-26T04:57:21.177711+0000","last_clean":"2024-02-26T04:57:21.177711+0000","last_became_active":"2024-02-26T04:55:32.232288+0000","last_became_peered":"2024-02-26T04:55:32.232288+0000","last_unstale":"2024-02-26T04:57:21.177711+0000","last_undegraded":"2024-02-26T04:57:21.177711+0000","last_fullsized":"2024-02-26T04:57:21.177711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:31:42.436516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.182351+0000","last_change":"2024-02-26T04:55:27.205993+0000","last_active":"2024-02-26T04:57:21.182351+0000","last_peered":"2024-02-26T04:57:21.182351+0000","last_clean":"2024-02-26T04:57:21.182351+0000","last_became_active":"2024-02-26T04:55:27.205594+0000","last_became_peered":"2024-02-26T04:55:27.205594+0000","last_unstale":"2024-02-26T04:57:21.182351+0000","last_undegraded":"2024-02-26T04:57:21.182351+0000","last_fullsized":"2024-02-26T04:57:21.182351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:44:19.993925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184074+0000","last_change":"2024-02-26T04:55:28.203429+0000","last_active":"2024-02-26T04:57:21.184074+0000","last_peered":"2024-02-26T04:57:21.184074+0000","last_clean":"2024-02-26T04:57:21.184074+0000","last_became_active":"2024-02-26T04:55:28.203002+0000","last_became_peered":"2024-02-26T04:55:28.203002+0000","last_unstale":"2024-02-26T04:57:21.184074+0000","last_undegraded":"2024-02-26T04:57:21.184074+0000","last_fullsized":"2024-02-26T04:57:21.184074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:08:05.872241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181329+0000","last_change":"2024-02-26T04:55:32.239323+0000","last_active":"2024-02-26T04:57:21.181329+0000","last_peered":"2024-02-26T04:57:21.181329+0000","last_clean":"2024-02-26T04:57:21.181329+0000","last_became_active":"2024-02-26T04:55:32.238843+0000","last_became_peered":"2024-02-26T04:55:32.238843+0000","last_unstale":"2024-02-26T04:57:21.181329+0000","last_undegraded":"2024-02-26T04:57:21.181329+0000","last_fullsized":"2024-02-26T04:57:21.181329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:39:56.645429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179549+0000","last_change":"2024-02-26T04:55:27.203222+0000","last_active":"2024-02-26T04:57:21.179549+0000","last_peered":"2024-02-26T04:57:21.179549+0000","last_clean":"2024-02-26T04:57:21.179549+0000","last_became_active":"2024-02-26T04:55:27.202845+0000","last_became_peered":"2024-02-26T04:55:27.202845+0000","last_unstale":"2024-02-26T04:57:21.179549+0000","last_undegraded":"2024-02-26T04:57:21.179549+0000","last_fullsized":"2024-02-26T04:57:21.179549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:16:07.524260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.246028+0000","last_change":"2024-02-26T04:55:28.209122+0000","last_active":"2024-02-26T04:57:21.246028+0000","last_peered":"2024-02-26T04:57:21.246028+0000","last_clean":"2024-02-26T04:57:21.246028+0000","last_became_active":"2024-02-26T04:55:28.207598+0000","last_became_peered":"2024-02-26T04:55:28.207598+0000","last_unstale":"2024-02-26T04:57:21.246028+0000","last_undegraded":"2024-02-26T04:57:21.246028+0000","last_fullsized":"2024-02-26T04:57:21.246028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T07:02:10.763515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176657+0000","last_change":"2024-02-26T04:55:32.227333+0000","last_active":"2024-02-26T04:57:21.176657+0000","last_peered":"2024-02-26T04:57:21.176657+0000","last_clean":"2024-02-26T04:57:21.176657+0000","last_became_active":"2024-02-26T04:55:32.226956+0000","last_became_peered":"2024-02-26T04:55:32.226956+0000","last_unstale":"2024-02-26T04:57:21.176657+0000","last_undegraded":"2024-02-26T04:57:21.176657+0000","last_fullsized":"2024-02-26T04:57:21.176657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:51:40.710242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180411+0000","last_change":"2024-02-26T04:55:27.198781+0000","last_active":"2024-02-26T04:57:21.180411+0000","last_peered":"2024-02-26T04:57:21.180411+0000","last_clean":"2024-02-26T04:57:21.180411+0000","last_became_active":"2024-02-26T04:55:27.198288+0000","last_became_peered":"2024-02-26T04:55:27.198288+0000","last_unstale":"2024-02-26T04:57:21.180411+0000","last_undegraded":"2024-02-26T04:57:21.180411+0000","last_fullsized":"2024-02-26T04:57:21.180411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:52:18.665816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.245833+0000","last_change":"2024-02-26T04:55:28.213044+0000","last_active":"2024-02-26T04:57:21.245833+0000","last_peered":"2024-02-26T04:57:21.245833+0000","last_clean":"2024-02-26T04:57:21.245833+0000","last_became_active":"2024-02-26T04:55:28.212712+0000","last_became_peered":"2024-02-26T04:55:28.212712+0000","last_unstale":"2024-02-26T04:57:21.245833+0000","last_undegraded":"2024-02-26T04:57:21.245833+0000","last_fullsized":"2024-02-26T04:57:21.245833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:02:09.155551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176495+0000","last_change":"2024-02-26T04:55:32.231867+0000","last_active":"2024-02-26T04:57:21.176495+0000","last_peered":"2024-02-26T04:57:21.176495+0000","last_clean":"2024-02-26T04:57:21.176495+0000","last_became_active":"2024-02-26T04:55:32.231450+0000","last_became_peered":"2024-02-26T04:55:32.231450+0000","last_unstale":"2024-02-26T04:57:21.176495+0000","last_undegraded":"2024-02-26T04:57:21.176495+0000","last_fullsized":"2024-02-26T04:57:21.176495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:12:14.367427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177833+0000","last_change":"2024-02-26T04:55:28.207746+0000","last_active":"2024-02-26T04:57:21.177833+0000","last_peered":"2024-02-26T04:57:21.177833+0000","last_clean":"2024-02-26T04:57:21.177833+0000","last_became_active":"2024-02-26T04:55:28.207438+0000","last_became_peered":"2024-02-26T04:55:28.207438+0000","last_unstale":"2024-02-26T04:57:21.177833+0000","last_undegraded":"2024-02-26T04:57:21.177833+0000","last_fullsized":"2024-02-26T04:57:21.177833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:55:09.409635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":156,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:51.454112+0000","last_change":"2024-02-26T04:55:27.192233+0000","last_active":"2024-02-26T04:57:51.454112+0000","last_peered":"2024-02-26T04:57:51.454112+0000","last_clean":"2024-02-26T04:57:51.454112+0000","last_became_active":"2024-02-26T04:55:27.191868+0000","last_became_peered":"2024-02-26T04:55:27.191868+0000","last_unstale":"2024-02-26T04:57:51.454112+0000","last_undegraded":"2024-02-26T04:57:51.454112+0000","last_fullsized":"2024-02-26T04:57:51.454112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:37:58.430504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181068+0000","last_change":"2024-02-26T04:55:32.226577+0000","last_active":"2024-02-26T04:57:21.181068+0000","last_peered":"2024-02-26T04:57:21.181068+0000","last_clean":"2024-02-26T04:57:21.181068+0000","last_became_active":"2024-02-26T04:55:32.226148+0000","last_became_peered":"2024-02-26T04:55:32.226148+0000","last_unstale":"2024-02-26T04:57:21.181068+0000","last_undegraded":"2024-02-26T04:57:21.181068+0000","last_fullsized":"2024-02-26T04:57:21.181068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:34:53.099211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186167+0000","last_change":"2024-02-26T04:55:28.217865+0000","last_active":"2024-02-26T04:57:21.186167+0000","last_peered":"2024-02-26T04:57:21.186167+0000","last_clean":"2024-02-26T04:57:21.186167+0000","last_became_active":"2024-02-26T04:55:28.217075+0000","last_became_peered":"2024-02-26T04:55:28.217075+0000","last_unstale":"2024-02-26T04:57:21.186167+0000","last_undegraded":"2024-02-26T04:57:21.186167+0000","last_fullsized":"2024-02-26T04:57:21.186167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:30:38.720694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177120+0000","last_change":"2024-02-26T04:55:27.204300+0000","last_active":"2024-02-26T04:57:21.177120+0000","last_peered":"2024-02-26T04:57:21.177120+0000","last_clean":"2024-02-26T04:57:21.177120+0000","last_became_active":"2024-02-26T04:55:27.203933+0000","last_became_peered":"2024-02-26T04:55:27.203933+0000","last_unstale":"2024-02-26T04:57:21.177120+0000","last_undegraded":"2024-02-26T04:57:21.177120+0000","last_fullsized":"2024-02-26T04:57:21.177120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:19:36.247209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180188+0000","last_change":"2024-02-26T04:54:13.145088+0000","last_active":"2024-02-26T04:57:21.180188+0000","last_peered":"2024-02-26T04:57:21.180188+0000","last_clean":"2024-02-26T04:57:21.180188+0000","last_became_active":"2024-02-26T04:54:13.144657+0000","last_became_peered":"2024-02-26T04:54:13.144657+0000","last_unstale":"2024-02-26T04:57:21.180188+0000","last_undegraded":"2024-02-26T04:57:21.180188+0000","last_fullsized":"2024-02-26T04:57:21.180188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:50:45.246330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:50:45.246330+0000","last_clean_scrub_stamp":"2024-02-26T04:50:45.246330+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:00:50.842020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178258+0000","last_change":"2024-02-26T04:55:32.240124+0000","last_active":"2024-02-26T04:57:21.178258+0000","last_peered":"2024-02-26T04:57:21.178258+0000","last_clean":"2024-02-26T04:57:21.178258+0000","last_became_active":"2024-02-26T04:55:32.239624+0000","last_became_peered":"2024-02-26T04:55:32.239624+0000","last_unstale":"2024-02-26T04:57:21.178258+0000","last_undegraded":"2024-02-26T04:57:21.178258+0000","last_fullsized":"2024-02-26T04:57:21.178258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:40:33.770917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176056+0000","last_change":"2024-02-26T04:55:28.204631+0000","last_active":"2024-02-26T04:57:21.176056+0000","last_peered":"2024-02-26T04:57:21.176056+0000","last_clean":"2024-02-26T04:57:21.176056+0000","last_became_active":"2024-02-26T04:55:28.204274+0000","last_became_peered":"2024-02-26T04:55:28.204274+0000","last_unstale":"2024-02-26T04:57:21.176056+0000","last_undegraded":"2024-02-26T04:57:21.176056+0000","last_fullsized":"2024-02-26T04:57:21.176056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:18:18.336015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.187620+0000","last_change":"2024-02-26T04:57:21.187202+0000","last_active":"2024-02-26T04:57:21.187620+0000","last_peered":"2024-02-26T04:57:21.187620+0000","last_clean":"2024-02-26T04:57:21.187620+0000","last_became_active":"2024-02-26T04:57:21.183709+0000","last_became_peered":"2024-02-26T04:57:21.183709+0000","last_unstale":"2024-02-26T04:57:21.187620+0000","last_undegraded":"2024-02-26T04:57:21.187620+0000","last_fullsized":"2024-02-26T04:57:21.187620+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:05.733443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183832+0000","last_change":"2024-02-26T04:55:32.228900+0000","last_active":"2024-02-26T04:57:21.183832+0000","last_peered":"2024-02-26T04:57:21.183832+0000","last_clean":"2024-02-26T04:57:21.183832+0000","last_became_active":"2024-02-26T04:55:32.228262+0000","last_became_peered":"2024-02-26T04:55:32.228262+0000","last_unstale":"2024-02-26T04:57:21.183832+0000","last_undegraded":"2024-02-26T04:57:21.183832+0000","last_fullsized":"2024-02-26T04:57:21.183832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:33.069756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183858+0000","last_change":"2024-02-26T04:55:27.205264+0000","last_active":"2024-02-26T04:57:21.183858+0000","last_peered":"2024-02-26T04:57:21.183858+0000","last_clean":"2024-02-26T04:57:21.183858+0000","last_became_active":"2024-02-26T04:55:27.204882+0000","last_became_peered":"2024-02-26T04:55:27.204882+0000","last_unstale":"2024-02-26T04:57:21.183858+0000","last_undegraded":"2024-02-26T04:57:21.183858+0000","last_fullsized":"2024-02-26T04:57:21.183858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:14:19.497725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176445+0000","last_change":"2024-02-26T04:55:28.200264+0000","last_active":"2024-02-26T04:57:21.176445+0000","last_peered":"2024-02-26T04:57:21.176445+0000","last_clean":"2024-02-26T04:57:21.176445+0000","last_became_active":"2024-02-26T04:55:28.199913+0000","last_became_peered":"2024-02-26T04:55:28.199913+0000","last_unstale":"2024-02-26T04:57:21.176445+0000","last_undegraded":"2024-02-26T04:57:21.176445+0000","last_fullsized":"2024-02-26T04:57:21.176445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:54:51.529149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.185839+0000","last_change":"2024-02-26T04:55:32.233362+0000","last_active":"2024-02-26T04:57:21.185839+0000","last_peered":"2024-02-26T04:57:21.185839+0000","last_clean":"2024-02-26T04:57:21.185839+0000","last_became_active":"2024-02-26T04:55:32.232907+0000","last_became_peered":"2024-02-26T04:55:32.232907+0000","last_unstale":"2024-02-26T04:57:21.185839+0000","last_undegraded":"2024-02-26T04:57:21.185839+0000","last_fullsized":"2024-02-26T04:57:21.185839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:49:11.889863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'10","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:40.284901+0000","last_change":"2024-02-26T04:55:27.207371+0000","last_active":"2024-02-26T04:57:40.284901+0000","last_peered":"2024-02-26T04:57:40.284901+0000","last_clean":"2024-02-26T04:57:40.284901+0000","last_became_active":"2024-02-26T04:55:27.207047+0000","last_became_peered":"2024-02-26T04:55:27.207047+0000","last_unstale":"2024-02-26T04:57:40.284901+0000","last_undegraded":"2024-02-26T04:57:40.284901+0000","last_fullsized":"2024-02-26T04:57:40.284901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:48:29.575633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186642+0000","last_change":"2024-02-26T04:55:28.207132+0000","last_active":"2024-02-26T04:57:21.186642+0000","last_peered":"2024-02-26T04:57:21.186642+0000","last_clean":"2024-02-26T04:57:21.186642+0000","last_became_active":"2024-02-26T04:55:28.206376+0000","last_became_peered":"2024-02-26T04:55:28.206376+0000","last_unstale":"2024-02-26T04:57:21.186642+0000","last_undegraded":"2024-02-26T04:57:21.186642+0000","last_fullsized":"2024-02-26T04:57:21.186642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:17:53.802386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180403+0000","last_change":"2024-02-26T04:55:32.226381+0000","last_active":"2024-02-26T04:57:21.180403+0000","last_peered":"2024-02-26T04:57:21.180403+0000","last_clean":"2024-02-26T04:57:21.180403+0000","last_became_active":"2024-02-26T04:55:32.225940+0000","last_became_peered":"2024-02-26T04:55:32.225940+0000","last_unstale":"2024-02-26T04:57:21.180403+0000","last_undegraded":"2024-02-26T04:57:21.180403+0000","last_fullsized":"2024-02-26T04:57:21.180403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:51:20.190514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177325+0000","last_change":"2024-02-26T04:55:27.200246+0000","last_active":"2024-02-26T04:57:21.177325+0000","last_peered":"2024-02-26T04:57:21.177325+0000","last_clean":"2024-02-26T04:57:21.177325+0000","last_became_active":"2024-02-26T04:55:27.199906+0000","last_became_peered":"2024-02-26T04:55:27.199906+0000","last_unstale":"2024-02-26T04:57:21.177325+0000","last_undegraded":"2024-02-26T04:57:21.177325+0000","last_fullsized":"2024-02-26T04:57:21.177325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:29.037230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391466+0000","last_change":"2024-02-26T04:56:20.204222+0000","last_active":"2024-02-26T04:57:21.391466+0000","last_peered":"2024-02-26T04:57:21.391466+0000","last_clean":"2024-02-26T04:57:21.391466+0000","last_became_active":"2024-02-26T04:56:20.203726+0000","last_became_peered":"2024-02-26T04:56:20.203726+0000","last_unstale":"2024-02-26T04:57:21.391466+0000","last_undegraded":"2024-02-26T04:57:21.391466+0000","last_fullsized":"2024-02-26T04:57:21.391466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T09:37:59.371734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177174+0000","last_change":"2024-02-26T04:55:32.227765+0000","last_active":"2024-02-26T04:57:21.177174+0000","last_peered":"2024-02-26T04:57:21.177174+0000","last_clean":"2024-02-26T04:57:21.177174+0000","last_became_active":"2024-02-26T04:55:32.227346+0000","last_became_peered":"2024-02-26T04:55:32.227346+0000","last_unstale":"2024-02-26T04:57:21.177174+0000","last_undegraded":"2024-02-26T04:57:21.177174+0000","last_fullsized":"2024-02-26T04:57:21.177174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:05:12.634958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363549+0000","last_change":"2024-02-26T04:55:27.199979+0000","last_active":"2024-02-26T04:57:21.363549+0000","last_peered":"2024-02-26T04:57:21.363549+0000","last_clean":"2024-02-26T04:57:21.363549+0000","last_became_active":"2024-02-26T04:55:27.199618+0000","last_became_peered":"2024-02-26T04:55:27.199618+0000","last_unstale":"2024-02-26T04:57:21.363549+0000","last_undegraded":"2024-02-26T04:57:21.363549+0000","last_fullsized":"2024-02-26T04:57:21.363549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:20:59.649473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180646+0000","last_change":"2024-02-26T04:55:28.202253+0000","last_active":"2024-02-26T04:57:21.180646+0000","last_peered":"2024-02-26T04:57:21.180646+0000","last_clean":"2024-02-26T04:57:21.180646+0000","last_became_active":"2024-02-26T04:55:28.199825+0000","last_became_peered":"2024-02-26T04:55:28.199825+0000","last_unstale":"2024-02-26T04:57:21.180646+0000","last_undegraded":"2024-02-26T04:57:21.180646+0000","last_fullsized":"2024-02-26T04:57:21.180646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:21:59.148995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.188691+0000","last_change":"2024-02-26T04:57:21.188067+0000","last_active":"2024-02-26T04:57:21.188691+0000","last_peered":"2024-02-26T04:57:21.188691+0000","last_clean":"2024-02-26T04:57:21.188691+0000","last_became_active":"2024-02-26T04:57:21.187498+0000","last_became_peered":"2024-02-26T04:57:21.187498+0000","last_unstale":"2024-02-26T04:57:21.188691+0000","last_undegraded":"2024-02-26T04:57:21.188691+0000","last_fullsized":"2024-02-26T04:57:21.188691+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:00:12.060520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186148+0000","last_change":"2024-02-26T04:55:27.206529+0000","last_active":"2024-02-26T04:57:21.186148+0000","last_peered":"2024-02-26T04:57:21.186148+0000","last_clean":"2024-02-26T04:57:21.186148+0000","last_became_active":"2024-02-26T04:55:27.206203+0000","last_became_peered":"2024-02-26T04:55:27.206203+0000","last_unstale":"2024-02-26T04:57:21.186148+0000","last_undegraded":"2024-02-26T04:57:21.186148+0000","last_fullsized":"2024-02-26T04:57:21.186148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:24.491832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.184260+0000","last_change":"2024-02-26T04:55:28.205915+0000","last_active":"2024-02-26T04:57:21.184260+0000","last_peered":"2024-02-26T04:57:21.184260+0000","last_clean":"2024-02-26T04:57:21.184260+0000","last_became_active":"2024-02-26T04:55:28.205576+0000","last_became_peered":"2024-02-26T04:55:28.205576+0000","last_unstale":"2024-02-26T04:57:21.184260+0000","last_undegraded":"2024-02-26T04:57:21.184260+0000","last_fullsized":"2024-02-26T04:57:21.184260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:18:54.639778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177053+0000","last_change":"2024-02-26T04:55:32.226253+0000","last_active":"2024-02-26T04:57:21.177053+0000","last_peered":"2024-02-26T04:57:21.177053+0000","last_clean":"2024-02-26T04:57:21.177053+0000","last_became_active":"2024-02-26T04:55:32.225815+0000","last_became_peered":"2024-02-26T04:55:32.225815+0000","last_unstale":"2024-02-26T04:57:21.177053+0000","last_undegraded":"2024-02-26T04:57:21.177053+0000","last_fullsized":"2024-02-26T04:57:21.177053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:48:22.764756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.196831+0000","last_change":"2024-02-26T04:57:21.196391+0000","last_active":"2024-02-26T04:57:21.196831+0000","last_peered":"2024-02-26T04:57:21.196831+0000","last_clean":"2024-02-26T04:57:21.196831+0000","last_became_active":"2024-02-26T04:57:21.187476+0000","last_became_peered":"2024-02-26T04:57:21.187476+0000","last_unstale":"2024-02-26T04:57:21.196831+0000","last_undegraded":"2024-02-26T04:57:21.196831+0000","last_fullsized":"2024-02-26T04:57:21.196831+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:04:51.219911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186401+0000","last_change":"2024-02-26T04:55:28.213065+0000","last_active":"2024-02-26T04:57:21.186401+0000","last_peered":"2024-02-26T04:57:21.186401+0000","last_clean":"2024-02-26T04:57:21.186401+0000","last_became_active":"2024-02-26T04:55:28.211112+0000","last_became_peered":"2024-02-26T04:55:28.211112+0000","last_unstale":"2024-02-26T04:57:21.186401+0000","last_undegraded":"2024-02-26T04:57:21.186401+0000","last_fullsized":"2024-02-26T04:57:21.186401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:41:10.070183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177186+0000","last_change":"2024-02-26T04:55:32.229838+0000","last_active":"2024-02-26T04:57:21.177186+0000","last_peered":"2024-02-26T04:57:21.177186+0000","last_clean":"2024-02-26T04:57:21.177186+0000","last_became_active":"2024-02-26T04:55:32.228696+0000","last_became_peered":"2024-02-26T04:55:32.228696+0000","last_unstale":"2024-02-26T04:57:21.177186+0000","last_undegraded":"2024-02-26T04:57:21.177186+0000","last_fullsized":"2024-02-26T04:57:21.177186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:59:42.359301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180801+0000","last_change":"2024-02-26T04:55:27.205688+0000","last_active":"2024-02-26T04:57:21.180801+0000","last_peered":"2024-02-26T04:57:21.180801+0000","last_clean":"2024-02-26T04:57:21.180801+0000","last_became_active":"2024-02-26T04:55:27.205306+0000","last_became_peered":"2024-02-26T04:55:27.205306+0000","last_unstale":"2024-02-26T04:57:21.180801+0000","last_undegraded":"2024-02-26T04:57:21.180801+0000","last_fullsized":"2024-02-26T04:57:21.180801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:23:50.203565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180575+0000","last_change":"2024-02-26T04:55:28.210073+0000","last_active":"2024-02-26T04:57:21.180575+0000","last_peered":"2024-02-26T04:57:21.180575+0000","last_clean":"2024-02-26T04:57:21.180575+0000","last_became_active":"2024-02-26T04:55:28.208475+0000","last_became_peered":"2024-02-26T04:55:28.208475+0000","last_unstale":"2024-02-26T04:57:21.180575+0000","last_undegraded":"2024-02-26T04:57:21.180575+0000","last_fullsized":"2024-02-26T04:57:21.180575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:06:49.513916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363553+0000","last_change":"2024-02-26T04:55:32.228708+0000","last_active":"2024-02-26T04:57:21.363553+0000","last_peered":"2024-02-26T04:57:21.363553+0000","last_clean":"2024-02-26T04:57:21.363553+0000","last_became_active":"2024-02-26T04:55:32.228304+0000","last_became_peered":"2024-02-26T04:55:32.228304+0000","last_unstale":"2024-02-26T04:57:21.363553+0000","last_undegraded":"2024-02-26T04:57:21.363553+0000","last_fullsized":"2024-02-26T04:57:21.363553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:53:02.390712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181305+0000","last_change":"2024-02-26T04:55:27.199761+0000","last_active":"2024-02-26T04:57:21.181305+0000","last_peered":"2024-02-26T04:57:21.181305+0000","last_clean":"2024-02-26T04:57:21.181305+0000","last_became_active":"2024-02-26T04:55:27.199113+0000","last_became_peered":"2024-02-26T04:55:27.199113+0000","last_unstale":"2024-02-26T04:57:21.181305+0000","last_undegraded":"2024-02-26T04:57:21.181305+0000","last_fullsized":"2024-02-26T04:57:21.181305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:14:10.631886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180844+0000","last_change":"2024-02-26T04:55:28.211484+0000","last_active":"2024-02-26T04:57:21.180844+0000","last_peered":"2024-02-26T04:57:21.180844+0000","last_clean":"2024-02-26T04:57:21.180844+0000","last_became_active":"2024-02-26T04:55:28.211131+0000","last_became_peered":"2024-02-26T04:55:28.211131+0000","last_unstale":"2024-02-26T04:57:21.180844+0000","last_undegraded":"2024-02-26T04:57:21.180844+0000","last_fullsized":"2024-02-26T04:57:21.180844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:54:09.393847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391396+0000","last_change":"2024-02-26T04:55:32.227493+0000","last_active":"2024-02-26T04:57:21.391396+0000","last_peered":"2024-02-26T04:57:21.391396+0000","last_clean":"2024-02-26T04:57:21.391396+0000","last_became_active":"2024-02-26T04:55:32.227106+0000","last_became_peered":"2024-02-26T04:55:32.227106+0000","last_unstale":"2024-02-26T04:57:21.391396+0000","last_undegraded":"2024-02-26T04:57:21.391396+0000","last_fullsized":"2024-02-26T04:57:21.391396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:32:33.040966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.178567+0000","last_change":"2024-02-26T04:55:27.205331+0000","last_active":"2024-02-26T04:57:21.178567+0000","last_peered":"2024-02-26T04:57:21.178567+0000","last_clean":"2024-02-26T04:57:21.178567+0000","last_became_active":"2024-02-26T04:55:27.204998+0000","last_became_peered":"2024-02-26T04:55:27.204998+0000","last_unstale":"2024-02-26T04:57:21.178567+0000","last_undegraded":"2024-02-26T04:57:21.178567+0000","last_fullsized":"2024-02-26T04:57:21.178567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:10:03.671016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180697+0000","last_change":"2024-02-26T04:56:20.508854+0000","last_active":"2024-02-26T04:57:21.180697+0000","last_peered":"2024-02-26T04:57:21.180697+0000","last_clean":"2024-02-26T04:57:21.180697+0000","last_became_active":"2024-02-26T04:56:20.508396+0000","last_became_peered":"2024-02-26T04:56:20.508396+0000","last_unstale":"2024-02-26T04:57:21.180697+0000","last_undegraded":"2024-02-26T04:57:21.180697+0000","last_fullsized":"2024-02-26T04:57:21.180697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T05:47:33.249035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177528+0000","last_change":"2024-02-26T04:55:32.232222+0000","last_active":"2024-02-26T04:57:21.177528+0000","last_peered":"2024-02-26T04:57:21.177528+0000","last_clean":"2024-02-26T04:57:21.177528+0000","last_became_active":"2024-02-26T04:55:32.231836+0000","last_became_peered":"2024-02-26T04:55:32.231836+0000","last_unstale":"2024-02-26T04:57:21.177528+0000","last_undegraded":"2024-02-26T04:57:21.177528+0000","last_fullsized":"2024-02-26T04:57:21.177528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:31.202454+0000","last_clean_scrub_stamp":"2024-02-26T04:55:31.202454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:18:00.766266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176570+0000","last_change":"2024-02-26T04:55:27.200385+0000","last_active":"2024-02-26T04:57:21.176570+0000","last_peered":"2024-02-26T04:57:21.176570+0000","last_clean":"2024-02-26T04:57:21.176570+0000","last_became_active":"2024-02-26T04:55:27.200046+0000","last_became_peered":"2024-02-26T04:55:27.200046+0000","last_unstale":"2024-02-26T04:57:21.176570+0000","last_undegraded":"2024-02-26T04:57:21.176570+0000","last_fullsized":"2024-02-26T04:57:21.176570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:26.167224+0000","last_clean_scrub_stamp":"2024-02-26T04:55:26.167224+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:01:58.112457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181168+0000","last_change":"2024-02-26T04:55:28.220702+0000","last_active":"2024-02-26T04:57:21.181168+0000","last_peered":"2024-02-26T04:57:21.181168+0000","last_clean":"2024-02-26T04:57:21.181168+0000","last_became_active":"2024-02-26T04:55:28.220386+0000","last_became_peered":"2024-02-26T04:55:28.220386+0000","last_unstale":"2024-02-26T04:57:21.181168+0000","last_undegraded":"2024-02-26T04:57:21.181168+0000","last_fullsized":"2024-02-26T04:57:21.181168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:40:37.774831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183841+0000","last_change":"2024-02-26T04:55:28.206409+0000","last_active":"2024-02-26T04:57:21.183841+0000","last_peered":"2024-02-26T04:57:21.183841+0000","last_clean":"2024-02-26T04:57:21.183841+0000","last_became_active":"2024-02-26T04:55:28.206046+0000","last_became_peered":"2024-02-26T04:55:28.206046+0000","last_unstale":"2024-02-26T04:57:21.183841+0000","last_undegraded":"2024-02-26T04:57:21.183841+0000","last_fullsized":"2024-02-26T04:57:21.183841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:35:43.277376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181921+0000","last_change":"2024-02-26T04:55:28.216146+0000","last_active":"2024-02-26T04:57:21.181921+0000","last_peered":"2024-02-26T04:57:21.181921+0000","last_clean":"2024-02-26T04:57:21.181921+0000","last_became_active":"2024-02-26T04:55:28.215800+0000","last_became_peered":"2024-02-26T04:55:28.215800+0000","last_unstale":"2024-02-26T04:57:21.181921+0000","last_undegraded":"2024-02-26T04:57:21.181921+0000","last_fullsized":"2024-02-26T04:57:21.181921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T08:18:47.902694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.180868+0000","last_change":"2024-02-26T04:55:28.221587+0000","last_active":"2024-02-26T04:57:21.180868+0000","last_peered":"2024-02-26T04:57:21.180868+0000","last_clean":"2024-02-26T04:57:21.180868+0000","last_became_active":"2024-02-26T04:55:28.220990+0000","last_became_peered":"2024-02-26T04:55:28.220990+0000","last_unstale":"2024-02-26T04:57:21.180868+0000","last_undegraded":"2024-02-26T04:57:21.180868+0000","last_fullsized":"2024-02-26T04:57:21.180868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:24:13.302081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.391622+0000","last_change":"2024-02-26T04:55:28.220586+0000","last_active":"2024-02-26T04:57:21.391622+0000","last_peered":"2024-02-26T04:57:21.391622+0000","last_clean":"2024-02-26T04:57:21.391622+0000","last_became_active":"2024-02-26T04:55:28.219585+0000","last_became_peered":"2024-02-26T04:55:28.219585+0000","last_unstale":"2024-02-26T04:57:21.391622+0000","last_undegraded":"2024-02-26T04:57:21.391622+0000","last_fullsized":"2024-02-26T04:57:21.391622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:41.786142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176530+0000","last_change":"2024-02-26T04:55:28.204599+0000","last_active":"2024-02-26T04:57:21.176530+0000","last_peered":"2024-02-26T04:57:21.176530+0000","last_clean":"2024-02-26T04:57:21.176530+0000","last_became_active":"2024-02-26T04:55:28.204252+0000","last_became_peered":"2024-02-26T04:55:28.204252+0000","last_unstale":"2024-02-26T04:57:21.176530+0000","last_undegraded":"2024-02-26T04:57:21.176530+0000","last_fullsized":"2024-02-26T04:57:21.176530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T06:30:19.807897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.183609+0000","last_change":"2024-02-26T04:55:28.214233+0000","last_active":"2024-02-26T04:57:21.183609+0000","last_peered":"2024-02-26T04:57:21.183609+0000","last_clean":"2024-02-26T04:57:21.183609+0000","last_became_active":"2024-02-26T04:55:28.213927+0000","last_became_peered":"2024-02-26T04:55:28.213927+0000","last_unstale":"2024-02-26T04:57:21.183609+0000","last_undegraded":"2024-02-26T04:57:21.183609+0000","last_fullsized":"2024-02-26T04:57:21.183609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:31:07.967134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.181680+0000","last_change":"2024-02-26T04:55:28.217610+0000","last_active":"2024-02-26T04:57:21.181680+0000","last_peered":"2024-02-26T04:57:21.181680+0000","last_clean":"2024-02-26T04:57:21.181680+0000","last_became_active":"2024-02-26T04:55:28.217263+0000","last_became_peered":"2024-02-26T04:55:28.217263+0000","last_unstale":"2024-02-26T04:57:21.181680+0000","last_undegraded":"2024-02-26T04:57:21.181680+0000","last_fullsized":"2024-02-26T04:57:21.181680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:28:18.831887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.179617+0000","last_change":"2024-02-26T04:56:20.510859+0000","last_active":"2024-02-26T04:57:21.179617+0000","last_peered":"2024-02-26T04:57:21.179617+0000","last_clean":"2024-02-26T04:57:21.179617+0000","last_became_active":"2024-02-26T04:56:20.510420+0000","last_became_peered":"2024-02-26T04:56:20.510420+0000","last_unstale":"2024-02-26T04:57:21.179617+0000","last_undegraded":"2024-02-26T04:57:21.179617+0000","last_fullsized":"2024-02-26T04:57:21.179617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:30:02.961996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.177656+0000","last_change":"2024-02-26T04:55:28.217247+0000","last_active":"2024-02-26T04:57:21.177656+0000","last_peered":"2024-02-26T04:57:21.177656+0000","last_clean":"2024-02-26T04:57:21.177656+0000","last_became_active":"2024-02-26T04:55:28.214691+0000","last_became_peered":"2024-02-26T04:55:28.214691+0000","last_unstale":"2024-02-26T04:57:21.177656+0000","last_undegraded":"2024-02-26T04:57:21.177656+0000","last_fullsized":"2024-02-26T04:57:21.177656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:27:45.564520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.186211+0000","last_change":"2024-02-26T04:55:28.210491+0000","last_active":"2024-02-26T04:57:21.186211+0000","last_peered":"2024-02-26T04:57:21.186211+0000","last_clean":"2024-02-26T04:57:21.186211+0000","last_became_active":"2024-02-26T04:55:28.210144+0000","last_became_peered":"2024-02-26T04:55:28.210144+0000","last_unstale":"2024-02-26T04:57:21.186211+0000","last_undegraded":"2024-02-26T04:57:21.186211+0000","last_fullsized":"2024-02-26T04:57:21.186211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:11.214716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.363822+0000","last_change":"2024-02-26T04:55:28.216037+0000","last_active":"2024-02-26T04:57:21.363822+0000","last_peered":"2024-02-26T04:57:21.363822+0000","last_clean":"2024-02-26T04:57:21.363822+0000","last_became_active":"2024-02-26T04:55:28.215712+0000","last_became_peered":"2024-02-26T04:55:28.215712+0000","last_unstale":"2024-02-26T04:57:21.363822+0000","last_undegraded":"2024-02-26T04:57:21.363822+0000","last_fullsized":"2024-02-26T04:57:21.363822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:38:55.437642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.176653+0000","last_change":"2024-02-26T04:55:28.205469+0000","last_active":"2024-02-26T04:57:21.176653+0000","last_peered":"2024-02-26T04:57:21.176653+0000","last_clean":"2024-02-26T04:57:21.176653+0000","last_became_active":"2024-02-26T04:55:28.205137+0000","last_became_peered":"2024-02-26T04:55:28.205137+0000","last_unstale":"2024-02-26T04:57:21.176653+0000","last_undegraded":"2024-02-26T04:57:21.176653+0000","last_fullsized":"2024-02-26T04:57:21.176653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:50.425010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-26T04:57:21.182926+0000","last_change":"2024-02-26T04:55:28.213066+0000","last_active":"2024-02-26T04:57:21.182926+0000","last_peered":"2024-02-26T04:57:21.182926+0000","last_clean":"2024-02-26T04:57:21.182926+0000","last_became_active":"2024-02-26T04:55:28.212706+0000","last_became_peered":"2024-02-26T04:55:28.212706+0000","last_unstale":"2024-02-26T04:57:21.182926+0000","last_undegraded":"2024-02-26T04:57:21.182926+0000","last_fullsized":"2024-02-26T04:57:21.182926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T04:55:27.176470+0000","last_clean_scrub_stamp":"2024-02-26T04:55:27.176470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:21:24.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281347,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73204,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":755,"num_read_kb":453,"num_write":172,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":582,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":219612,"data_compressed":4869,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":520717,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.43099999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68100000000000005}]},{"osd":1,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.51500000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.45900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.56799999999999995}]},{"osd":2,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.33600000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.504,"15min":0.5},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":4,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.67800000000000005,"15min":0.67500000000000004},"min":{"1min":0.443,"5min":0.442,"15min":0.442},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.48299999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77000000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.41499999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84799999999999998,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.623},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91400000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59099999999999997}]},{"osd":6,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.56000000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1559999999999999,"5min":1.175,"15min":1.175},"last":0.72399999999999998}]},{"osd":7,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.74399999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.75,"15min":0.748},"min":{"1min":0.59299999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.88400000000000001}]},{"osd":8,"last update":"Mon Feb 26 04:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72299999999999998,"15min":0.71799999999999997},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.73899999999999999,"15min":0.73699999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.90000000000000002}]},{"osd":9,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.79900000000000004,"15min":0.79300000000000004},"min":{"1min":0.68100000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0369999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.80400000000000005,"15min":0.79800000000000004},"min":{"1min":0.622,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0960000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.81699999999999995}]},{"osd":10,"last update":"Mon Feb 26 04:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.63500000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.63200000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.86599999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":1176997,"data_compressed":63914,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.91800000000000004,"15min":0.98299999999999998},"min":{"1min":0.37,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.749,"5min":9.6609999999999996,"15min":9.6609999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.96599999999999997,"15min":1.0269999999999999},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90500000000000003,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":0.41899999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.499},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70799999999999996,"5min":0.871,"15min":0.871},"last":0.44700000000000001}]},{"osd":2,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.435,"5min":0.309,"15min":0.309},"max":{"1min":0.73199999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81699999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58899999999999997}]},{"osd":3,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.69999999999999996,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":8.3840000000000003,"5min":8.3840000000000003,"15min":8.3840000000000003},"last":0.501},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.67500000000000004,"15min":0.63100000000000001},"min":{"1min":0.38600000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":8.4160000000000004,"5min":8.4160000000000004,"15min":8.4160000000000004},"last":0.63200000000000001}]},{"osd":4,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56100000000000005,"15min":0.57599999999999996},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.77300000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56100000000000005,"15min":0.57299999999999995},"min":{"1min":0.35499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80300000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.75}]},{"osd":5,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68200000000000005,"15min":0.68700000000000006},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.87,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.52900000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.88400000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61799999999999999}]},{"osd":6,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":0.79900000000000004,"15min":0.753},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":9.7119999999999997,"5min":9.7119999999999997,"15min":9.7119999999999997},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.54400000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89800000000000002,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Mon Feb 26 04:55:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.72699999999999998}]},{"osd":8,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.751,"15min":0.76500000000000001},"min":{"1min":0.52900000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93500000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.79200000000000004,"15min":0.81399999999999995},"min":{"1min":0.52800000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94699999999999995,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.80000000000000004}]},{"osd":9,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":0.84299999999999997,"15min":0.80400000000000005},"min":{"1min":0.57099999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":8.3439999999999994,"5min":8.3439999999999994,"15min":8.3439999999999994},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81000000000000005,"15min":0.83299999999999996},"min":{"1min":0.55500000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97199999999999998,"5min":1.274,"15min":1.274},"last":0.83099999999999996}]},{"osd":11,"last update":"Mon Feb 26 04:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.63300000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91500000000000004,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78500000000000003,"15min":0.78800000000000003},"min":{"1min":0.58399999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88800000000000001,"5min":1.21,"15min":1.21},"last":0.76900000000000002}]}]},{"osd":9,"up_from":67,"seq":287762808886,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":524467,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42099999999999999,"15min":0.46700000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.51400000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.44600000000000001,"15min":0.48799999999999999},"min":{"1min":0.32600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65800000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.34899999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59099999999999997,"15min":0.623},"min":{"1min":0.433,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.59699999999999998,"15min":0.63800000000000001},"min":{"1min":0.36799999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77400000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63,"15min":0.66400000000000003},"min":{"1min":0.41599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.86399999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.68300000000000005,"15min":0.77700000000000002},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78200000000000003,"5min":3.173,"15min":3.173},"last":0.56000000000000005}]},{"osd":3,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47799999999999998,"15min":0.51600000000000001},"min":{"1min":0.31,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.442},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.48099999999999998,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.747,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.37}]},{"osd":4,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.67600000000000005},"min":{"1min":0.39500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78300000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.629,"15min":0.66200000000000003},"min":{"1min":0.45900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84099999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.41499999999999998}]},{"osd":5,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":0.79200000000000004,"15min":0.74099999999999999},"min":{"1min":0.46700000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":11.49,"5min":11.49,"15min":11.49},"last":0.49199999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":0.79000000000000004,"15min":0.751},"min":{"1min":0.41299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":11.470000000000001,"5min":11.470000000000001,"15min":11.470000000000001},"last":0.57599999999999996}]},{"osd":6,"last update":"Mon Feb 26 04:54:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69399999999999995,"15min":0.748},"min":{"1min":0.51800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92100000000000004,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.70199999999999996,"15min":0.76100000000000001},"min":{"1min":0.441,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83599999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.50900000000000001}]},{"osd":8,"last update":"Mon Feb 26 04:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.79600000000000004,"15min":0.90200000000000002},"min":{"1min":0.501,"5min":0.434,"15min":0.434},"max":{"1min":0.90200000000000002,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.79600000000000004},"min":{"1min":0.52300000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96299999999999997,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.58899999999999997}]},{"osd":10,"last update":"Mon Feb 26 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.74199999999999999,"15min":0.75800000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.89200000000000002,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73999999999999999,"15min":0.752},"min":{"1min":0.58699999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.872,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.54100000000000004}]},{"osd":11,"last update":"Mon Feb 26 04:57:35 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.73999999999999999,"15min":0.73399999999999999},"min":{"1min":0.628,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.61099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.61199999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070523,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":520291,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.49099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.84799999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.37},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.66600000000000004,"15min":0.57399999999999995},"min":{"1min":0.44900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.81699999999999995,"5min":12.359999999999999,"15min":12.359999999999999},"last":0.39800000000000002}]},{"osd":1,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78200000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.66000000000000003,"15min":0.626},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.87,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.502}]},{"osd":2,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.46800000000000003,"15min":0.44},"min":{"1min":0.42699999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.48699999999999999,"15min":0.44800000000000001},"min":{"1min":0.41099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73599999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55200000000000005}]},{"osd":3,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58699999999999997,"15min":0.56000000000000005},"min":{"1min":0.48499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.499,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76700000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.48299999999999998}]},{"osd":4,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64300000000000002,"15min":0.60899999999999999},"min":{"1min":0.45100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88100000000000001,"5min":1.2,"15min":1.2},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64400000000000002,"15min":0.621},"min":{"1min":0.51900000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89500000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.52400000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.59699999999999998,"15min":0.54900000000000004},"min":{"1min":0.55100000000000005,"5min":0.309,"15min":0.309},"max":{"1min":0.92500000000000004,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.59399999999999997,"15min":0.55200000000000005},"min":{"1min":0.47099999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.91300000000000003,"5min":1.27,"15min":1.27},"last":0.69199999999999995}]},{"osd":6,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63900000000000001,"15min":0.61499999999999999},"min":{"1min":0.53900000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.84299999999999997,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.84299999999999997,"15min":0.73099999999999998},"min":{"1min":0.66700000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97499999999999998,"5min":12.384,"15min":12.384},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.752,"15min":0.71399999999999997},"min":{"1min":0.72999999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.73999999999999999,"15min":0.68799999999999994},"min":{"1min":0.63800000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.97399999999999998,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.73299999999999998}]},{"osd":9,"last update":"Mon Feb 26 04:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.76600000000000001,"15min":0.71899999999999997},"min":{"1min":0.65100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.996,"5min":1.073,"15min":1.073},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.746,"15min":0.69299999999999995},"min":{"1min":0.65500000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94299999999999995,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.65500000000000003}]},{"osd":10,"last update":"Mon Feb 26 04:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.78000000000000003,"15min":0.748},"min":{"1min":0.67900000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0229999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.80700000000000005,"15min":0.77500000000000002},"min":{"1min":0.71499999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.84099999999999997}]},{"osd":11,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.72999999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0660000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.879},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.85299999999999998},"min":{"1min":0.66100000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.995,"5min":1.238,"15min":1.238},"last":0.82299999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1081344,"data_stored":1115535,"data_compressed":62291,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:54:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38400000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67000000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.40799999999999997}]},{"osd":1,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42399999999999999,"15min":0.41799999999999998},"min":{"1min":0.27900000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45700000000000002,"15min":0.443},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.58999999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.55000000000000004}]},{"osd":2,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82299999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84599999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72599999999999998}]},{"osd":3,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.49299999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79600000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81599999999999995,"5min":0.877,"15min":0.877},"last":0.69999999999999996}]},{"osd":4,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.30499999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.753,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.58699999999999997}]},{"osd":5,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.46500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86499999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66500000000000004,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.79200000000000004}]},{"osd":6,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.46200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.56200000000000006,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85499999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.83799999999999997}]},{"osd":8,"last update":"Mon Feb 26 04:57:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.53700000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93000000000000005,"5min":1.044,"15min":1.044},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72099999999999997,"15min":0.71399999999999997},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.89900000000000002,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.89700000000000002}]},{"osd":9,"last update":"Mon Feb 26 04:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.78000000000000003,"15min":0.74299999999999999},"min":{"1min":0.57699999999999996,"5min":0.504,"15min":0.504},"max":{"1min":4.7160000000000002,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.71899999999999997},"min":{"1min":0.59899999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.93100000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.93100000000000005}]},{"osd":10,"last update":"Mon Feb 26 04:57:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.747,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.48899999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.995,"5min":1.119,"15min":1.119},"last":0.86899999999999999}]},{"osd":11,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79600000000000004,"15min":0.79400000000000004},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95399999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.65300000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96799999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.88700000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":515547,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42799999999999999,"15min":0.39200000000000002},"min":{"1min":0.29299999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.434},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.433}]},{"osd":1,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64200000000000002,"15min":0.62},"min":{"1min":0.497,"5min":0.376,"15min":0.376},"max":{"1min":0.878,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.498},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.621,"15min":0.57899999999999996},"min":{"1min":0.40699999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85899999999999999,"5min":1.032,"15min":1.032},"last":0.59999999999999998}]},{"osd":2,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.627,"15min":0.58099999999999996},"min":{"1min":0.51800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.95399999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.751,"15min":1.0209999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94299999999999995,"5min":14.884,"15min":14.884},"last":0.57099999999999995}]},{"osd":3,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.48099999999999998,"15min":0.45600000000000002},"min":{"1min":0.33200000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71099999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54200000000000004,"15min":0.51800000000000002},"min":{"1min":0.38200000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.83199999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55300000000000005}]},{"osd":4,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66400000000000003,"15min":0.63700000000000001},"min":{"1min":0.441,"5min":0.38,"15min":0.38},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.79400000000000004,"15min":1.0720000000000001},"min":{"1min":0.45600000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.97699999999999998,"5min":14.898,"15min":14.898},"last":0.53500000000000003}]},{"osd":5,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68100000000000005,"15min":0.64700000000000002},"min":{"1min":0.50800000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90100000000000002,"5min":1.288,"15min":1.288},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70899999999999996,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0009999999999999,"5min":1.151,"15min":1.151},"last":0.61599999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76500000000000001,"15min":0.74199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71399999999999997,"15min":0.69199999999999995},"min":{"1min":0.501,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.044,"5min":1.075,"15min":1.075},"last":0.63600000000000001}]},{"osd":8,"last update":"Mon Feb 26 04:57:26 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":0.90200000000000002,"15min":0.81599999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":12.880000000000001,"5min":12.880000000000001,"15min":12.880000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.4570000000000001,"5min":0.90400000000000003,"15min":0.81200000000000006},"min":{"1min":0.58499999999999996,"5min":0.432,"15min":0.432},"max":{"1min":12.863,"5min":12.863,"15min":12.863},"last":0.69899999999999995}]},{"osd":9,"last update":"Mon Feb 26 04:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.745,"15min":0.79100000000000004},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96299999999999997,"5min":1.111,"15min":1.111},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.78500000000000003},"min":{"1min":0.437,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.008,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.72499999999999998}]},{"osd":10,"last update":"Mon Feb 26 04:57:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.84799999999999998,"15min":0.86899999999999999},"min":{"1min":0.622,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1200000000000001,"5min":1.133,"15min":1.133},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.58999999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0700000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.74299999999999999}]},{"osd":11,"last update":"Mon Feb 26 04:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81100000000000005,"15min":0.80900000000000005},"min":{"1min":0.68799999999999994,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.099,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83099999999999996,"15min":0.83399999999999996},"min":{"1min":0.67600000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1399999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.755}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":584743,"data_compressed":28131,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55400000000000005,"15min":0.502},"min":{"1min":0.36099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.625,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.54500000000000004,"15min":0.505},"min":{"1min":0.35799999999999998,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.70499999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.34899999999999998}]},{"osd":2,"last update":"Mon Feb 26 04:57:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.64400000000000002,"15min":0.55900000000000005},"min":{"1min":0.45900000000000002,"5min":0.371,"15min":0.35199999999999998},"max":{"1min":0.91600000000000004,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.55100000000000005},"min":{"1min":0.52100000000000002,"5min":0.35699999999999998,"15min":0.35599999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.56699999999999995}]},{"osd":3,"last update":"Mon Feb 26 04:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.55200000000000005},"min":{"1min":0.41199999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69999999999999996,"5min":0.997,"15min":0.997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63800000000000001,"15min":0.57899999999999996},"min":{"1min":0.45300000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73199999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.51600000000000001}]},{"osd":4,"last update":"Mon Feb 26 04:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.42799999999999999},"min":{"1min":0.34100000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.48699999999999999,"15min":0.435},"min":{"1min":0.36699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46600000000000003}]},{"osd":5,"last update":"Mon Feb 26 04:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64400000000000002,"15min":0.57199999999999995},"min":{"1min":0.44900000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85599999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63,"15min":0.56100000000000005},"min":{"1min":0.42799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97499999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.75700000000000001}]},{"osd":6,"last update":"Mon Feb 26 04:57:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.49099999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66500000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.71699999999999997,"15min":0.66600000000000004},"min":{"1min":0.60499999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.79800000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58399999999999996,"15min":0.55200000000000005},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88300000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75600000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54400000000000004}]},{"osd":8,"last update":"Mon Feb 26 04:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.748,"15min":0.71899999999999997},"min":{"1min":0.53600000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.90800000000000003,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.74199999999999999,"15min":0.71799999999999997},"min":{"1min":0.58399999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":1.04,"15min":1.04},"last":0.67900000000000005}]},{"osd":9,"last update":"Mon Feb 26 04:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1080000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.81100000000000005}]},{"osd":10,"last update":"Mon Feb 26 04:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.79400000000000004,"15min":0.79500000000000004},"min":{"1min":0.57099999999999995,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96499999999999997,"5min":1.095,"15min":1.095},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92300000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.72099999999999997}]},{"osd":11,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.89100000000000001,"15min":0.90300000000000002},"min":{"1min":0.625,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.93899999999999995,"5min":1.141,"15min":1.141},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.83999999999999997,"15min":0.84899999999999998},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.79800000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":516935,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 26 04:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.51300000000000001},"min":{"1min":0.38400000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83099999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.88700000000000001,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Mon Feb 26 04:56:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56899999999999995,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.33500000000000002,"15min":0.28999999999999998},"max":{"1min":0.81899999999999995,"5min":1.006,"15min":1.006},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60899999999999999,"15min":0.55700000000000005},"min":{"1min":0.45000000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.83799999999999997}]},{"osd":3,"last update":"Mon Feb 26 04:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.45700000000000002,"15min":0.46300000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.57799999999999996,"5min":1.208,"15min":1.208},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Feb 26 04:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65900000000000003,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.98399999999999999,"5min":1.073,"15min":1.073},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63100000000000001,"15min":0.56299999999999994},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.65200000000000002}]},{"osd":5,"last update":"Mon Feb 26 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60999999999999999,"15min":0.56200000000000006},"min":{"1min":0.42399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85199999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66600000000000004,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.89200000000000002}]},{"osd":6,"last update":"Mon Feb 26 04:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52500000000000002,"15min":0.51600000000000001},"min":{"1min":0.34100000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.754,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.54400000000000004},"min":{"1min":0.38,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80400000000000005,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Feb 26 04:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.54600000000000004,"5min":0.374,"15min":0.374},"max":{"1min":0.87,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.749,"15min":0.751},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93600000000000005,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.69599999999999995}]},{"osd":8,"last update":"Mon Feb 26 04:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.44900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96799999999999997,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73199999999999998,"15min":0.75},"min":{"1min":0.48299999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96599999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.78800000000000003}]},{"osd":9,"last update":"Mon Feb 26 04:56:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.46500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.80100000000000005},"min":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.51800000000000002}]},{"osd":10,"last update":"Mon Feb 26 04:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.83099999999999996,"15min":0.84099999999999997},"min":{"1min":0.61599999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0609999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.61799999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.036,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.73699999999999999}]},{"osd":11,"last update":"Mon Feb 26 04:57:38 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.65100000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.96899999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.75},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78400000000000003,"15min":0.78300000000000003},"min":{"1min":0.64800000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0349999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.82199999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":586093,"data_compressed":28131,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.53900000000000003,"15min":0.47299999999999998},"min":{"1min":0.41599999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.96399999999999997,"5min":1.002,"15min":1.002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53000000000000003,"15min":0.47399999999999998},"min":{"1min":0.378,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64700000000000002}]},{"osd":1,"last update":"Mon Feb 26 04:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.50600000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.57899999999999996,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.75900000000000001}]},{"osd":3,"last update":"Mon Feb 26 04:57:27 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.623,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.32000000000000001},"max":{"1min":0.93799999999999994,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.873},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.64300000000000002,"15min":0.58499999999999996},"min":{"1min":0.49299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94599999999999995,"5min":1.004,"15min":1.004},"last":0.94399999999999995}]},{"osd":4,"last update":"Mon Feb 26 04:57:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.75600000000000001,"15min":0.77000000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.85799999999999998,"5min":1.002,"15min":1.002},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.875,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.0940000000000001}]},{"osd":5,"last update":"Mon Feb 26 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.35199999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.48299999999999998,"15min":0.45100000000000001},"min":{"1min":0.35599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.71099999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.52400000000000002}]},{"osd":6,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63700000000000001,"15min":0.56899999999999995},"min":{"1min":0.51100000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.877,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.64000000000000001,"15min":0.57699999999999996},"min":{"1min":0.46899999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91400000000000003,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Mon Feb 26 04:57:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75900000000000001,"15min":0.77000000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91400000000000003,"5min":1.038,"15min":1.038},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.78700000000000003,"15min":0.80300000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":1.014,"15min":1.014},"last":1.044}]},{"osd":8,"last update":"Mon Feb 26 04:57:27 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.871,"5min":1.115,"15min":1.115},"last":0.621},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.35099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.88700000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.78200000000000003}]},{"osd":9,"last update":"Mon Feb 26 04:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70599999999999996,"15min":0.67600000000000005},"min":{"1min":0.52900000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.95899999999999996,"5min":1.018,"15min":1.018},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.748,"15min":0.72199999999999998},"min":{"1min":0.54100000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.96599999999999997,"5min":1.056,"15min":1.056},"last":0.82699999999999996}]},{"osd":10,"last update":"Mon Feb 26 04:57:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.70099999999999996},"min":{"1min":0.55600000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":1.127},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72999999999999998,"15min":0.69699999999999995},"min":{"1min":0.48199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96499999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.92700000000000005}]},{"osd":11,"last update":"Mon Feb 26 04:57:27 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.82499999999999996,"15min":0.83099999999999996},"min":{"1min":0.53400000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.008,"5min":1.109,"15min":1.109},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.80700000000000005,"15min":0.81200000000000006},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.89700000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":515741,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.34000000000000002},"min":{"1min":0.309,"5min":0.21199999999999999,"15min":0.17899999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.315},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.33800000000000002},"min":{"1min":0.28100000000000003,"5min":0.22700000000000001,"15min":0.20699999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.28999999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.52700000000000002},"min":{"1min":0.38,"5min":0.36599999999999999,"15min":0.35299999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57099999999999995,"15min":0.53100000000000003},"min":{"1min":0.40100000000000002,"5min":0.34499999999999997,"15min":0.32600000000000001},"max":{"1min":0.76200000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.61599999999999999}]},{"osd":2,"last update":"Mon Feb 26 04:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.42799999999999999,"5min":0.36699999999999999,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60199999999999998,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.32700000000000001},"max":{"1min":0.76300000000000001,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.59299999999999997}]},{"osd":4,"last update":"Mon Feb 26 04:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.48499999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82699999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64900000000000002,"15min":0.623},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.56100000000000005}]},{"osd":5,"last update":"Mon Feb 26 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.56899999999999995},"min":{"1min":0.40699999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.69499999999999995}]},{"osd":6,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50800000000000001,"15min":0.47999999999999998},"min":{"1min":0.33600000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.93899999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54100000000000004,"15min":0.51400000000000001},"min":{"1min":0.40300000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85599999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.36499999999999999}]},{"osd":7,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66600000000000004,"15min":0.629},"min":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95999999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.629}]},{"osd":8,"last update":"Mon Feb 26 04:55:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.874,"5min":0.995,"15min":0.995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.878,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63100000000000001}]},{"osd":9,"last update":"Mon Feb 26 04:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90500000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.44700000000000001}]},{"osd":10,"last update":"Mon Feb 26 04:57:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78000000000000003,"15min":0.79100000000000004},"min":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.038,"5min":1.232,"15min":1.232},"last":0.65700000000000003}]},{"osd":11,"last update":"Mon Feb 26 04:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.57899999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.751,"15min":0.75},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.67700000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":520505,"data_compressed":26508,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:54:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.51100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79500000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42899999999999999}]},{"osd":1,"last update":"Mon Feb 26 04:57:42 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.436,"15min":0.378},"min":{"1min":0.33600000000000002,"5min":0.26300000000000001,"15min":0.24299999999999999},"max":{"1min":0.58399999999999996,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.438,"15min":0.39200000000000002},"min":{"1min":0.36399999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.54200000000000004,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.621}]},{"osd":2,"last update":"Mon Feb 26 04:57:42 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58399999999999996,"15min":0.53400000000000003},"min":{"1min":0.442,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82199999999999995,"5min":1.657,"15min":1.657},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.60399999999999998,"15min":0.55200000000000005},"min":{"1min":0.496,"5min":0.40899999999999997,"15min":0.38},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.73399999999999999}]},{"osd":3,"last update":"Mon Feb 26 04:57:42 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61899999999999999,"15min":0.56200000000000006},"min":{"1min":0.41599999999999998,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.70499999999999996,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60499999999999998,"15min":0.54700000000000004},"min":{"1min":0.46300000000000002,"5min":0.36699999999999999,"15min":0.35599999999999998},"max":{"1min":0.81599999999999995,"5min":2.133,"15min":2.133},"last":0.59099999999999997}]},{"osd":5,"last update":"Mon Feb 26 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59299999999999997,"15min":0.55400000000000005},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89000000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.748},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63,"15min":0.56399999999999995},"min":{"1min":0.41799999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82199999999999995,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.84599999999999997}]},{"osd":6,"last update":"Mon Feb 26 04:57:40 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64000000000000001,"15min":0.58799999999999997},"min":{"1min":0.441,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75700000000000001,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.65200000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84499999999999997,"5min":4.1639999999999997,"15min":4.1639999999999997},"last":0.52100000000000002}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.55300000000000005},"min":{"1min":0.34200000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.55500000000000005},"min":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75900000000000001,"5min":4.2300000000000004,"15min":4.2300000000000004},"last":0.66400000000000003}]},{"osd":8,"last update":"Mon Feb 26 04:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71299999999999997,"15min":0.68600000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92700000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68500000000000005,"15min":0.64500000000000002},"min":{"1min":0.54400000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93200000000000005,"5min":1.907,"15min":1.907},"last":0.71899999999999997}]},{"osd":9,"last update":"Mon Feb 26 04:57:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.82899999999999996,"15min":0.749},"min":{"1min":0.55800000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.84199999999999997,"5min":10.051,"15min":10.051},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.82899999999999996,"15min":0.76500000000000001},"min":{"1min":0.58099999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94499999999999995,"5min":6.3360000000000003,"15min":6.3360000000000003},"last":0.76400000000000001}]},{"osd":10,"last update":"Mon Feb 26 04:57:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.78200000000000003,"15min":0.77800000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94799999999999995,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95499999999999996,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.70299999999999996}]},{"osd":11,"last update":"Mon Feb 26 04:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.97099999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82099999999999995,"15min":0.82899999999999996},"min":{"1min":0.59099999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.95899999999999996,"5min":1.946,"15min":1.946},"last":0.79900000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757322,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":1111397,"data_compressed":62291,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51300000000000001,"15min":0.49099999999999999},"min":{"1min":0.39600000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.32900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.60099999999999998}]},{"osd":1,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77400000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58599999999999997,"15min":0.53900000000000003},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85799999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61499999999999999}]},{"osd":2,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.41299999999999998,"15min":0.36899999999999999},"min":{"1min":0.30099999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.72199999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.42099999999999999,"15min":0.38700000000000001},"min":{"1min":0.32800000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.54800000000000004,"15min":0.52000000000000002},"min":{"1min":0.41399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.56899999999999995,"15min":0.51300000000000001},"min":{"1min":0.40100000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.8010000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":0.48899999999999999}]},{"osd":4,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63900000000000001,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83999999999999997,"5min":0.871,"15min":0.871},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58999999999999997,"15min":0.56000000000000005},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.82399999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65400000000000003}]},{"osd":6,"last update":"Mon Feb 26 04:57:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63500000000000001,"15min":0.59699999999999998},"min":{"1min":0.52300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91800000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63200000000000001,"15min":0.60999999999999999},"min":{"1min":0.51300000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80000000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.752}]},{"osd":7,"last update":"Mon Feb 26 04:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67400000000000004,"15min":0.65700000000000003},"min":{"1min":0.46500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89600000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70899999999999996}]},{"osd":8,"last update":"Mon Feb 26 04:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61099999999999999,"15min":0.59599999999999997},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58799999999999997,"15min":0.58599999999999997},"min":{"1min":0.36399999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.68600000000000005}]},{"osd":9,"last update":"Mon Feb 26 04:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69299999999999995,"15min":0.67600000000000005},"min":{"1min":0.59499999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.73199999999999998,"15min":0.69399999999999995},"min":{"1min":0.71899999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95899999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.82699999999999996}]},{"osd":10,"last update":"Mon Feb 26 04:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.48299999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.90900000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.81399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":69286,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70636,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71172,"data_compressed":1623,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T04:57:56.841 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:57:56.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.975+0000 7f8bcf236640 1 -- 172.21.15.60:0/3760858298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc81000a0 msgr2=0x7f8bc81004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.975+0000 7f8bcf236640 1 --2- 172.21.15.60:0/3760858298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc81000a0 0x7f8bc81004c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009ca0 tx=0x7f8bc002c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.975+0000 7f8bcf236640 1 -- 172.21.15.60:0/3760858298 shutdown_connections 2024-02-26T04:57:56.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.975+0000 7f8bcf236640 1 --2- 172.21.15.60:0/3760858298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bc8101c50 0x7f8bc81084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.975+0000 7f8bcf236640 1 --2- 172.21.15.60:0/3760858298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bc8101290 0x7f8bc8101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.975+0000 7f8bcf236640 1 --2- 172.21.15.60:0/3760858298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc81000a0 0x7f8bc81004c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.975+0000 7f8bcf236640 1 -- 172.21.15.60:0/3760858298 >> 172.21.15.60:0/3760858298 conn(0x7f8bc80fb610 msgr2=0x7f8bc80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:56.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.975+0000 7f8bcf236640 1 -- 172.21.15.60:0/3760858298 shutdown_connections 2024-02-26T04:57:56.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.975+0000 7f8bcf236640 1 -- 172.21.15.60:0/3760858298 wait complete. 2024-02-26T04:57:56.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bcf236640 1 Processor -- start 2024-02-26T04:57:56.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bcf236640 1 -- start start 2024-02-26T04:57:56.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bcf236640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bc81000a0 0x7f8bc8104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:56.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bcf236640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8101290 0x7f8bc8102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:56.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bcf236640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bc8101c50 0x7f8bc8102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:56.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bcf236640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc8103310 con 0x7f8bc8101290 2024-02-26T04:57:56.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bcf236640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc8103480 con 0x7f8bc81000a0 2024-02-26T04:57:56.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bcf236640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a12e0 con 0x7f8bc8101c50 2024-02-26T04:57:56.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8101290 0x7f8bc8102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:56.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8101290 0x7f8bc8102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44246/0 (socket says 172.21.15.60:44246) 2024-02-26T04:57:56.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bc7fff640 1 -- 172.21.15.60:0/1573212381 learned_addr learned my addr 172.21.15.60:0/1573212381 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:56.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bccfd2640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bc81000a0 0x7f8bc8104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:56.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bcd7d3640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bc8101c50 0x7f8bc8102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:56.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bc7fff640 1 -- 172.21.15.60:0/1573212381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bc8101c50 msgr2=0x7f8bc8102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bc7fff640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bc8101c50 0x7f8bc8102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bc7fff640 1 -- 172.21.15.60:0/1573212381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bc81000a0 msgr2=0x7f8bc8104030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:56.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bc7fff640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bc81000a0 0x7f8bc8104030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:56.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.976+0000 7f8bc7fff640 1 -- 172.21.15.60:0/1573212381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc0009900 con 0x7f8bc8101290 2024-02-26T04:57:56.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.977+0000 7f8bcd7d3640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bc8101c50 0x7f8bc8102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:56.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.977+0000 7f8bc7fff640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8101290 0x7f8bc8102670 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800d980 tx=0x7f8bb800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:56.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.977+0000 7f8bc5ffb640 1 -- 172.21.15.60:0/1573212381 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8009c30 con 0x7f8bc8101290 2024-02-26T04:57:56.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.977+0000 7f8bc5ffb640 1 -- 172.21.15.60:0/1573212381 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bb800b660 con 0x7f8bc8101290 2024-02-26T04:57:56.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.977+0000 7f8bc5ffb640 1 -- 172.21.15.60:0/1573212381 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb800f6b0 con 0x7f8bc8101290 2024-02-26T04:57:56.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.977+0000 7f8bcf236640 1 -- 172.21.15.60:0/1573212381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc81a1590 con 0x7f8bc8101290 2024-02-26T04:57:56.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.977+0000 7f8bcf236640 1 -- 172.21.15.60:0/1573212381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc81a1aa0 con 0x7f8bc8101290 2024-02-26T04:57:56.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.978+0000 7f8bcf236640 1 -- 172.21.15.60:0/1573212381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc810bab0 con 0x7f8bc8101290 2024-02-26T04:57:56.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.979+0000 7f8bc5ffb640 1 -- 172.21.15.60:0/1573212381 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8bb8010460 con 0x7f8bc8101290 2024-02-26T04:57:56.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.980+0000 7f8bc5ffb640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8bb40a20c0 0x7f8bb40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:56.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.980+0000 7f8bccfd2640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8bb40a20c0 0x7f8bb40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:56.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.980+0000 7f8bc5ffb640 1 -- 172.21.15.60:0/1573212381 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8bb80cddb0 con 0x7f8bc8101290 2024-02-26T04:57:56.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.980+0000 7f8bccfd2640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8bb40a20c0 0x7f8bb40a4580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8bc002c700 tx=0x7f8bc00326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:56.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:56.984+0000 7f8bc5ffb640 1 -- 172.21.15.60:0/1573212381 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8bb80d2070 con 0x7f8bc8101290 2024-02-26T04:57:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.166+0000 7f8bcf236640 1 -- 172.21.15.60:0/1573212381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bc810bc40 con 0x7f8bc8101290 2024-02-26T04:57:57.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.167+0000 7f8bc5ffb640 1 -- 172.21.15.60:0/1573212381 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24972 (secure 0 0 0) 0x7f8bb80968e0 con 0x7f8bc8101290 2024-02-26T04:57:57.170 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:57.170 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:57:21.168776+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:57:57.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.169+0000 7f8bcf236640 1 -- 172.21.15.60:0/1573212381 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8bb40a20c0 msgr2=0x7f8bb40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.169+0000 7f8bcf236640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8bb40a20c0 0x7f8bb40a4580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8bc002c700 tx=0x7f8bc00326b0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.169+0000 7f8bcf236640 1 -- 172.21.15.60:0/1573212381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8101290 msgr2=0x7f8bc8102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.169+0000 7f8bcf236640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8101290 0x7f8bc8102670 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800d980 tx=0x7f8bb800de40 comp rx=0 tx=0).stop 2024-02-26T04:57:57.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.169+0000 7f8bcf236640 1 -- 172.21.15.60:0/1573212381 shutdown_connections 2024-02-26T04:57:57.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.169+0000 7f8bcf236640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8bb40a20c0 0x7f8bb40a4580 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.169+0000 7f8bcf236640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bc8101c50 0x7f8bc8102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.169+0000 7f8bcf236640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc8101290 0x7f8bc8102670 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.169+0000 7f8bcf236640 1 --2- 172.21.15.60:0/1573212381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bc81000a0 0x7f8bc8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.169+0000 7f8bcf236640 1 -- 172.21.15.60:0/1573212381 >> 172.21.15.60:0/1573212381 conn(0x7f8bc80fb610 msgr2=0x7f8bc80fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:57.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.169+0000 7f8bcf236640 1 -- 172.21.15.60:0/1573212381 shutdown_connections 2024-02-26T04:57:57.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.169+0000 7f8bcf236640 1 -- 172.21.15.60:0/1573212381 wait complete. 2024-02-26T04:57:57.183 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T04:50:44.572921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T04:55:25.530720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T04:55:26.741604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T04:55:30.453569+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T04:57:57.184 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T04:57:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.287+0000 7fd94a4de640 1 -- 172.21.15.60:0/2727772618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd944100080 msgr2=0x7fd9441004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.287+0000 7fd94a4de640 1 --2- 172.21.15.60:0/2727772618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd944100080 0x7fd9441004a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd934009900 tx=0x7fd93402bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.287+0000 7fd94a4de640 1 -- 172.21.15.60:0/2727772618 shutdown_connections 2024-02-26T04:57:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.287+0000 7fd94a4de640 1 --2- 172.21.15.60:0/2727772618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd944101c30 0x7fd9441084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.287+0000 7fd94a4de640 1 --2- 172.21.15.60:0/2727772618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd944101270 0x7fd9441016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.287+0000 7fd94a4de640 1 --2- 172.21.15.60:0/2727772618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd944100080 0x7fd9441004a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.287+0000 7fd94a4de640 1 -- 172.21.15.60:0/2727772618 >> 172.21.15.60:0/2727772618 conn(0x7fd9440fb610 msgr2=0x7fd9440fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.287+0000 7fd94a4de640 1 -- 172.21.15.60:0/2727772618 shutdown_connections 2024-02-26T04:57:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.287+0000 7fd94a4de640 1 -- 172.21.15.60:0/2727772618 wait complete. 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.288+0000 7fd94a4de640 1 Processor -- start 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.288+0000 7fd94a4de640 1 -- start start 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.288+0000 7fd94a4de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd944100080 0x7fd944197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.288+0000 7fd94a4de640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd944101270 0x7fd944197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.288+0000 7fd94a4de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd944101c30 0x7fd94419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.288+0000 7fd94a4de640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94419ed20 con 0x7fd944101c30 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.288+0000 7fd94a4de640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94419ee90 con 0x7fd944101270 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.288+0000 7fd94a4de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94419f050 con 0x7fd944100080 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.288+0000 7fd948a7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd944101c30 0x7fd94419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.288+0000 7fd948a7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd944101c30 0x7fd94419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44268/0 (socket says 172.21.15.60:44268) 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.288+0000 7fd948a7b640 1 -- 172.21.15.60:0/1270990850 learned_addr learned my addr 172.21.15.60:0/1270990850 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.288+0000 7fd9437fe640 1 --2- 172.21.15.60:0/1270990850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd944101270 0x7fd944197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.288+0000 7fd943fff640 1 --2- 172.21.15.60:0/1270990850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd944100080 0x7fd944197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.289+0000 7fd948a7b640 1 -- 172.21.15.60:0/1270990850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd944100080 msgr2=0x7fd944197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.289+0000 7fd948a7b640 1 --2- 172.21.15.60:0/1270990850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd944100080 0x7fd944197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.289+0000 7fd948a7b640 1 -- 172.21.15.60:0/1270990850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd944101270 msgr2=0x7fd944197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.289+0000 7fd948a7b640 1 --2- 172.21.15.60:0/1270990850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd944101270 0x7fd944197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.289+0000 7fd948a7b640 1 -- 172.21.15.60:0/1270990850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd934009560 con 0x7fd944101c30 2024-02-26T04:57:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.289+0000 7fd948a7b640 1 --2- 172.21.15.60:0/1270990850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd944101c30 0x7fd94419e7e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00eaa0 tx=0x7fd93c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.289+0000 7fd9417fa640 1 -- 172.21.15.60:0/1270990850 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c00cdb0 con 0x7fd944101c30 2024-02-26T04:57:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.289+0000 7fd94a4de640 1 -- 172.21.15.60:0/1270990850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94419f330 con 0x7fd944101c30 2024-02-26T04:57:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.289+0000 7fd9417fa640 1 -- 172.21.15.60:0/1270990850 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd93c005020 con 0x7fd944101c30 2024-02-26T04:57:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.289+0000 7fd9417fa640 1 -- 172.21.15.60:0/1270990850 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0116d0 con 0x7fd944101c30 2024-02-26T04:57:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.289+0000 7fd94a4de640 1 -- 172.21.15.60:0/1270990850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94419f840 con 0x7fd944101c30 2024-02-26T04:57:57.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.290+0000 7fd94a4de640 1 -- 172.21.15.60:0/1270990850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9441073f0 con 0x7fd944101c30 2024-02-26T04:57:57.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.291+0000 7fd9417fa640 1 -- 172.21.15.60:0/1270990850 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd93c005490 con 0x7fd944101c30 2024-02-26T04:57:57.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.291+0000 7fd9417fa640 1 --2- 172.21.15.60:0/1270990850 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd92c0a20c0 0x7fd92c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:57.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.292+0000 7fd9417fa640 1 -- 172.21.15.60:0/1270990850 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fd93c0ce200 con 0x7fd944101c30 2024-02-26T04:57:57.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.292+0000 7fd943fff640 1 --2- 172.21.15.60:0/1270990850 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd92c0a20c0 0x7fd92c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:57.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.292+0000 7fd943fff640 1 --2- 172.21.15.60:0/1270990850 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd92c0a20c0 0x7fd92c0a4580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd934005ab0 tx=0x7fd934005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.296+0000 7fd9417fa640 1 -- 172.21.15.60:0/1270990850 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd93c096e40 con 0x7fd944101c30 2024-02-26T04:57:57.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:57 smithi060 ceph-mon[32382]: pgmap v372: 129 pgs: 129 active+clean; 648 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-26T04:57:57.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:57 smithi060 ceph-mon[32382]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:57.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:57 smithi060 ceph-mon[32382]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:57.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:57 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1573212381' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:57.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.478+0000 7fd94a4de640 1 -- 172.21.15.60:0/1270990850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd94402ce40 con 0x7fd944101c30 2024-02-26T04:57:57.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.478+0000 7fd9417fa640 1 -- 172.21.15.60:0/1270990850 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd93c01f0a0 con 0x7fd944101c30 2024-02-26T04:57:57.481 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T04:57:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.480+0000 7fd94a4de640 1 -- 172.21.15.60:0/1270990850 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd92c0a20c0 msgr2=0x7fd92c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.480+0000 7fd94a4de640 1 --2- 172.21.15.60:0/1270990850 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd92c0a20c0 0x7fd92c0a4580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd934005ab0 tx=0x7fd934005fd0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.480+0000 7fd94a4de640 1 -- 172.21.15.60:0/1270990850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd944101c30 msgr2=0x7fd94419e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.480+0000 7fd94a4de640 1 --2- 172.21.15.60:0/1270990850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd944101c30 0x7fd94419e7e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00eaa0 tx=0x7fd93c00ef60 comp rx=0 tx=0).stop 2024-02-26T04:57:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.480+0000 7fd94a4de640 1 -- 172.21.15.60:0/1270990850 shutdown_connections 2024-02-26T04:57:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.480+0000 7fd94a4de640 1 --2- 172.21.15.60:0/1270990850 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd92c0a20c0 0x7fd92c0a4580 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.480+0000 7fd94a4de640 1 --2- 172.21.15.60:0/1270990850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd944101c30 0x7fd94419e7e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.480+0000 7fd94a4de640 1 --2- 172.21.15.60:0/1270990850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd944101270 0x7fd944197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.480+0000 7fd94a4de640 1 --2- 172.21.15.60:0/1270990850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd944100080 0x7fd944197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.480+0000 7fd94a4de640 1 -- 172.21.15.60:0/1270990850 >> 172.21.15.60:0/1270990850 conn(0x7fd9440fb610 msgr2=0x7fd9441045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.480+0000 7fd94a4de640 1 -- 172.21.15.60:0/1270990850 shutdown_connections 2024-02-26T04:57:57.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.480+0000 7fd94a4de640 1 -- 172.21.15.60:0/1270990850 wait complete. 2024-02-26T04:57:57.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T04:57:57.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.597+0000 7f8273805640 1 -- 172.21.15.60:0/356328464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826c101290 msgr2=0x7f826c101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.597+0000 7f8273805640 1 --2- 172.21.15.60:0/356328464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826c101290 0x7f826c101710 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8260009900 tx=0x7f826002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:57.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.597+0000 7f8273805640 1 -- 172.21.15.60:0/356328464 shutdown_connections 2024-02-26T04:57:57.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.597+0000 7f8273805640 1 --2- 172.21.15.60:0/356328464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f826c101c50 0x7f826c1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.597+0000 7f8273805640 1 --2- 172.21.15.60:0/356328464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826c101290 0x7f826c101710 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.597+0000 7f8273805640 1 --2- 172.21.15.60:0/356328464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f826c1000a0 0x7f826c1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.597+0000 7f8273805640 1 -- 172.21.15.60:0/356328464 >> 172.21.15.60:0/356328464 conn(0x7f826c0fb630 msgr2=0x7f826c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.597+0000 7f8273805640 1 -- 172.21.15.60:0/356328464 shutdown_connections 2024-02-26T04:57:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.597+0000 7f8273805640 1 -- 172.21.15.60:0/356328464 wait complete. 2024-02-26T04:57:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.597+0000 7f8273805640 1 Processor -- start 2024-02-26T04:57:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.597+0000 7f8273805640 1 -- start start 2024-02-26T04:57:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8273805640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f826c1000a0 0x7f826c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8273805640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826c101290 0x7f826c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8273805640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f826c101c50 0x7f826c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8273805640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c19edd0 con 0x7f826c101290 2024-02-26T04:57:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8273805640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c19ef40 con 0x7f826c1000a0 2024-02-26T04:57:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8273805640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c19f100 con 0x7f826c101c50 2024-02-26T04:57:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8270da0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826c101290 0x7f826c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8270da0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826c101290 0x7f826c197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44288/0 (socket says 172.21.15.60:44288) 2024-02-26T04:57:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8270da0640 1 -- 172.21.15.60:0/105074700 learned_addr learned my addr 172.21.15.60:0/105074700 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8271da2640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f826c101c50 0x7f826c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f82715a1640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f826c1000a0 0x7f826c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8270da0640 1 -- 172.21.15.60:0/105074700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f826c101c50 msgr2=0x7f826c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8270da0640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f826c101c50 0x7f826c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8270da0640 1 -- 172.21.15.60:0/105074700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f826c1000a0 msgr2=0x7f826c1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8270da0640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f826c1000a0 0x7f826c1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8270da0640 1 -- 172.21.15.60:0/105074700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8260009560 con 0x7f826c101290 2024-02-26T04:57:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8271da2640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f826c101c50 0x7f826c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8270da0640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826c101290 0x7f826c197820 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f826002c840 tx=0x7f8260004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f826a7fc640 1 -- 172.21.15.60:0/105074700 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260004df0 con 0x7f826c101290 2024-02-26T04:57:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f826a7fc640 1 -- 172.21.15.60:0/105074700 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8260005430 con 0x7f826c101290 2024-02-26T04:57:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8273805640 1 -- 172.21.15.60:0/105074700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c19f380 con 0x7f826c101290 2024-02-26T04:57:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f826a7fc640 1 -- 172.21.15.60:0/105074700 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260003e60 con 0x7f826c101290 2024-02-26T04:57:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.598+0000 7f8273805640 1 -- 172.21.15.60:0/105074700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826c19f8e0 con 0x7f826c101290 2024-02-26T04:57:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.599+0000 7f8273805640 1 -- 172.21.15.60:0/105074700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826c10bab0 con 0x7f826c101290 2024-02-26T04:57:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.601+0000 7f826a7fc640 1 -- 172.21.15.60:0/105074700 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f82600035f0 con 0x7f826c101290 2024-02-26T04:57:57.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.601+0000 7f826a7fc640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f82540a2140 0x7f82540a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:57.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.601+0000 7f82715a1640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f82540a2140 0x7f82540a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:57.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.602+0000 7f826a7fc640 1 -- 172.21.15.60:0/105074700 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f82600f0bc0 con 0x7f826c101290 2024-02-26T04:57:57.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.602+0000 7f82715a1640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f82540a2140 0x7f82540a4600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8264005fd0 tx=0x7f8264005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:57.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.605+0000 7f826a7fc640 1 -- 172.21.15.60:0/105074700 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f82600b9750 con 0x7f826c101290 2024-02-26T04:57:57.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:57 smithi156 ceph-mon[35805]: pgmap v372: 129 pgs: 129 active+clean; 648 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-26T04:57:57.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:57 smithi156 ceph-mon[35805]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:57.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:57 smithi156 ceph-mon[35805]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:57.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:57 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1573212381' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:57.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:57 smithi191 ceph-mon[36352]: pgmap v372: 129 pgs: 129 active+clean; 648 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-02-26T04:57:57.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:57 smithi191 ceph-mon[36352]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:57.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:57 smithi191 ceph-mon[36352]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-26T04:57:57.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:57 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1573212381' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:57.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.784+0000 7f8273805640 1 -- 172.21.15.60:0/105074700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f826c19fb00 con 0x7f826c101290 2024-02-26T04:57:57.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.785+0000 7f826a7fc640 1 -- 172.21.15.60:0/105074700 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f82600bce10 con 0x7f826c101290 2024-02-26T04:57:57.787 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:57:57.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.787+0000 7f8273805640 1 -- 172.21.15.60:0/105074700 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f82540a2140 msgr2=0x7f82540a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.787+0000 7f8273805640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f82540a2140 0x7f82540a4600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8264005fd0 tx=0x7f8264005850 comp rx=0 tx=0).stop 2024-02-26T04:57:57.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.787+0000 7f8273805640 1 -- 172.21.15.60:0/105074700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826c101290 msgr2=0x7f826c197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.787+0000 7f8273805640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826c101290 0x7f826c197820 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f826002c840 tx=0x7f8260004270 comp rx=0 tx=0).stop 2024-02-26T04:57:57.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.787+0000 7f8273805640 1 -- 172.21.15.60:0/105074700 shutdown_connections 2024-02-26T04:57:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.787+0000 7f8273805640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f82540a2140 0x7f82540a4600 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.787+0000 7f8273805640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f826c101c50 0x7f826c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.787+0000 7f8273805640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826c101290 0x7f826c197820 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.787+0000 7f8273805640 1 --2- 172.21.15.60:0/105074700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f826c1000a0 0x7f826c1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.787+0000 7f8273805640 1 -- 172.21.15.60:0/105074700 >> 172.21.15.60:0/105074700 conn(0x7f826c0fb630 msgr2=0x7f826c0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.787+0000 7f8273805640 1 -- 172.21.15.60:0/105074700 shutdown_connections 2024-02-26T04:57:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.787+0000 7f8273805640 1 -- 172.21.15.60:0/105074700 wait complete. 2024-02-26T04:57:57.800 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T04:57:57.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.903+0000 7f9313d5e640 1 -- 172.21.15.60:0/2703256506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c102380 msgr2=0x7f930c102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.903+0000 7f9313d5e640 1 --2- 172.21.15.60:0/2703256506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c102380 0x7f930c102800 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009900 tx=0x7f92fc02bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:57.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.903+0000 7f9313d5e640 1 -- 172.21.15.60:0/2703256506 shutdown_connections 2024-02-26T04:57:57.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.903+0000 7f9313d5e640 1 --2- 172.21.15.60:0/2703256506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f930c102d40 0x7f930c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.903+0000 7f9313d5e640 1 --2- 172.21.15.60:0/2703256506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c102380 0x7f930c102800 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.903+0000 7f9313d5e640 1 --2- 172.21.15.60:0/2703256506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f930c101190 0x7f930c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.903+0000 7f9313d5e640 1 -- 172.21.15.60:0/2703256506 >> 172.21.15.60:0/2703256506 conn(0x7f930c0fc720 msgr2=0x7f930c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:57.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.903+0000 7f9313d5e640 1 -- 172.21.15.60:0/2703256506 shutdown_connections 2024-02-26T04:57:57.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.903+0000 7f9313d5e640 1 -- 172.21.15.60:0/2703256506 wait complete. 2024-02-26T04:57:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f9313d5e640 1 Processor -- start 2024-02-26T04:57:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f9313d5e640 1 -- start start 2024-02-26T04:57:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f9313d5e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f930c101190 0x7f930c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f9313d5e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f930c102380 0x7f930c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f9313d5e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c102d40 0x7f930c1a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f9313d5e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c1a0f20 con 0x7f930c102d40 2024-02-26T04:57:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f9313d5e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c1a1090 con 0x7f930c101190 2024-02-26T04:57:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f9313d5e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c1a1250 con 0x7f930c102380 2024-02-26T04:57:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f93122fb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c102d40 0x7f930c1a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f93122fb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c102d40 0x7f930c1a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44306/0 (socket says 172.21.15.60:44306) 2024-02-26T04:57:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f93122fb640 1 -- 172.21.15.60:0/3561210448 learned_addr learned my addr 172.21.15.60:0/3561210448 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f93112f9640 1 --2- 172.21.15.60:0/3561210448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f930c102380 0x7f930c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f9311afa640 1 --2- 172.21.15.60:0/3561210448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f930c101190 0x7f930c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f93122fb640 1 -- 172.21.15.60:0/3561210448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f930c102380 msgr2=0x7f930c199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f93122fb640 1 --2- 172.21.15.60:0/3561210448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f930c102380 0x7f930c199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f93122fb640 1 -- 172.21.15.60:0/3561210448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f930c101190 msgr2=0x7f930c199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f93122fb640 1 --2- 172.21.15.60:0/3561210448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f930c101190 0x7f930c199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f93122fb640 1 -- 172.21.15.60:0/3561210448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc009560 con 0x7f930c102d40 2024-02-26T04:57:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.904+0000 7f93122fb640 1 --2- 172.21.15.60:0/3561210448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c102d40 0x7f930c1a09e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f930400eaa0 tx=0x7f930400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.905+0000 7f9302ffd640 1 -- 172.21.15.60:0/3561210448 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930400cdb0 con 0x7f930c102d40 2024-02-26T04:57:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.905+0000 7f9302ffd640 1 -- 172.21.15.60:0/3561210448 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9304005020 con 0x7f930c102d40 2024-02-26T04:57:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.905+0000 7f9313d5e640 1 -- 172.21.15.60:0/3561210448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930c1a1530 con 0x7f930c102d40 2024-02-26T04:57:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.905+0000 7f9302ffd640 1 -- 172.21.15.60:0/3561210448 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93040116b0 con 0x7f930c102d40 2024-02-26T04:57:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.905+0000 7f9313d5e640 1 -- 172.21.15.60:0/3561210448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930c1a1a10 con 0x7f930c102d40 2024-02-26T04:57:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.905+0000 7f9313d5e640 1 -- 172.21.15.60:0/3561210448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930c10cba0 con 0x7f930c102d40 2024-02-26T04:57:57.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.907+0000 7f9302ffd640 1 -- 172.21.15.60:0/3561210448 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9304005520 con 0x7f930c102d40 2024-02-26T04:57:57.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.907+0000 7f9302ffd640 1 --2- 172.21.15.60:0/3561210448 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f92f80a20c0 0x7f92f80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:57.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.907+0000 7f9311afa640 1 --2- 172.21.15.60:0/3561210448 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f92f80a20c0 0x7f92f80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:57.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.907+0000 7f9302ffd640 1 -- 172.21.15.60:0/3561210448 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f93040ce5d0 con 0x7f930c102d40 2024-02-26T04:57:57.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.908+0000 7f9311afa640 1 --2- 172.21.15.60:0/3561210448 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f92f80a20c0 0x7f92f80a4580 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f930c1021e0 tx=0x7f930800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:57.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:57.911+0000 7f9302ffd640 1 -- 172.21.15.60:0/3561210448 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9304097210 con 0x7f930c102d40 2024-02-26T04:57:58.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.091+0000 7f9313d5e640 1 -- 172.21.15.60:0/3561210448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f930c1a1c90 con 0x7f930c102d40 2024-02-26T04:57:58.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.092+0000 7f9302ffd640 1 -- 172.21.15.60:0/3561210448 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f930c1a1c90 con 0x7f930c102d40 2024-02-26T04:57:58.094 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T04:57:58.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.094+0000 7f9313d5e640 1 -- 172.21.15.60:0/3561210448 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f92f80a20c0 msgr2=0x7f92f80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.094+0000 7f9313d5e640 1 --2- 172.21.15.60:0/3561210448 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f92f80a20c0 0x7f92f80a4580 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f930c1021e0 tx=0x7f930800b390 comp rx=0 tx=0).stop 2024-02-26T04:57:58.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.094+0000 7f9313d5e640 1 -- 172.21.15.60:0/3561210448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c102d40 msgr2=0x7f930c1a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.094+0000 7f9313d5e640 1 --2- 172.21.15.60:0/3561210448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c102d40 0x7f930c1a09e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f930400eaa0 tx=0x7f930400ef60 comp rx=0 tx=0).stop 2024-02-26T04:57:58.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.094+0000 7f9313d5e640 1 -- 172.21.15.60:0/3561210448 shutdown_connections 2024-02-26T04:57:58.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.094+0000 7f9313d5e640 1 --2- 172.21.15.60:0/3561210448 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f92f80a20c0 0x7f92f80a4580 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.094+0000 7f9313d5e640 1 --2- 172.21.15.60:0/3561210448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c102d40 0x7f930c1a09e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.094+0000 7f9313d5e640 1 --2- 172.21.15.60:0/3561210448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f930c102380 0x7f930c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.094+0000 7f9313d5e640 1 --2- 172.21.15.60:0/3561210448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f930c101190 0x7f930c199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.094+0000 7f9313d5e640 1 -- 172.21.15.60:0/3561210448 >> 172.21.15.60:0/3561210448 conn(0x7f930c0fc720 msgr2=0x7f930c0733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:58.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.094+0000 7f9313d5e640 1 -- 172.21.15.60:0/3561210448 shutdown_connections 2024-02-26T04:57:58.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.094+0000 7f9313d5e640 1 -- 172.21.15.60:0/3561210448 wait complete. 2024-02-26T04:57:58.107 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T04:57:58.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.212+0000 7f1c400de640 1 -- 172.21.15.60:0/409470077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c38101280 msgr2=0x7f1c38101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.212+0000 7f1c400de640 1 --2- 172.21.15.60:0/409470077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c38101280 0x7f1c38101700 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009900 tx=0x7f1c2802bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.212+0000 7f1c400de640 1 -- 172.21.15.60:0/409470077 shutdown_connections 2024-02-26T04:57:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.212+0000 7f1c400de640 1 --2- 172.21.15.60:0/409470077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c38101c40 0x7f1c381084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.212+0000 7f1c400de640 1 --2- 172.21.15.60:0/409470077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c38101280 0x7f1c38101700 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.212+0000 7f1c400de640 1 --2- 172.21.15.60:0/409470077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c38100090 0x7f1c381004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.212+0000 7f1c400de640 1 -- 172.21.15.60:0/409470077 >> 172.21.15.60:0/409470077 conn(0x7f1c380fb620 msgr2=0x7f1c380fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.212+0000 7f1c400de640 1 -- 172.21.15.60:0/409470077 shutdown_connections 2024-02-26T04:57:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.212+0000 7f1c400de640 1 -- 172.21.15.60:0/409470077 wait complete. 2024-02-26T04:57:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.212+0000 7f1c400de640 1 Processor -- start 2024-02-26T04:57:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.212+0000 7f1c400de640 1 -- start start 2024-02-26T04:57:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c400de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c38100090 0x7f1c38197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c400de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c38101280 0x7f1c38197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c400de640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c38101c40 0x7f1c3819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c400de640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3819ed00 con 0x7f1c38101280 2024-02-26T04:57:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c400de640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3819ee70 con 0x7f1c38101c40 2024-02-26T04:57:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c400de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3819f030 con 0x7f1c38100090 2024-02-26T04:57:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c3d679640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c38101280 0x7f1c38197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c3d679640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c38101280 0x7f1c38197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44320/0 (socket says 172.21.15.60:44320) 2024-02-26T04:57:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c3d679640 1 -- 172.21.15.60:0/2039249240 learned_addr learned my addr 172.21.15.60:0/2039249240 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c3e67b640 1 --2- 172.21.15.60:0/2039249240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c38101c40 0x7f1c3819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c3de7a640 1 --2- 172.21.15.60:0/2039249240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c38100090 0x7f1c38197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c3d679640 1 -- 172.21.15.60:0/2039249240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c38100090 msgr2=0x7f1c38197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c3d679640 1 --2- 172.21.15.60:0/2039249240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c38100090 0x7f1c38197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c3d679640 1 -- 172.21.15.60:0/2039249240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c38101c40 msgr2=0x7f1c3819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c3d679640 1 --2- 172.21.15.60:0/2039249240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c38101c40 0x7f1c3819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c3d679640 1 -- 172.21.15.60:0/2039249240 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c28009560 con 0x7f1c38101280 2024-02-26T04:57:58.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c3d679640 1 --2- 172.21.15.60:0/2039249240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c38101280 0x7f1c38197750 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1c28006010 tx=0x7f1c280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:58.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c2effd640 1 -- 172.21.15.60:0/2039249240 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c280044e0 con 0x7f1c38101280 2024-02-26T04:57:58.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c400de640 1 -- 172.21.15.60:0/2039249240 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3819f2b0 con 0x7f1c38101280 2024-02-26T04:57:58.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c2effd640 1 -- 172.21.15.60:0/2039249240 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c280054e0 con 0x7f1c38101280 2024-02-26T04:57:58.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c2effd640 1 -- 172.21.15.60:0/2039249240 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2803e450 con 0x7f1c38101280 2024-02-26T04:57:58.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.213+0000 7f1c400de640 1 -- 172.21.15.60:0/2039249240 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3819f810 con 0x7f1c38101280 2024-02-26T04:57:58.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.214+0000 7f1c400de640 1 -- 172.21.15.60:0/2039249240 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3810baa0 con 0x7f1c38101280 2024-02-26T04:57:58.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.215+0000 7f1c2effd640 1 -- 172.21.15.60:0/2039249240 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1c280035f0 con 0x7f1c38101280 2024-02-26T04:57:58.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.216+0000 7f1c2effd640 1 --2- 172.21.15.60:0/2039249240 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1c240a20c0 0x7f1c240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:58.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.216+0000 7f1c3de7a640 1 --2- 172.21.15.60:0/2039249240 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1c240a20c0 0x7f1c240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:58.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.216+0000 7f1c2effd640 1 -- 172.21.15.60:0/2039249240 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f1c280f03f0 con 0x7f1c38101280 2024-02-26T04:57:58.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.216+0000 7f1c3de7a640 1 --2- 172.21.15.60:0/2039249240 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1c240a20c0 0x7f1c240a4580 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1c34005db0 tx=0x7f1c34005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:58.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.220+0000 7f1c2effd640 1 -- 172.21.15.60:0/2039249240 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1c280b9020 con 0x7f1c38101280 2024-02-26T04:57:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.401+0000 7f1c400de640 1 -- 172.21.15.60:0/2039249240 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1c3819fc40 con 0x7f1c38101280 2024-02-26T04:57:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.402+0000 7f1c2effd640 1 -- 172.21.15.60:0/2039249240 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1c280bc6e0 con 0x7f1c38101280 2024-02-26T04:57:58.404 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:57:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.404+0000 7f1c400de640 1 -- 172.21.15.60:0/2039249240 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1c240a20c0 msgr2=0x7f1c240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.404+0000 7f1c400de640 1 --2- 172.21.15.60:0/2039249240 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1c240a20c0 0x7f1c240a4580 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1c34005db0 tx=0x7f1c34005d40 comp rx=0 tx=0).stop 2024-02-26T04:57:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.404+0000 7f1c400de640 1 -- 172.21.15.60:0/2039249240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c38101280 msgr2=0x7f1c38197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.404+0000 7f1c400de640 1 --2- 172.21.15.60:0/2039249240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c38101280 0x7f1c38197750 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1c28006010 tx=0x7f1c280042f0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.404+0000 7f1c400de640 1 -- 172.21.15.60:0/2039249240 shutdown_connections 2024-02-26T04:57:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.404+0000 7f1c400de640 1 --2- 172.21.15.60:0/2039249240 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1c240a20c0 0x7f1c240a4580 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.404+0000 7f1c400de640 1 --2- 172.21.15.60:0/2039249240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c38101c40 0x7f1c3819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.404+0000 7f1c400de640 1 --2- 172.21.15.60:0/2039249240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c38101280 0x7f1c38197750 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.404+0000 7f1c400de640 1 --2- 172.21.15.60:0/2039249240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c38100090 0x7f1c38197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.404+0000 7f1c400de640 1 -- 172.21.15.60:0/2039249240 >> 172.21.15.60:0/2039249240 conn(0x7f1c380fb620 msgr2=0x7f1c380fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.404+0000 7f1c400de640 1 -- 172.21.15.60:0/2039249240 shutdown_connections 2024-02-26T04:57:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.404+0000 7f1c400de640 1 -- 172.21.15.60:0/2039249240 wait complete. 2024-02-26T04:57:58.416 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:58 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1270990850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:58.416 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:58 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/105074700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:58.416 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:58 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3561210448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:58.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:57:58.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.518+0000 7f5bf6236640 1 -- 172.21.15.60:0/2060984930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bf0102d50 msgr2=0x7f5bf01095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.518+0000 7f5bf6236640 1 --2- 172.21.15.60:0/2060984930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bf0102d50 0x7f5bf01095e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009ca0 tx=0x7f5be802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.518+0000 7f5bf6236640 1 -- 172.21.15.60:0/2060984930 shutdown_connections 2024-02-26T04:57:58.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.518+0000 7f5bf6236640 1 --2- 172.21.15.60:0/2060984930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bf0102d50 0x7f5bf01095e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.518+0000 7f5bf6236640 1 --2- 172.21.15.60:0/2060984930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf0102390 0x7f5bf0102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.518+0000 7f5bf6236640 1 --2- 172.21.15.60:0/2060984930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf01011a0 0x7f5bf01015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.518+0000 7f5bf6236640 1 -- 172.21.15.60:0/2060984930 >> 172.21.15.60:0/2060984930 conn(0x7f5bf00fc730 msgr2=0x7f5bf00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:58.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.518+0000 7f5bf6236640 1 -- 172.21.15.60:0/2060984930 shutdown_connections 2024-02-26T04:57:58.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.518+0000 7f5bf6236640 1 -- 172.21.15.60:0/2060984930 wait complete. 2024-02-26T04:57:58.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5bf6236640 1 Processor -- start 2024-02-26T04:57:58.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5bf6236640 1 -- start start 2024-02-26T04:57:58.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5bf6236640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf01011a0 0x7f5bf01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:58.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5bf6236640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bf0102390 0x7f5bf01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:58.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5bf6236640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf0102d50 0x7f5bf01a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:58.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5bf6236640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf01a0e90 con 0x7f5bf0102390 2024-02-26T04:57:58.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5bf6236640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf01a1000 con 0x7f5bf0102d50 2024-02-26T04:57:58.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5bf6236640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf01a11c0 con 0x7f5bf01011a0 2024-02-26T04:57:58.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5beeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bf0102390 0x7f5bf01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:58.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5beeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bf0102390 0x7f5bf01998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44336/0 (socket says 172.21.15.60:44336) 2024-02-26T04:57:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5beeffd640 1 -- 172.21.15.60:0/1155616948 learned_addr learned my addr 172.21.15.60:0/1155616948 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5bef7fe640 1 --2- 172.21.15.60:0/1155616948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf01011a0 0x7f5bf01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5beffff640 1 --2- 172.21.15.60:0/1155616948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf0102d50 0x7f5bf01a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5beeffd640 1 -- 172.21.15.60:0/1155616948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf01011a0 msgr2=0x7f5bf01993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5beeffd640 1 --2- 172.21.15.60:0/1155616948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf01011a0 0x7f5bf01993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.519+0000 7f5beeffd640 1 -- 172.21.15.60:0/1155616948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf0102d50 msgr2=0x7f5bf01a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.520+0000 7f5beeffd640 1 --2- 172.21.15.60:0/1155616948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf0102d50 0x7f5bf01a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.520+0000 7f5beeffd640 1 -- 172.21.15.60:0/1155616948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be8009900 con 0x7f5bf0102390 2024-02-26T04:57:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.520+0000 7f5beeffd640 1 --2- 172.21.15.60:0/1155616948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bf0102390 0x7f5bf01998e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5be400c920 tx=0x7f5be400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.520+0000 7f5becff9640 1 -- 172.21.15.60:0/1155616948 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4007c70 con 0x7f5bf0102390 2024-02-26T04:57:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.520+0000 7f5bf6236640 1 -- 172.21.15.60:0/1155616948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf01a14a0 con 0x7f5bf0102390 2024-02-26T04:57:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.520+0000 7f5becff9640 1 -- 172.21.15.60:0/1155616948 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5be4007dd0 con 0x7f5bf0102390 2024-02-26T04:57:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.520+0000 7f5becff9640 1 -- 172.21.15.60:0/1155616948 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4018860 con 0x7f5bf0102390 2024-02-26T04:57:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.520+0000 7f5bf6236640 1 -- 172.21.15.60:0/1155616948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf01a19b0 con 0x7f5bf0102390 2024-02-26T04:57:58.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.521+0000 7f5bf6236640 1 -- 172.21.15.60:0/1155616948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf010cbb0 con 0x7f5bf0102390 2024-02-26T04:57:58.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.522+0000 7f5becff9640 1 -- 172.21.15.60:0/1155616948 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5be400fac0 con 0x7f5bf0102390 2024-02-26T04:57:58.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.522+0000 7f5becff9640 1 --2- 172.21.15.60:0/1155616948 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5bd80a20c0 0x7f5bd80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:58.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.523+0000 7f5becff9640 1 -- 172.21.15.60:0/1155616948 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f5be40ce1c0 con 0x7f5bf0102390 2024-02-26T04:57:58.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.523+0000 7f5bef7fe640 1 --2- 172.21.15.60:0/1155616948 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5bd80a20c0 0x7f5bd80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:58.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.523+0000 7f5bef7fe640 1 --2- 172.21.15.60:0/1155616948 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5bd80a20c0 0x7f5bd80a4580 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5be0005fd0 tx=0x7f5be000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:58.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.527+0000 7f5becff9640 1 -- 172.21.15.60:0/1155616948 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5be4096cf0 con 0x7f5bf0102390 2024-02-26T04:57:58.660 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:58 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1270990850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:58.660 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:58 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/105074700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:58.661 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:58 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3561210448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:58.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:58 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1270990850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:57:58.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:58 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/105074700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:57:58.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:58 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3561210448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:57:58.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.747+0000 7f5bf6236640 1 -- 172.21.15.60:0/1155616948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bf01a1c30 con 0x7f5bf0102390 2024-02-26T04:57:58.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.748+0000 7f5becff9640 1 -- 172.21.15.60:0/1155616948 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11347 (secure 0 0 0) 0x7f5be409a3b0 con 0x7f5bf0102390 2024-02-26T04:57:58.751 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:57:52.162616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24476,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T04:57:58.751 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-02-26T04:57:58.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.750+0000 7f5bf6236640 1 -- 172.21.15.60:0/1155616948 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5bd80a20c0 msgr2=0x7f5bd80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.750+0000 7f5bf6236640 1 --2- 172.21.15.60:0/1155616948 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5bd80a20c0 0x7f5bd80a4580 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5be0005fd0 tx=0x7f5be000b390 comp rx=0 tx=0).stop 2024-02-26T04:57:58.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.750+0000 7f5bf6236640 1 -- 172.21.15.60:0/1155616948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bf0102390 msgr2=0x7f5bf01998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.750+0000 7f5bf6236640 1 --2- 172.21.15.60:0/1155616948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bf0102390 0x7f5bf01998e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5be400c920 tx=0x7f5be400cde0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.750+0000 7f5bf6236640 1 -- 172.21.15.60:0/1155616948 shutdown_connections 2024-02-26T04:57:58.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.750+0000 7f5bf6236640 1 --2- 172.21.15.60:0/1155616948 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5bd80a20c0 0x7f5bd80a4580 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.750+0000 7f5bf6236640 1 --2- 172.21.15.60:0/1155616948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf0102d50 0x7f5bf01a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.750+0000 7f5bf6236640 1 --2- 172.21.15.60:0/1155616948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bf0102390 0x7f5bf01998e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.750+0000 7f5bf6236640 1 --2- 172.21.15.60:0/1155616948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bf01011a0 0x7f5bf01993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.750+0000 7f5bf6236640 1 -- 172.21.15.60:0/1155616948 >> 172.21.15.60:0/1155616948 conn(0x7f5bf00fc730 msgr2=0x7f5bf00733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:58.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.750+0000 7f5bf6236640 1 -- 172.21.15.60:0/1155616948 shutdown_connections 2024-02-26T04:57:58.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.750+0000 7f5bf6236640 1 -- 172.21.15.60:0/1155616948 wait complete. 2024-02-26T04:57:58.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:57:58.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.864+0000 7f18e20de640 1 -- 172.21.15.60:0/2395583693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18dc1012b0 msgr2=0x7f18dc101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.864+0000 7f18e20de640 1 --2- 172.21.15.60:0/2395583693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18dc1012b0 0x7f18dc101730 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009900 tx=0x7f18d002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:58.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.864+0000 7f18e20de640 1 -- 172.21.15.60:0/2395583693 shutdown_connections 2024-02-26T04:57:58.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.864+0000 7f18e20de640 1 --2- 172.21.15.60:0/2395583693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18dc101c70 0x7f18dc108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.864+0000 7f18e20de640 1 --2- 172.21.15.60:0/2395583693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18dc1012b0 0x7f18dc101730 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.864+0000 7f18e20de640 1 --2- 172.21.15.60:0/2395583693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18dc1000c0 0x7f18dc1004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.864+0000 7f18e20de640 1 -- 172.21.15.60:0/2395583693 >> 172.21.15.60:0/2395583693 conn(0x7f18dc0fb630 msgr2=0x7f18dc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:58.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.864+0000 7f18e20de640 1 -- 172.21.15.60:0/2395583693 shutdown_connections 2024-02-26T04:57:58.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.864+0000 7f18e20de640 1 -- 172.21.15.60:0/2395583693 wait complete. 2024-02-26T04:57:58.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18e20de640 1 Processor -- start 2024-02-26T04:57:58.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18e20de640 1 -- start start 2024-02-26T04:57:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18e20de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18dc1000c0 0x7f18dc197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18e20de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18dc1012b0 0x7f18dc1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18e20de640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18dc101c70 0x7f18dc19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18e20de640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc19ed10 con 0x7f18dc1012b0 2024-02-26T04:57:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18e20de640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc19ee80 con 0x7f18dc101c70 2024-02-26T04:57:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18e20de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc19f040 con 0x7f18dc1000c0 2024-02-26T04:57:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18daffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18dc1012b0 0x7f18dc1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18daffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18dc1012b0 0x7f18dc1977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44356/0 (socket says 172.21.15.60:44356) 2024-02-26T04:57:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18daffd640 1 -- 172.21.15.60:0/1322759851 learned_addr learned my addr 172.21.15.60:0/1322759851 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18dbfff640 1 --2- 172.21.15.60:0/1322759851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18dc101c70 0x7f18dc19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18db7fe640 1 --2- 172.21.15.60:0/1322759851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18dc1000c0 0x7f18dc197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18daffd640 1 -- 172.21.15.60:0/1322759851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18dc1000c0 msgr2=0x7f18dc197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18daffd640 1 --2- 172.21.15.60:0/1322759851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18dc1000c0 0x7f18dc197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18daffd640 1 -- 172.21.15.60:0/1322759851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18dc101c70 msgr2=0x7f18dc19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:58.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18daffd640 1 --2- 172.21.15.60:0/1322759851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18dc101c70 0x7f18dc19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:58.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.865+0000 7f18daffd640 1 -- 172.21.15.60:0/1322759851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d0009560 con 0x7f18dc1012b0 2024-02-26T04:57:58.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.866+0000 7f18daffd640 1 --2- 172.21.15.60:0/1322759851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18dc1012b0 0x7f18dc1977b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f18d0006010 tx=0x7f18d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:58.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.866+0000 7f18d8ff9640 1 -- 172.21.15.60:0/1322759851 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d00044e0 con 0x7f18dc1012b0 2024-02-26T04:57:58.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.866+0000 7f18e20de640 1 -- 172.21.15.60:0/1322759851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc19f2c0 con 0x7f18dc1012b0 2024-02-26T04:57:58.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.866+0000 7f18d8ff9640 1 -- 172.21.15.60:0/1322759851 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18d00054e0 con 0x7f18dc1012b0 2024-02-26T04:57:58.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.866+0000 7f18d8ff9640 1 -- 172.21.15.60:0/1322759851 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d003e450 con 0x7f18dc1012b0 2024-02-26T04:57:58.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.866+0000 7f18e20de640 1 -- 172.21.15.60:0/1322759851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc19f820 con 0x7f18dc1012b0 2024-02-26T04:57:58.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.867+0000 7f18e20de640 1 -- 172.21.15.60:0/1322759851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc10bad0 con 0x7f18dc1012b0 2024-02-26T04:57:58.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.868+0000 7f18d8ff9640 1 -- 172.21.15.60:0/1322759851 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f18d00035f0 con 0x7f18dc1012b0 2024-02-26T04:57:58.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.868+0000 7f18d8ff9640 1 --2- 172.21.15.60:0/1322759851 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f18c40a20c0 0x7f18c40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:58.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.868+0000 7f18d8ff9640 1 -- 172.21.15.60:0/1322759851 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f18d00f0a50 con 0x7f18dc1012b0 2024-02-26T04:57:58.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.868+0000 7f18db7fe640 1 --2- 172.21.15.60:0/1322759851 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f18c40a20c0 0x7f18c40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:58.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.869+0000 7f18db7fe640 1 --2- 172.21.15.60:0/1322759851 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f18c40a20c0 0x7f18c40a4580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f18cc005fd0 tx=0x7f18cc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:58.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:58.872+0000 7f18d8ff9640 1 -- 172.21.15.60:0/1322759851 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18d00b95e0 con 0x7f18dc1012b0 2024-02-26T04:57:59.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.054+0000 7f18e20de640 1 -- 172.21.15.60:0/1322759851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18dc19fc50 con 0x7f18dc1012b0 2024-02-26T04:57:59.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.055+0000 7f18d8ff9640 1 -- 172.21.15.60:0/1322759851 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24972 (secure 0 0 0) 0x7f18d00bcca0 con 0x7f18dc1012b0 2024-02-26T04:57:59.058 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:57:59.058 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:57:21.168776+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:57:59.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.057+0000 7f18e20de640 1 -- 172.21.15.60:0/1322759851 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f18c40a20c0 msgr2=0x7f18c40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.057+0000 7f18e20de640 1 --2- 172.21.15.60:0/1322759851 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f18c40a20c0 0x7f18c40a4580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f18cc005fd0 tx=0x7f18cc005850 comp rx=0 tx=0).stop 2024-02-26T04:57:59.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.057+0000 7f18e20de640 1 -- 172.21.15.60:0/1322759851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18dc1012b0 msgr2=0x7f18dc1977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.057+0000 7f18e20de640 1 --2- 172.21.15.60:0/1322759851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18dc1012b0 0x7f18dc1977b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f18d0006010 tx=0x7f18d00042f0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.058+0000 7f18e20de640 1 -- 172.21.15.60:0/1322759851 shutdown_connections 2024-02-26T04:57:59.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.058+0000 7f18e20de640 1 --2- 172.21.15.60:0/1322759851 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f18c40a20c0 0x7f18c40a4580 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.058+0000 7f18e20de640 1 --2- 172.21.15.60:0/1322759851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18dc101c70 0x7f18dc19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.058+0000 7f18e20de640 1 --2- 172.21.15.60:0/1322759851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18dc1012b0 0x7f18dc1977b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.058+0000 7f18e20de640 1 --2- 172.21.15.60:0/1322759851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18dc1000c0 0x7f18dc197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.058+0000 7f18e20de640 1 -- 172.21.15.60:0/1322759851 >> 172.21.15.60:0/1322759851 conn(0x7f18dc0fb630 msgr2=0x7f18dc0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:59.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.058+0000 7f18e20de640 1 -- 172.21.15.60:0/1322759851 shutdown_connections 2024-02-26T04:57:59.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.058+0000 7f18e20de640 1 -- 172.21.15.60:0/1322759851 wait complete. 2024-02-26T04:57:59.072 DEBUG:tasks.fwd_scrub:Yielding 2024-02-26T04:57:59.072 INFO:teuthology.run_tasks:Running task workunit... 2024-02-26T04:57:59.085 INFO:tasks.workunit:Pulling workunits from ref bc0b84a094b787eb15ce0eff5f7a342217d95e39 2024-02-26T04:57:59.085 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-26T04:57:59.085 INFO:tasks.workunit:timeout=3h 2024-02-26T04:57:59.085 INFO:tasks.workunit:cleanup=True 2024-02-26T04:57:59.086 DEBUG:teuthology.orchestra.run.smithi060:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-26T04:57:59.086 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-02-26T04:57:59.086 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T04:57:59.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:57:59.086 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T04:57:59.101 INFO:teuthology.orchestra.run.smithi060.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-26T04:57:59.101 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-02-26T04:57:59.101 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-02-26T04:57:59.101 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T04:57:59.101 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-02-26T04:57:59.101 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-02-26 04:55:55.565040522 +0000 2024-02-26T04:57:59.101 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-02-26 04:55:55.565040522 +0000 2024-02-26T04:57:59.101 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-02-26 04:57:51.488384141 +0000 2024-02-26T04:57:59.101 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-02-26 04:55:55.565040522 +0000 2024-02-26T04:57:59.102 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-26T04:57:59.102 DEBUG:teuthology.orchestra.run.smithi060:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-26T04:57:59.173 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/vshankar/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bc0b84a094b787eb15ce0eff5f7a342217d95e39 2024-02-26T04:57:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.192+0000 7f86f7a20640 1 -- 172.21.15.60:0/2491950833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f0101270 msgr2=0x7f86f01016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.192+0000 7f86f7a20640 1 --2- 172.21.15.60:0/2491950833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f0101270 0x7f86f01016f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009900 tx=0x7f86e002bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.192+0000 7f86f7a20640 1 -- 172.21.15.60:0/2491950833 shutdown_connections 2024-02-26T04:57:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.192+0000 7f86f7a20640 1 --2- 172.21.15.60:0/2491950833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f0101c30 0x7f86f01084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.192+0000 7f86f7a20640 1 --2- 172.21.15.60:0/2491950833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f0101270 0x7f86f01016f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.192+0000 7f86f7a20640 1 --2- 172.21.15.60:0/2491950833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86f0100080 0x7f86f01004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.192+0000 7f86f7a20640 1 -- 172.21.15.60:0/2491950833 >> 172.21.15.60:0/2491950833 conn(0x7f86f00fb610 msgr2=0x7f86f00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.192+0000 7f86f7a20640 1 -- 172.21.15.60:0/2491950833 shutdown_connections 2024-02-26T04:57:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.192+0000 7f86f7a20640 1 -- 172.21.15.60:0/2491950833 wait complete. 2024-02-26T04:57:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f7a20640 1 Processor -- start 2024-02-26T04:57:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f7a20640 1 -- start start 2024-02-26T04:57:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f7a20640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86f0100080 0x7f86f010a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f7a20640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f0101270 0x7f86f0108f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f7a20640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f0101c30 0x7f86f01094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f7a20640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f0109c20 con 0x7f86f0101c30 2024-02-26T04:57:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f7a20640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f0109d90 con 0x7f86f0101270 2024-02-26T04:57:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f7a20640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01a12b0 con 0x7f86f0100080 2024-02-26T04:57:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f5fbd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f0101c30 0x7f86f01094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f5fbd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f0101c30 0x7f86f01094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44368/0 (socket says 172.21.15.60:44368) 2024-02-26T04:57:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f5fbd640 1 -- 172.21.15.60:0/3713142265 learned_addr learned my addr 172.21.15.60:0/3713142265 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f0101270 0x7f86f0108f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f57bc640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86f0100080 0x7f86f010a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f5fbd640 1 -- 172.21.15.60:0/3713142265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86f0100080 msgr2=0x7f86f010a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f5fbd640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86f0100080 0x7f86f010a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f5fbd640 1 -- 172.21.15.60:0/3713142265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f0101270 msgr2=0x7f86f0108f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f5fbd640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f0101270 0x7f86f0108f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f5fbd640 1 -- 172.21.15.60:0/3713142265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0009560 con 0x7f86f0101c30 2024-02-26T04:57:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.193+0000 7f86f57bc640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86f0100080 0x7f86f010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.194+0000 7f86f5fbd640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f0101c30 0x7f86f01094c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f86e800cb00 tx=0x7f86e800a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.194+0000 7f86e67fc640 1 -- 172.21.15.60:0/3713142265 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e800ae90 con 0x7f86f0101c30 2024-02-26T04:57:59.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.194+0000 7f86f7a20640 1 -- 172.21.15.60:0/3713142265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f01a1560 con 0x7f86f0101c30 2024-02-26T04:57:59.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.194+0000 7f86e67fc640 1 -- 172.21.15.60:0/3713142265 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86e8005020 con 0x7f86f0101c30 2024-02-26T04:57:59.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.194+0000 7f86e67fc640 1 -- 172.21.15.60:0/3713142265 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e800f6b0 con 0x7f86f0101c30 2024-02-26T04:57:59.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.194+0000 7f86f7a20640 1 -- 172.21.15.60:0/3713142265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f01a19f0 con 0x7f86f0101c30 2024-02-26T04:57:59.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86fa7f7740 1 Processor -- start 2024-02-26T04:57:59.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86e67fc640 1 -- 172.21.15.60:0/3713142265 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f86e8005520 con 0x7f86f0101c30 2024-02-26T04:57:59.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86fa7f7740 1 -- start start 2024-02-26T04:57:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86fa7f7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75eb490 0x5582f75eb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86fa7f7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86fa7f7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75ed020 0x5582f75f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86fa7f7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f755af60 con 0x5582f75ec660 2024-02-26T04:57:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86fa7f7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f7558490 con 0x5582f75ed020 2024-02-26T04:57:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86fa7f7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f7553c70 con 0x5582f75eb490 2024-02-26T04:57:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86f57bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86f57bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75ecae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44378/0 (socket says 172.21.15.60:44378) 2024-02-26T04:57:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86f57bc640 1 -- 172.21.15.60:0/1693608014 learned_addr learned my addr 172.21.15.60:0/1693608014 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/1693608014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75eb490 0x5582f75eb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86e67fc640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f86dc0a20c0 0x7f86dc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/1693608014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75ed020 0x5582f75f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86f57bc640 1 -- 172.21.15.60:0/1693608014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75eb490 msgr2=0x5582f75eb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.197+0000 7f86f57bc640 1 --2- 172.21.15.60:0/1693608014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75eb490 0x5582f75eb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.198+0000 7f86f57bc640 1 -- 172.21.15.60:0/1693608014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75ed020 msgr2=0x5582f75f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.198+0000 7f86f5fbd640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f86dc0a20c0 0x7f86dc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.198+0000 7f86f57bc640 1 --2- 172.21.15.60:0/1693608014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75ed020 0x5582f75f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.198+0000 7f86f57bc640 1 -- 172.21.15.60:0/1693608014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582f7516080 con 0x5582f75ec660 2024-02-26T04:57:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.198+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/1693608014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75ed020 0x5582f75f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.198+0000 7f86e67fc640 1 -- 172.21.15.60:0/3713142265 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f86e80ce0f0 con 0x7f86f0101c30 2024-02-26T04:57:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.198+0000 7f86f57bc640 1 --2- 172.21.15.60:0/1693608014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75ecae0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00cc50 tx=0x7f86ec00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca11cc6984e40ac server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.198+0000 7f86db7fe640 1 -- 172.21.15.60:0/1693608014 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec019660 con 0x5582f75ec660 2024-02-26T04:57:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.198+0000 7f86f5fbd640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f86dc0a20c0 0x7f86dc0a4580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f86e809a6e0 tx=0x7f86e806a890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.198+0000 7f86db7fe640 1 -- 172.21.15.60:0/1693608014 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86ec019ca0 con 0x5582f75ec660 2024-02-26T04:57:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.198+0000 7f86db7fe640 1 -- 172.21.15.60:0/1693608014 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec010d10 con 0x5582f75ec660 2024-02-26T04:57:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.199+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1693608014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 msgr2=0x5582f75ecae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.199+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/1693608014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75ecae0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00cc50 tx=0x7f86ec00e720 comp rx=0 tx=0).stop 2024-02-26T04:57:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.199+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1693608014 shutdown_connections 2024-02-26T04:57:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.199+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/1693608014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75ed020 0x5582f75f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.199+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/1693608014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75ecae0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.199+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/1693608014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75eb490 0x5582f75eb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.199+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1693608014 >> 172.21.15.60:0/1693608014 conn(0x5582f75e6670 msgr2=0x5582f75e6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.199+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1693608014 shutdown_connections 2024-02-26T04:57:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.199+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1693608014 wait complete. 2024-02-26T04:57:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.199+0000 7f86fa7f7740 1 -- start start 2024-02-26T04:57:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.199+0000 7f86fa7f7740 1 Processor -- start 2024-02-26T04:57:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86fa7f7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75eb490 0x5582f75f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86fa7f7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75ec660 0x5582f75f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86fa7f7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ed020 0x5582f75fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86fa7f7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f74f8f10 con 0x5582f75ed020 2024-02-26T04:57:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86fa7f7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f72274b0 con 0x5582f75eb490 2024-02-26T04:57:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86fa7f7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f74d9640 con 0x5582f75ec660 2024-02-26T04:57:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86fa7f7740 10 client.0 _ll_register_callbacks cb 0x5582f7546f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86f4fbb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ed020 0x5582f75fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86f4fbb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ed020 0x5582f75fcc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44382/0 (socket says 172.21.15.60:44382) 2024-02-26T04:57:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86f4fbb640 1 -- 172.21.15.60:0/1147845319 learned_addr learned my addr 172.21.15.60:0/1147845319 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86f57bc640 1 --2- 172.21.15.60:0/1147845319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75ec660 0x5582f75f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/1147845319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75eb490 0x5582f75f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86f4fbb640 1 -- 172.21.15.60:0/1147845319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75ec660 msgr2=0x5582f75f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/1147845319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75ec660 0x5582f75f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86f4fbb640 1 -- 172.21.15.60:0/1147845319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75eb490 msgr2=0x5582f75f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/1147845319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75eb490 0x5582f75f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86f4fbb640 1 -- 172.21.15.60:0/1147845319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ec00c8b0 con 0x5582f75ed020 2024-02-26T04:57:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/1147845319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ed020 0x5582f75fcc60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f86e002c170 tx=0x7f86e002cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fc7afe43ee2d6f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.200+0000 7f86d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.201+0000 7f86d8ff9640 1 -- 172.21.15.60:0/1147845319 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0004500 con 0x5582f75ed020 2024-02-26T04:57:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.201+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1147845319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582f752df10 con 0x5582f75ed020 2024-02-26T04:57:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.201+0000 7f86d8ff9640 1 -- 172.21.15.60:0/1147845319 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f86e00035f0 con 0x5582f75ed020 2024-02-26T04:57:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.201+0000 7f86d8ff9640 1 -- 172.21.15.60:0/1147845319 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86e00518f0 con 0x5582f75ed020 2024-02-26T04:57:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.201+0000 7f86fa7f7740 10 client.15576 fetch_fsmap learned FSMap version 23 2024-02-26T04:57:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.201+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1147845319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5582f75522f0 con 0x5582f75ed020 2024-02-26T04:57:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.201+0000 7f86d8ff9640 1 -- 172.21.15.60:0/1147845319 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5582f752df10 con 0x5582f75ed020 2024-02-26T04:57:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.201+0000 7f86d8ff9640 1 -- 172.21.15.60:0/1147845319 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f86e003c040 con 0x5582f75ed020 2024-02-26T04:57:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.201+0000 7f86fa7f7740 10 client.15576 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:57:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.201+0000 7f86fa7f7740 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:57:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.201+0000 7f86fa7f7740 20 client.15576 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.201+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/1147845319 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5582f7601880 0x5582f7603c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.201+0000 7f86fa7f7740 4 client.15576 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:57:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.201+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1147845319 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582f754a2e0 con 0x5582f7601880 2024-02-26T04:57:59.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.202+0000 7f86f57bc640 1 --2- 172.21.15.60:0/1147845319 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5582f7601880 0x5582f7603c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.202+0000 7f86d8ff9640 10 client.15576 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:59.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.202+0000 7f86f57bc640 1 --2- 172.21.15.60:0/1147845319 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5582f7601880 0x5582f7603c70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33e94806614added server_cookie=b2a34fadb4fd9514 in_seq=0 out_seq=0 2024-02-26T04:57:59.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86d8ff9640 1 -- 172.21.15.60:0/1147845319 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5582f754a2e0 con 0x5582f7601880 2024-02-26T04:57:59.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86d8ff9640 10 client.15576 handle_command_reply: tid=0 2024-02-26T04:57:59.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86fa7f7740 1 client.15576 shutdown 2024-02-26T04:57:59.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1147845319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ed020 msgr2=0x5582f75fcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/1147845319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ed020 0x5582f75fcc60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f86e002c170 tx=0x7f86e002cb10 comp rx=0 tx=0).stop 2024-02-26T04:57:59.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1147845319 shutdown_connections 2024-02-26T04:57:59.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/1147845319 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5582f7601880 0x5582f7603c70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/1147845319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ed020 0x5582f75fcc60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86d8ff9640 0 client.15576 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:59.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/1147845319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75ec660 0x5582f75f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/1147845319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75eb490 0x5582f75f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1147845319 >> 172.21.15.60:0/1147845319 conn(0x5582f75e6670 msgr2=0x5582f75e58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:59.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1147845319 shutdown_connections 2024-02-26T04:57:59.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1147845319 wait complete. 2024-02-26T04:57:59.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.203+0000 7f86fa7f7740 20 client.15576 trim_cache size 0 max 16384 2024-02-26T04:57:59.215 INFO:tasks.workunit.client.0.smithi060.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-26T04:57:59.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86fa7f7740 1 Processor -- start 2024-02-26T04:57:59.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86fa7f7740 1 -- start start 2024-02-26T04:57:59.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86fa7f7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75eb490 0x5582f75f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86fa7f7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75fcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86fa7f7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75ed020 0x5582f75f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86fa7f7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f75efb60 con 0x5582f75ec660 2024-02-26T04:57:59.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86fa7f7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f75db320 con 0x5582f75ed020 2024-02-26T04:57:59.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86fa7f7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f754a2e0 con 0x5582f75eb490 2024-02-26T04:57:59.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86f57bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75fcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86f57bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75fcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44396/0 (socket says 172.21.15.60:44396) 2024-02-26T04:57:59.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86f57bc640 1 -- 172.21.15.60:0/1105054691 learned_addr learned my addr 172.21.15.60:0/1105054691 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:59.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/1105054691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75eb490 0x5582f75f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/1105054691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75ed020 0x5582f75f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86f57bc640 1 -- 172.21.15.60:0/1105054691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75eb490 msgr2=0x5582f75f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86f57bc640 1 --2- 172.21.15.60:0/1105054691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75eb490 0x5582f75f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86f57bc640 1 -- 172.21.15.60:0/1105054691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75ed020 msgr2=0x5582f75f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86f57bc640 1 --2- 172.21.15.60:0/1105054691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75ed020 0x5582f75f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86f57bc640 1 -- 172.21.15.60:0/1105054691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582f74eebd0 con 0x5582f75ec660 2024-02-26T04:57:59.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.220+0000 7f86f57bc640 1 --2- 172.21.15.60:0/1105054691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75fcc20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x5582f75e75e0 tx=0x7f86ec00c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321784708395e63 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:59.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.221+0000 7f86daffd640 1 -- 172.21.15.60:0/1105054691 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec010550 con 0x5582f75ec660 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.221+0000 7f86daffd640 1 -- 172.21.15.60:0/1105054691 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86ec004030 con 0x5582f75ec660 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.221+0000 7f86daffd640 1 -- 172.21.15.60:0/1105054691 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec005460 con 0x5582f75ec660 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.221+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1105054691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 msgr2=0x5582f75fcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.221+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/1105054691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75fcc20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x5582f75e75e0 tx=0x7f86ec00c340 comp rx=0 tx=0).stop 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.221+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1105054691 shutdown_connections 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.221+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/1105054691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75ed020 0x5582f75f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.221+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/1105054691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75fcc20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.221+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/1105054691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75eb490 0x5582f75f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.221+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1105054691 >> 172.21.15.60:0/1105054691 conn(0x5582f75e6670 msgr2=0x5582f75f0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.221+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1105054691 shutdown_connections 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.221+0000 7f86fa7f7740 1 -- 172.21.15.60:0/1105054691 wait complete. 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86fa7f7740 1 -- start start 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86fa7f7740 1 Processor -- start 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86fa7f7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75eb490 0x5582f75f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86fa7f7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86fa7f7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75ed020 0x5582f75fb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86fa7f7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f75fb830 con 0x5582f75ec660 2024-02-26T04:57:59.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86fa7f7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f75fb9a0 con 0x5582f75eb490 2024-02-26T04:57:59.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86fa7f7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f75fbb60 con 0x5582f75ed020 2024-02-26T04:57:59.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86fa7f7740 10 client.0 _ll_register_callbacks cb 0x7f86ec00c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:59.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86f57bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86f57bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75fa2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44406/0 (socket says 172.21.15.60:44406) 2024-02-26T04:57:59.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86f57bc640 1 -- 172.21.15.60:0/2186857314 learned_addr learned my addr 172.21.15.60:0/2186857314 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:59.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/2186857314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75ed020 0x5582f75fb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86f57bc640 1 -- 172.21.15.60:0/2186857314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75ed020 msgr2=0x5582f75fb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86f57bc640 1 --2- 172.21.15.60:0/2186857314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75ed020 0x5582f75fb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/2186857314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75eb490 0x5582f75f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.222+0000 7f86f57bc640 1 -- 172.21.15.60:0/2186857314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75eb490 msgr2=0x5582f75f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86f57bc640 1 --2- 172.21.15.60:0/2186857314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75eb490 0x5582f75f9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86f57bc640 1 -- 172.21.15.60:0/2186857314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582f7516080 con 0x5582f75ec660 2024-02-26T04:57:59.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/2186857314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75eb490 0x5582f75f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:59.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86f57bc640 1 --2- 172.21.15.60:0/2186857314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75fa2e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e760 tx=0x7f86ec00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ef58169b43532e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:59.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:59.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86d8ff9640 1 -- 172.21.15.60:0/2186857314 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0108d0 con 0x5582f75ec660 2024-02-26T04:57:59.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86d8ff9640 1 -- 172.21.15.60:0/2186857314 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f86ec010b10 con 0x5582f75ec660 2024-02-26T04:57:59.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86fa7f7740 1 -- 172.21.15.60:0/2186857314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582f75798d0 con 0x5582f75ec660 2024-02-26T04:57:59.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86d8ff9640 1 -- 172.21.15.60:0/2186857314 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86ec010cd0 con 0x5582f75ec660 2024-02-26T04:57:59.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86fa7f7740 10 client.15582 fetch_fsmap learned FSMap version 23 2024-02-26T04:57:59.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86fa7f7740 1 -- 172.21.15.60:0/2186857314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5582f75148e0 con 0x5582f75ec660 2024-02-26T04:57:59.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86d8ff9640 1 -- 172.21.15.60:0/2186857314 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5582f75798d0 con 0x5582f75ec660 2024-02-26T04:57:59.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86d8ff9640 1 -- 172.21.15.60:0/2186857314 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f86ec027d20 con 0x5582f75ec660 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86fa7f7740 10 client.15582 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86fa7f7740 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86fa7f7740 20 client.15582 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/2186857314 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5582f7601880 0x5582f765bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86fa7f7740 4 client.15582 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.223+0000 7f86fa7f7740 1 -- 172.21.15.60:0/2186857314 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5582f7253860 con 0x5582f7601880 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.224+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/2186857314 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5582f7601880 0x5582f765bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.224+0000 7f86d8ff9640 10 client.15582 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.224+0000 7f86f4fbb640 1 --2- 172.21.15.60:0/2186857314 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5582f7601880 0x5582f765bfe0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb4f338eac49e1a9 server_cookie=7b0bff95b54369d2 in_seq=0 out_seq=0 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.225+0000 7f86d8ff9640 1 -- 172.21.15.60:0/2186857314 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5582f7253860 con 0x5582f7601880 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.225+0000 7f86d8ff9640 10 client.15582 handle_command_reply: tid=0 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.225+0000 7f86fa7f7740 1 client.15582 shutdown 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.225+0000 7f86fa7f7740 1 -- 172.21.15.60:0/2186857314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 msgr2=0x5582f75fa2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.225+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/2186857314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75fa2e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e760 tx=0x7f86ec00c7c0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.225+0000 7f86fa7f7740 1 -- 172.21.15.60:0/2186857314 shutdown_connections 2024-02-26T04:57:59.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.225+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/2186857314 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5582f7601880 0x5582f765bfe0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.225+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/2186857314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582f75ed020 0x5582f75fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.225+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/2186857314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5582f75ec660 0x5582f75fa2e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.225+0000 7f86fa7f7740 1 --2- 172.21.15.60:0/2186857314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582f75eb490 0x5582f75f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.225+0000 7f86d8ff9640 0 client.15582 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:59.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.225+0000 7f86fa7f7740 1 -- 172.21.15.60:0/2186857314 >> 172.21.15.60:0/2186857314 conn(0x5582f75e6670 msgr2=0x5582f75f5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:59.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86fa7f7740 1 -- 172.21.15.60:0/2186857314 shutdown_connections 2024-02-26T04:57:59.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86fa7f7740 1 -- 172.21.15.60:0/2186857314 wait complete. 2024-02-26T04:57:59.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86fa7f7740 20 client.15582 trim_cache size 0 max 16384 2024-02-26T04:57:59.234 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:57:59.235 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-02-26T04:57:59.235 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "5c65524b-a581-4347-8136-767f0f20980e", 2024-02-26T04:57:59.235 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-02-26T04:57:59.235 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:57:59.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86f7a20640 1 -- 172.21.15.60:0/3713142265 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f86dc0a20c0 msgr2=0x7f86dc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86f7a20640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f86dc0a20c0 0x7f86dc0a4580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f86e809a6e0 tx=0x7f86e806a890 comp rx=0 tx=0).stop 2024-02-26T04:57:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86f7a20640 1 -- 172.21.15.60:0/3713142265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f0101c30 msgr2=0x7f86f01094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86f7a20640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f0101c30 0x7f86f01094c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f86e800cb00 tx=0x7f86e800a250 comp rx=0 tx=0).stop 2024-02-26T04:57:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86f7a20640 1 -- 172.21.15.60:0/3713142265 shutdown_connections 2024-02-26T04:57:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86f7a20640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f86dc0a20c0 0x7f86dc0a4580 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86f7a20640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86f0101c30 0x7f86f01094c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86f7a20640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f0101270 0x7f86f0108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86f7a20640 1 --2- 172.21.15.60:0/3713142265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86f0100080 0x7f86f010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86f7a20640 1 -- 172.21.15.60:0/3713142265 >> 172.21.15.60:0/3713142265 conn(0x7f86f00fb610 msgr2=0x7f86f0104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86f7a20640 1 -- 172.21.15.60:0/3713142265 shutdown_connections 2024-02-26T04:57:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.226+0000 7f86f7a20640 1 -- 172.21.15.60:0/3713142265 wait complete. 2024-02-26T04:57:59.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T04:57:59.242 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T04:57:59.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.386+0000 7f6383308640 1 -- 172.21.15.60:0/781958366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c101c60 msgr2=0x7f637c104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.386+0000 7f6383308640 1 --2- 172.21.15.60:0/781958366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c101c60 0x7f637c104050 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f636c009900 tx=0x7f636c02bc30 comp rx=0 tx=0).stop 2024-02-26T04:57:59.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.386+0000 7f6383308640 1 -- 172.21.15.60:0/781958366 shutdown_connections 2024-02-26T04:57:59.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.386+0000 7f6383308640 1 --2- 172.21.15.60:0/781958366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f637c106ec0 0x7f637c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.386+0000 7f6383308640 1 --2- 172.21.15.60:0/781958366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f637c104590 0x7f637c106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.386+0000 7f6383308640 1 --2- 172.21.15.60:0/781958366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c101c60 0x7f637c104050 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.386+0000 7f6383308640 1 -- 172.21.15.60:0/781958366 >> 172.21.15.60:0/781958366 conn(0x7f637c0fb5f0 msgr2=0x7f637c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:59.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.386+0000 7f6383308640 1 -- 172.21.15.60:0/781958366 shutdown_connections 2024-02-26T04:57:59.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.386+0000 7f6383308640 1 -- 172.21.15.60:0/781958366 wait complete. 2024-02-26T04:57:59.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.386+0000 7f6383308640 1 Processor -- start 2024-02-26T04:57:59.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.386+0000 7f6383308640 1 -- start start 2024-02-26T04:57:59.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.386+0000 7f6383308640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f637c101c60 0x7f637c1971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.386+0000 7f6383308640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c104590 0x7f637c197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f6383308640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f637c106ec0 0x7f637c19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f6383308640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c19ecb0 con 0x7f637c104590 2024-02-26T04:57:59.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f6383308640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c19ee20 con 0x7f637c106ec0 2024-02-26T04:57:59.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f6383308640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c19efe0 con 0x7f637c101c60 2024-02-26T04:57:59.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f63808a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c104590 0x7f637c197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f63808a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c104590 0x7f637c197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44412/0 (socket says 172.21.15.60:44412) 2024-02-26T04:57:59.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f63808a3640 1 -- 172.21.15.60:0/3782401028 learned_addr learned my addr 172.21.15.60:0/3782401028 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:59.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f63818a5640 1 --2- 172.21.15.60:0/3782401028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f637c106ec0 0x7f637c19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f63810a4640 1 --2- 172.21.15.60:0/3782401028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f637c101c60 0x7f637c1971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f63808a3640 1 -- 172.21.15.60:0/3782401028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f637c101c60 msgr2=0x7f637c1971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f63808a3640 1 --2- 172.21.15.60:0/3782401028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f637c101c60 0x7f637c1971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f63808a3640 1 -- 172.21.15.60:0/3782401028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f637c106ec0 msgr2=0x7f637c19e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f63808a3640 1 --2- 172.21.15.60:0/3782401028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f637c106ec0 0x7f637c19e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f63808a3640 1 -- 172.21.15.60:0/3782401028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c009560 con 0x7f637c104590 2024-02-26T04:57:59.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f63808a3640 1 --2- 172.21.15.60:0/3782401028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c104590 0x7f637c197700 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f637800c970 tx=0x7f637800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:59.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f63727fc640 1 -- 172.21.15.60:0/3782401028 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6378004440 con 0x7f637c104590 2024-02-26T04:57:59.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f63727fc640 1 -- 172.21.15.60:0/3782401028 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6378007d40 con 0x7f637c104590 2024-02-26T04:57:59.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f6383308640 1 -- 172.21.15.60:0/3782401028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637c19f2c0 con 0x7f637c104590 2024-02-26T04:57:59.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f63727fc640 1 -- 172.21.15.60:0/3782401028 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637800f450 con 0x7f637c104590 2024-02-26T04:57:59.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.387+0000 7f6383308640 1 -- 172.21.15.60:0/3782401028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c19f7d0 con 0x7f637c104590 2024-02-26T04:57:59.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.389+0000 7f63727fc640 1 -- 172.21.15.60:0/3782401028 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6378007860 con 0x7f637c104590 2024-02-26T04:57:59.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63861b7740 1 Processor -- start 2024-02-26T04:57:59.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63861b7740 1 -- start start 2024-02-26T04:57:59.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63727fc640 1 --2- 172.21.15.60:0/3782401028 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f63680a20c0 0x7f63680a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63861b7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea22a0 0x55b4d2ea26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63861b7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea2c90 0x55b4d2ea7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63861b7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eaa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63861b7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d2e12250 con 0x55b4d2ea2c90 2024-02-26T04:57:59.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63861b7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d2e0f780 con 0x55b4d2ea22a0 2024-02-26T04:57:59.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63861b7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d2e0af60 con 0x55b4d2ea7e00 2024-02-26T04:57:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63818a5640 1 --2- 172.21.15.60:0/3782401028 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f63680a20c0 0x7f63680a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63727fc640 1 -- 172.21.15.60:0/3782401028 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f63780ce2f0 con 0x7f637c104590 2024-02-26T04:57:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63808a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea2c90 0x55b4d2ea7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63808a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea2c90 0x55b4d2ea7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44420/0 (socket says 172.21.15.60:44420) 2024-02-26T04:57:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63808a3640 1 -- 172.21.15.60:0/3913274496 learned_addr learned my addr 172.21.15.60:0/3913274496 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.390+0000 7f63810a4640 1 --2- 172.21.15.60:0/3913274496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea22a0 0x55b4d2ea26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.391+0000 7f63808a3640 1 -- 172.21.15.60:0/3913274496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea7e00 msgr2=0x55b4d2eaa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.391+0000 7f63808a3640 1 --2- 172.21.15.60:0/3913274496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eaa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.391+0000 7f63808a3640 1 -- 172.21.15.60:0/3913274496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea22a0 msgr2=0x55b4d2ea26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.391+0000 7f63808a3640 1 --2- 172.21.15.60:0/3913274496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea22a0 0x55b4d2ea26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.391+0000 7f63808a3640 1 -- 172.21.15.60:0/3913274496 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4d2dc9540 con 0x55b4d2ea2c90 2024-02-26T04:57:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.391+0000 7f63810a4640 1 --2- 172.21.15.60:0/3913274496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea22a0 0x55b4d2ea26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.391+0000 7f63818a5640 1 --2- 172.21.15.60:0/3782401028 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f63680a20c0 0x7f63680a4580 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6374005fd0 tx=0x7f6374010740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.391+0000 7f63808a3640 1 --2- 172.21.15.60:0/3913274496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea2c90 0x55b4d2ea7790 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f63780a2df0 tx=0x7f63780035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49f2b85694b4fad server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.391+0000 7f63677fe640 1 -- 172.21.15.60:0/3913274496 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637809a6c0 con 0x55b4d2ea2c90 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.391+0000 7f63677fe640 1 -- 172.21.15.60:0/3913274496 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f637809a4e0 con 0x55b4d2ea2c90 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.391+0000 7f63677fe640 1 -- 172.21.15.60:0/3913274496 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63780cba60 con 0x55b4d2ea2c90 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 -- 172.21.15.60:0/3913274496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea2c90 msgr2=0x55b4d2ea7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 --2- 172.21.15.60:0/3913274496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea2c90 0x55b4d2ea7790 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f63780a2df0 tx=0x7f63780035f0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 -- 172.21.15.60:0/3913274496 shutdown_connections 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 --2- 172.21.15.60:0/3913274496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eaa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 --2- 172.21.15.60:0/3913274496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea2c90 0x55b4d2ea7790 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 --2- 172.21.15.60:0/3913274496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea22a0 0x55b4d2ea26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 -- 172.21.15.60:0/3913274496 >> 172.21.15.60:0/3913274496 conn(0x55b4d2e9d7c0 msgr2=0x55b4d2e9dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 -- 172.21.15.60:0/3913274496 shutdown_connections 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 -- 172.21.15.60:0/3913274496 wait complete. 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 -- start start 2024-02-26T04:57:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 Processor -- start 2024-02-26T04:57:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea22a0 0x55b4d2eac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eb3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d2ade430 con 0x55b4d2ea7e00 2024-02-26T04:57:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d2a9ae00 con 0x55b4d2ea22a0 2024-02-26T04:57:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d2d8ffa0 con 0x55b4d2ea2c90 2024-02-26T04:57:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.392+0000 7f63861b7740 10 client.0 _ll_register_callbacks cb 0x55b4d2de5fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63818a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eb3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63818a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eb3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44426/0 (socket says 172.21.15.60:44426) 2024-02-26T04:57:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63818a5640 1 -- 172.21.15.60:0/688101869 learned_addr learned my addr 172.21.15.60:0/688101869 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63808a3640 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63810a4640 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea22a0 0x55b4d2eac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63818a5640 1 -- 172.21.15.60:0/688101869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea2c90 msgr2=0x55b4d2eac6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63818a5640 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eac6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63818a5640 1 -- 172.21.15.60:0/688101869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea22a0 msgr2=0x55b4d2eac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63818a5640 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea22a0 0x55b4d2eac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63818a5640 1 -- 172.21.15.60:0/688101869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c19f7d0 con 0x55b4d2ea7e00 2024-02-26T04:57:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63808a3640 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63810a4640 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea22a0 0x55b4d2eac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63818a5640 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eb3730 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6374012810 tx=0x7f6374004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27a800426eec79a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f6364ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f6364ff9640 1 -- 172.21.15.60:0/688101869 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6374011b60 con 0x55b4d2ea7e00 2024-02-26T04:57:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f6364ff9640 1 -- 172.21.15.60:0/688101869 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f6374019070 con 0x55b4d2ea7e00 2024-02-26T04:57:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63861b7740 1 -- 172.21.15.60:0/688101869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4d2b0a7e0 con 0x55b4d2ea7e00 2024-02-26T04:57:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f6364ff9640 1 -- 172.21.15.60:0/688101869 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6374011cc0 con 0x55b4d2ea7e00 2024-02-26T04:57:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63861b7740 10 client.15594 fetch_fsmap learned FSMap version 23 2024-02-26T04:57:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f63861b7740 1 -- 172.21.15.60:0/688101869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b4d2e095e0 con 0x55b4d2ea7e00 2024-02-26T04:57:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.393+0000 7f6364ff9640 1 -- 172.21.15.60:0/688101869 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b4d2b0a7e0 con 0x55b4d2ea7e00 2024-02-26T04:57:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.394+0000 7f6364ff9640 1 -- 172.21.15.60:0/688101869 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f63740273e0 con 0x55b4d2ea7e00 2024-02-26T04:57:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.394+0000 7f63861b7740 10 client.15594 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:57:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.394+0000 7f63861b7740 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:57:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.394+0000 7f63861b7740 20 client.15594 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.394+0000 7f63861b7740 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55b4d2eb84b0 0x55b4d2eba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.394+0000 7f63861b7740 4 client.15594 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:57:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.394+0000 7f63861b7740 1 -- 172.21.15.60:0/688101869 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4d2b47720 con 0x55b4d2eb84b0 2024-02-26T04:57:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.394+0000 7f63810a4640 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55b4d2eb84b0 0x55b4d2eba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.394+0000 7f6364ff9640 10 client.15594 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.394+0000 7f63810a4640 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55b4d2eb84b0 0x55b4d2eba8c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d83e14ff46b43ea8 server_cookie=5fe32f44f8dd61e6 in_seq=0 out_seq=0 2024-02-26T04:57:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.395+0000 7f6364ff9640 1 -- 172.21.15.60:0/688101869 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b4d2b47720 con 0x55b4d2eb84b0 2024-02-26T04:57:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.395+0000 7f6364ff9640 10 client.15594 handle_command_reply: tid=0 2024-02-26T04:57:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.395+0000 7f63861b7740 1 client.15594 shutdown 2024-02-26T04:57:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.395+0000 7f63861b7740 1 -- 172.21.15.60:0/688101869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea7e00 msgr2=0x55b4d2eb3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.395+0000 7f63861b7740 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eb3730 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6374012810 tx=0x7f6374004ab0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.395+0000 7f63861b7740 1 -- 172.21.15.60:0/688101869 shutdown_connections 2024-02-26T04:57:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.395+0000 7f63861b7740 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55b4d2eb84b0 0x55b4d2eba8c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.395+0000 7f63861b7740 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eb3730 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.395+0000 7f63861b7740 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.395+0000 7f63861b7740 1 --2- 172.21.15.60:0/688101869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea22a0 0x55b4d2eac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.395+0000 7f6364ff9640 0 client.15594 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.395+0000 7f63861b7740 1 -- 172.21.15.60:0/688101869 >> 172.21.15.60:0/688101869 conn(0x55b4d2e9d7c0 msgr2=0x55b4d2e9ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.395+0000 7f63861b7740 1 -- 172.21.15.60:0/688101869 shutdown_connections 2024-02-26T04:57:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.395+0000 7f63861b7740 1 -- 172.21.15.60:0/688101869 wait complete. 2024-02-26T04:57:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.396+0000 7f63861b7740 20 client.15594 trim_cache size 0 max 16384 2024-02-26T04:57:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.410+0000 7f63861b7740 1 Processor -- start 2024-02-26T04:57:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.410+0000 7f63861b7740 1 -- start start 2024-02-26T04:57:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63861b7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea22a0 0x55b4d2eaa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63861b7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eb3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63861b7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63861b7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d2e990f0 con 0x55b4d2ea2c90 2024-02-26T04:57:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63861b7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d2eb6a10 con 0x55b4d2ea7e00 2024-02-26T04:57:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63861b7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d2e97e30 con 0x55b4d2ea22a0 2024-02-26T04:57:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63808a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eb3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63808a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eb3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44434/0 (socket says 172.21.15.60:44434) 2024-02-26T04:57:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63808a3640 1 -- 172.21.15.60:0/4069734098 learned_addr learned my addr 172.21.15.60:0/4069734098 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63808a3640 1 -- 172.21.15.60:0/4069734098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea22a0 msgr2=0x55b4d2eaa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63808a3640 1 --2- 172.21.15.60:0/4069734098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea22a0 0x55b4d2eaa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63818a5640 1 --2- 172.21.15.60:0/4069734098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63808a3640 1 -- 172.21.15.60:0/4069734098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea7e00 msgr2=0x55b4d2eac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63808a3640 1 --2- 172.21.15.60:0/4069734098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63808a3640 1 -- 172.21.15.60:0/4069734098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4d2da5a90 con 0x55b4d2ea2c90 2024-02-26T04:57:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63818a5640 1 --2- 172.21.15.60:0/4069734098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:57:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f63808a3640 1 --2- 172.21.15.60:0/4069734098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eb3650 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f637809a880 tx=0x7f63780076e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413b2999a164dc61 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f6366ffd640 1 -- 172.21.15.60:0/4069734098 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63780a3320 con 0x55b4d2ea2c90 2024-02-26T04:57:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f6366ffd640 1 -- 172.21.15.60:0/4069734098 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f637809b280 con 0x55b4d2ea2c90 2024-02-26T04:57:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.411+0000 7f6366ffd640 1 -- 172.21.15.60:0/4069734098 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6378003680 con 0x55b4d2ea2c90 2024-02-26T04:57:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.412+0000 7f63861b7740 1 -- 172.21.15.60:0/4069734098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea2c90 msgr2=0x55b4d2eb3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.412+0000 7f63861b7740 1 --2- 172.21.15.60:0/4069734098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eb3650 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f637809a880 tx=0x7f63780076e0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.412+0000 7f63861b7740 1 -- 172.21.15.60:0/4069734098 shutdown_connections 2024-02-26T04:57:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.412+0000 7f63861b7740 1 --2- 172.21.15.60:0/4069734098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.412+0000 7f63861b7740 1 --2- 172.21.15.60:0/4069734098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eb3650 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.412+0000 7f63861b7740 1 --2- 172.21.15.60:0/4069734098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea22a0 0x55b4d2eaa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.412+0000 7f63861b7740 1 -- 172.21.15.60:0/4069734098 >> 172.21.15.60:0/4069734098 conn(0x55b4d2e9d7c0 msgr2=0x55b4d2ea3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.412+0000 7f63861b7740 1 -- 172.21.15.60:0/4069734098 shutdown_connections 2024-02-26T04:57:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.412+0000 7f63861b7740 1 -- 172.21.15.60:0/4069734098 wait complete. 2024-02-26T04:57:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.412+0000 7f63861b7740 1 -- start start 2024-02-26T04:57:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.412+0000 7f63861b7740 1 Processor -- start 2024-02-26T04:57:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63861b7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea22a0 0x55b4d2eb32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63861b7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eb1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63861b7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eb1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63861b7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d2eb2380 con 0x55b4d2ea22a0 2024-02-26T04:57:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63861b7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d2eb24f0 con 0x55b4d2ea2c90 2024-02-26T04:57:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63861b7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d2eb26b0 con 0x55b4d2ea7e00 2024-02-26T04:57:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63861b7740 10 client.0 _ll_register_callbacks cb 0x7f637800a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:57:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63810a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea22a0 0x55b4d2eb32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63810a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea22a0 0x55b4d2eb32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44436/0 (socket says 172.21.15.60:44436) 2024-02-26T04:57:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63810a4640 1 -- 172.21.15.60:0/71560813 learned_addr learned my addr 172.21.15.60:0/71560813 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:57:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63818a5640 1 --2- 172.21.15.60:0/71560813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eb1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63808a3640 1 --2- 172.21.15.60:0/71560813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eb1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63810a4640 1 -- 172.21.15.60:0/71560813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea7e00 msgr2=0x55b4d2eb1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63810a4640 1 --2- 172.21.15.60:0/71560813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eb1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63810a4640 1 -- 172.21.15.60:0/71560813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea2c90 msgr2=0x55b4d2eb1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:57:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63810a4640 1 --2- 172.21.15.60:0/71560813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eb1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63810a4640 1 -- 172.21.15.60:0/71560813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4d2dc9540 con 0x55b4d2ea22a0 2024-02-26T04:57:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63808a3640 1 --2- 172.21.15.60:0/71560813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eb1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:57:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63810a4640 1 --2- 172.21.15.60:0/71560813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea22a0 0x55b4d2eb32f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f636c005c40 tx=0x7f636c02c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9da699c7d306891 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:57:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f6364ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:57:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f6364ff9640 1 -- 172.21.15.60:0/71560813 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c004040 con 0x55b4d2ea22a0 2024-02-26T04:57:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f6364ff9640 1 -- 172.21.15.60:0/71560813 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f636c0041a0 con 0x55b4d2ea22a0 2024-02-26T04:57:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.413+0000 7f63861b7740 1 -- 172.21.15.60:0/71560813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f637809a4e0 con 0x55b4d2ea22a0 2024-02-26T04:57:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.414+0000 7f6364ff9640 1 -- 172.21.15.60:0/71560813 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f636c004360 con 0x55b4d2ea22a0 2024-02-26T04:57:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.414+0000 7f63861b7740 10 client.15600 fetch_fsmap learned FSMap version 23 2024-02-26T04:57:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.414+0000 7f63861b7740 1 -- 172.21.15.60:0/71560813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b4d2dcb670 con 0x55b4d2ea22a0 2024-02-26T04:57:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.414+0000 7f6364ff9640 1 -- 172.21.15.60:0/71560813 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f637809a4e0 con 0x55b4d2ea22a0 2024-02-26T04:57:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.414+0000 7f6364ff9640 1 -- 172.21.15.60:0/71560813 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f636c0038f0 con 0x55b4d2ea22a0 2024-02-26T04:57:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.414+0000 7f63861b7740 10 client.15600 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:57:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.414+0000 7f63861b7740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:57:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.414+0000 7f63861b7740 20 client.15600 populate_metadata read hostname 'smithi060' 2024-02-26T04:57:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.414+0000 7f63861b7740 1 --2- 172.21.15.60:0/71560813 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55b4d2f10110 0x55b4d2f12530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:57:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.414+0000 7f63861b7740 4 client.15600 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T04:57:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.414+0000 7f63861b7740 1 -- 172.21.15.60:0/71560813 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b4d2b96890 con 0x55b4d2f10110 2024-02-26T04:57:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.414+0000 7f63808a3640 1 --2- 172.21.15.60:0/71560813 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55b4d2f10110 0x55b4d2f12530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:57:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f6364ff9640 10 client.15600 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f63808a3640 1 --2- 172.21.15.60:0/71560813 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55b4d2f10110 0x55b4d2f12530 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a8d38add0d47db8 server_cookie=73d6b298d975aa4b in_seq=0 out_seq=0 2024-02-26T04:57:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f6364ff9640 1 -- 172.21.15.60:0/71560813 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b4d2b96890 con 0x55b4d2f10110 2024-02-26T04:57:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f6364ff9640 10 client.15600 handle_command_reply: tid=0 2024-02-26T04:57:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f63861b7740 1 client.15600 shutdown 2024-02-26T04:57:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f63861b7740 1 -- 172.21.15.60:0/71560813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea22a0 msgr2=0x55b4d2eb32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f63861b7740 1 --2- 172.21.15.60:0/71560813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea22a0 0x55b4d2eb32f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f636c005c40 tx=0x7f636c02c3e0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f63861b7740 1 -- 172.21.15.60:0/71560813 shutdown_connections 2024-02-26T04:57:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f63861b7740 1 --2- 172.21.15.60:0/71560813 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55b4d2f10110 0x55b4d2f12530 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f63861b7740 1 --2- 172.21.15.60:0/71560813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b4d2ea7e00 0x55b4d2eb1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f63861b7740 1 --2- 172.21.15.60:0/71560813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4d2ea2c90 0x55b4d2eb1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f6364ff9640 0 client.15600 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:57:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f63861b7740 1 --2- 172.21.15.60:0/71560813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b4d2ea22a0 0x55b4d2eb32f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f63861b7740 1 -- 172.21.15.60:0/71560813 >> 172.21.15.60:0/71560813 conn(0x55b4d2e9d7c0 msgr2=0x55b4d2ea3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f63861b7740 1 -- 172.21.15.60:0/71560813 shutdown_connections 2024-02-26T04:57:59.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f63861b7740 1 -- 172.21.15.60:0/71560813 wait complete. 2024-02-26T04:57:59.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.415+0000 7f63861b7740 20 client.15600 trim_cache size 0 max 16384 2024-02-26T04:57:59.425 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:57:59.425 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T04:57:59.425 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-02-26T04:57:59.425 INFO:teuthology.orchestra.run.smithi060.stdout: "5c65524b-a581-4347-8136-767f0f20980e": { 2024-02-26T04:57:59.425 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-02-26T04:57:59.425 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "5c65524b-a581-4347-8136-767f0f20980e", 2024-02-26T04:57:59.425 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-02-26T04:57:59.425 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:57:59.426 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:57:59.426 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:57:59.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.416+0000 7f6383308640 1 -- 172.21.15.60:0/3782401028 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f63680a20c0 msgr2=0x7f63680a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.416+0000 7f6383308640 1 --2- 172.21.15.60:0/3782401028 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f63680a20c0 0x7f63680a4580 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6374005fd0 tx=0x7f6374010740 comp rx=0 tx=0).stop 2024-02-26T04:57:59.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.416+0000 7f6383308640 1 -- 172.21.15.60:0/3782401028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c104590 msgr2=0x7f637c197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:57:59.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.416+0000 7f6383308640 1 --2- 172.21.15.60:0/3782401028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c104590 0x7f637c197700 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f637800c970 tx=0x7f637800ce30 comp rx=0 tx=0).stop 2024-02-26T04:57:59.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.416+0000 7f6383308640 1 -- 172.21.15.60:0/3782401028 shutdown_connections 2024-02-26T04:57:59.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.416+0000 7f6383308640 1 --2- 172.21.15.60:0/3782401028 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f63680a20c0 0x7f63680a4580 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.416+0000 7f6383308640 1 --2- 172.21.15.60:0/3782401028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f637c106ec0 0x7f637c19e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.416+0000 7f6383308640 1 --2- 172.21.15.60:0/3782401028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c104590 0x7f637c197700 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.416+0000 7f6383308640 1 --2- 172.21.15.60:0/3782401028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f637c101c60 0x7f637c1971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:57:59.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.416+0000 7f6383308640 1 -- 172.21.15.60:0/3782401028 >> 172.21.15.60:0/3782401028 conn(0x7f637c0fb5f0 msgr2=0x7f637c1025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:57:59.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.416+0000 7f6383308640 1 -- 172.21.15.60:0/3782401028 shutdown_connections 2024-02-26T04:57:59.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:57:59.416+0000 7f6383308640 1 -- 172.21.15.60:0/3782401028 wait complete. 2024-02-26T04:57:59.429 INFO:tasks.cephfs.filesystem:scrub status for tag:5c65524b-a581-4347-8136-767f0f20980e - {'path': '/', 'tag': '5c65524b-a581-4347-8136-767f0f20980e', 'options': 'recursive,force'} 2024-02-26T04:57:59.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:59 smithi156 ceph-mon[35805]: pgmap v373: 129 pgs: 129 active+clean; 648 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-02-26T04:57:59.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:59 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2039249240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:59.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:59 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1155616948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:59.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:57:59 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1322759851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:59.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:59 smithi060 ceph-mon[32382]: pgmap v373: 129 pgs: 129 active+clean; 648 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-02-26T04:57:59.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:59 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2039249240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:59.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:59 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1155616948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:59.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:57:59 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1322759851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:57:59.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:59 smithi191 ceph-mon[36352]: pgmap v373: 129 pgs: 129 active+clean; 648 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-02-26T04:57:59.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:59 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2039249240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:57:59.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:59 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1155616948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:57:59.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:57:59 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1322759851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:58:00.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:00 smithi060 ceph-mon[32382]: scrub queued for path: / 2024-02-26T04:58:00.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:00 smithi060 ceph-mon[32382]: scrub summary: idle+waiting paths [/] 2024-02-26T04:58:00.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:00 smithi060 ceph-mon[32382]: scrub summary: active paths [/] 2024-02-26T04:58:00.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:00 smithi060 ceph-mon[32382]: scrub summary: idle+waiting paths [/] 2024-02-26T04:58:00.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:00 smithi156 ceph-mon[35805]: scrub queued for path: / 2024-02-26T04:58:00.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:00 smithi156 ceph-mon[35805]: scrub summary: idle+waiting paths [/] 2024-02-26T04:58:00.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:00 smithi156 ceph-mon[35805]: scrub summary: active paths [/] 2024-02-26T04:58:00.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:00 smithi156 ceph-mon[35805]: scrub summary: idle+waiting paths [/] 2024-02-26T04:58:00.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:00 smithi191 ceph-mon[36352]: scrub queued for path: / 2024-02-26T04:58:00.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:00 smithi191 ceph-mon[36352]: scrub summary: idle+waiting paths [/] 2024-02-26T04:58:00.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:00 smithi191 ceph-mon[36352]: scrub summary: active paths [/] 2024-02-26T04:58:00.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:00 smithi191 ceph-mon[36352]: scrub summary: idle+waiting paths [/] 2024-02-26T04:58:01.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:01 smithi156 ceph-mon[35805]: pgmap v374: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-02-26T04:58:01.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:01 smithi060 ceph-mon[32382]: pgmap v374: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-02-26T04:58:01.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:01 smithi191 ceph-mon[36352]: pgmap v374: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-02-26T04:58:03.679 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:03 smithi156 ceph-mon[35805]: pgmap v375: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-02-26T04:58:03.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:03 smithi060 ceph-mon[32382]: pgmap v375: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-02-26T04:58:03.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:03 smithi191 ceph-mon[36352]: pgmap v375: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-02-26T04:58:04.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:04 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:58:04.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:04 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:58:04.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:04 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:58:05.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:05 smithi156 ceph-mon[35805]: pgmap v376: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-02-26T04:58:05.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:05 smithi060 ceph-mon[32382]: pgmap v376: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-02-26T04:58:05.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:05 smithi191 ceph-mon[36352]: pgmap v376: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-02-26T04:58:06.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:06 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:06.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:06 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:06.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:06 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:06.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:06 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:06.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:06 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:06.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:06 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:06.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:06 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:58:06.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:06 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:58:06.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:06 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:07.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:06 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:07.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:06 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:07.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:06 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:07.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:06 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:07.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:06 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:07.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:06 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:07.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:06 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:58:07.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:06 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:58:07.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:06 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:07.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:06 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:07.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:06 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:07.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:06 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:07.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:06 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:07.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:06 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:07.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:06 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:07.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:06 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:58:07.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:06 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:58:07.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:06 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:58:08.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:07 smithi156 ceph-mon[35805]: pgmap v377: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-02-26T04:58:08.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:07 smithi060 ceph-mon[32382]: pgmap v377: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-02-26T04:58:08.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:07 smithi191 ceph-mon[36352]: pgmap v377: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-02-26T04:58:10.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:09 smithi156 ceph-mon[35805]: pgmap v378: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-26T04:58:10.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:09 smithi060 ceph-mon[32382]: pgmap v378: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-26T04:58:10.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:09 smithi191 ceph-mon[36352]: pgmap v378: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-26T04:58:12.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:11 smithi156 ceph-mon[35805]: pgmap v379: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-26T04:58:12.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:11 smithi060 ceph-mon[32382]: pgmap v379: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-26T04:58:12.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:11 smithi191 ceph-mon[36352]: pgmap v379: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-26T04:58:13.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:12 smithi156 ceph-mon[35805]: scrub summary: idle 2024-02-26T04:58:13.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:12 smithi060 ceph-mon[32382]: scrub summary: idle 2024-02-26T04:58:13.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:12 smithi191 ceph-mon[36352]: scrub summary: idle 2024-02-26T04:58:14.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:13 smithi156 ceph-mon[35805]: pgmap v380: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T04:58:14.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:13 smithi060 ceph-mon[32382]: pgmap v380: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T04:58:14.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:13 smithi191 ceph-mon[36352]: pgmap v380: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T04:58:16.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:15 smithi156 ceph-mon[35805]: pgmap v381: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T04:58:16.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:15 smithi060 ceph-mon[32382]: pgmap v381: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T04:58:16.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:15 smithi191 ceph-mon[36352]: pgmap v381: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T04:58:18.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:17 smithi156 ceph-mon[35805]: pgmap v382: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:58:18.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:17 smithi060 ceph-mon[32382]: pgmap v382: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:58:18.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:17 smithi191 ceph-mon[36352]: pgmap v382: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:58:20.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:19 smithi156 ceph-mon[35805]: pgmap v383: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:58:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:19 smithi060 ceph-mon[32382]: pgmap v383: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:58:20.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:19 smithi191 ceph-mon[36352]: pgmap v383: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:58:22.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:21 smithi156 ceph-mon[35805]: pgmap v384: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:58:22.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:21 smithi060 ceph-mon[32382]: pgmap v384: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:58:22.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:21 smithi191 ceph-mon[36352]: pgmap v384: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:58:24.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:23 smithi156 ceph-mon[35805]: pgmap v385: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:58:24.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:23 smithi060 ceph-mon[32382]: pgmap v385: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:58:24.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:23 smithi191 ceph-mon[36352]: pgmap v385: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:58:26.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:25 smithi156 ceph-mon[35805]: pgmap v386: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:58:26.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:25 smithi060 ceph-mon[32382]: pgmap v386: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:58:26.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:25 smithi191 ceph-mon[36352]: pgmap v386: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:58:28.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:27 smithi156 ceph-mon[35805]: pgmap v387: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:58:28.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:27 smithi060 ceph-mon[32382]: pgmap v387: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:58:28.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:27 smithi191 ceph-mon[36352]: pgmap v387: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:58:29.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T04:58:29.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T04:58:29.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.594+0000 7f8a86115640 1 -- 172.21.15.60:0/1242062840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a80102d60 msgr2=0x7f8a801095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.594+0000 7f8a86115640 1 --2- 172.21.15.60:0/1242062840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a80102d60 0x7f8a801095f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009ca0 tx=0x7f8a7802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.594+0000 7f8a86115640 1 -- 172.21.15.60:0/1242062840 shutdown_connections 2024-02-26T04:58:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.594+0000 7f8a86115640 1 --2- 172.21.15.60:0/1242062840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a80102d60 0x7f8a801095f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.594+0000 7f8a86115640 1 --2- 172.21.15.60:0/1242062840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a801023a0 0x7f8a80102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.594+0000 7f8a86115640 1 --2- 172.21.15.60:0/1242062840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a801011b0 0x7f8a801015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.594+0000 7f8a86115640 1 -- 172.21.15.60:0/1242062840 >> 172.21.15.60:0/1242062840 conn(0x7f8a800fc740 msgr2=0x7f8a800feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.594+0000 7f8a86115640 1 -- 172.21.15.60:0/1242062840 shutdown_connections 2024-02-26T04:58:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.594+0000 7f8a86115640 1 -- 172.21.15.60:0/1242062840 wait complete. 2024-02-26T04:58:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.595+0000 7f8a86115640 1 Processor -- start 2024-02-26T04:58:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.595+0000 7f8a86115640 1 -- start start 2024-02-26T04:58:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.595+0000 7f8a86115640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a801011b0 0x7f8a80074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.595+0000 7f8a86115640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a801023a0 0x7f8a80073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.595+0000 7f8a86115640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a80102d60 0x7f8a80073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.595+0000 7f8a86115640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80074210 con 0x7f8a801023a0 2024-02-26T04:58:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.595+0000 7f8a86115640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80074380 con 0x7f8a80102d60 2024-02-26T04:58:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.595+0000 7f8a86115640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a801a1220 con 0x7f8a801011b0 2024-02-26T04:58:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.595+0000 7f8a7effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a801023a0 0x7f8a80073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.595+0000 7f8a7effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a801023a0 0x7f8a80073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59024/0 (socket says 172.21.15.60:59024) 2024-02-26T04:58:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.595+0000 7f8a7effd640 1 -- 172.21.15.60:0/4066190202 learned_addr learned my addr 172.21.15.60:0/4066190202 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.595+0000 7f8a7f7fe640 1 --2- 172.21.15.60:0/4066190202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a801011b0 0x7f8a80074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.595+0000 7f8a7ffff640 1 --2- 172.21.15.60:0/4066190202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a80102d60 0x7f8a80073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.596+0000 7f8a7effd640 1 -- 172.21.15.60:0/4066190202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a801011b0 msgr2=0x7f8a80074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.596+0000 7f8a7effd640 1 --2- 172.21.15.60:0/4066190202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a801011b0 0x7f8a80074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.596+0000 7f8a7effd640 1 -- 172.21.15.60:0/4066190202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a80102d60 msgr2=0x7f8a80073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.596+0000 7f8a7effd640 1 --2- 172.21.15.60:0/4066190202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a80102d60 0x7f8a80073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.596+0000 7f8a7effd640 1 -- 172.21.15.60:0/4066190202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a78009900 con 0x7f8a801023a0 2024-02-26T04:58:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.596+0000 7f8a7effd640 1 --2- 172.21.15.60:0/4066190202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a801023a0 0x7f8a80073570 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400ea20 tx=0x7f8a7400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.596+0000 7f8a7cff9640 1 -- 172.21.15.60:0/4066190202 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74009700 con 0x7f8a801023a0 2024-02-26T04:58:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.596+0000 7f8a86115640 1 -- 172.21.15.60:0/4066190202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a801a14d0 con 0x7f8a801023a0 2024-02-26T04:58:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.596+0000 7f8a7cff9640 1 -- 172.21.15.60:0/4066190202 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a7400cd80 con 0x7f8a801023a0 2024-02-26T04:58:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.596+0000 7f8a86115640 1 -- 172.21.15.60:0/4066190202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a801a1a30 con 0x7f8a801023a0 2024-02-26T04:58:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.596+0000 7f8a7cff9640 1 -- 172.21.15.60:0/4066190202 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74018840 con 0x7f8a801023a0 2024-02-26T04:58:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.598+0000 7f8a7cff9640 1 -- 172.21.15.60:0/4066190202 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8a740189a0 con 0x7f8a801023a0 2024-02-26T04:58:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.599+0000 7f8a7cff9640 1 --2- 172.21.15.60:0/4066190202 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a680a2140 0x7f8a680a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.599+0000 7f8a7cff9640 1 -- 172.21.15.60:0/4066190202 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8a74014070 con 0x7f8a801023a0 2024-02-26T04:58:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.599+0000 7f8a7f7fe640 1 --2- 172.21.15.60:0/4066190202 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a680a2140 0x7f8a680a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.599+0000 7f8a88fc5740 1 Processor -- start 2024-02-26T04:58:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.599+0000 7f8a88fc5740 1 -- start start 2024-02-26T04:58:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a88fc5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd52a0 0x5558e5fd56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a88fc5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd5c90 0x5558e5fda790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a88fc5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 0x5558e5fdd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a88fc5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e5f45250 con 0x5558e5fd5c90 2024-02-26T04:58:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a88fc5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e5f42780 con 0x5558e5fd52a0 2024-02-26T04:58:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a88fc5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e5f3df60 con 0x5558e5fdae00 2024-02-26T04:58:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a7f7fe640 1 --2- 172.21.15.60:0/4066190202 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a680a2140 0x7f8a680a4600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8a80102200 tx=0x7f8a70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a7effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd5c90 0x5558e5fda790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a7effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd5c90 0x5558e5fda790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59036/0 (socket says 172.21.15.60:59036) 2024-02-26T04:58:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a7effd640 1 -- 172.21.15.60:0/2583343970 learned_addr learned my addr 172.21.15.60:0/2583343970 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a7f7fe640 1 --2- 172.21.15.60:0/2583343970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd52a0 0x5558e5fd56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a7ffff640 1 --2- 172.21.15.60:0/2583343970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 0x5558e5fdd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a7effd640 1 -- 172.21.15.60:0/2583343970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 msgr2=0x5558e5fdd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a7effd640 1 --2- 172.21.15.60:0/2583343970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 0x5558e5fdd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a7effd640 1 -- 172.21.15.60:0/2583343970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd52a0 msgr2=0x5558e5fd56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a7effd640 1 --2- 172.21.15.60:0/2583343970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd52a0 0x5558e5fd56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.600+0000 7f8a7effd640 1 -- 172.21.15.60:0/2583343970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558e5efc540 con 0x5558e5fd5c90 2024-02-26T04:58:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.601+0000 7f8a7effd640 1 --2- 172.21.15.60:0/2583343970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd5c90 0x5558e5fda790 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400e9f0 tx=0x7f8a7409a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307c9a49c6f92c6d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.601+0000 7f8a65ffb640 1 -- 172.21.15.60:0/2583343970 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74098ec0 con 0x5558e5fd5c90 2024-02-26T04:58:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.601+0000 7f8a65ffb640 1 -- 172.21.15.60:0/2583343970 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a74097230 con 0x5558e5fd5c90 2024-02-26T04:58:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.601+0000 7f8a65ffb640 1 -- 172.21.15.60:0/2583343970 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7409a380 con 0x5558e5fd5c90 2024-02-26T04:58:29.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.602+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2583343970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd5c90 msgr2=0x5558e5fda790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:29.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.602+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/2583343970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd5c90 0x5558e5fda790 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400e9f0 tx=0x7f8a7409a5e0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.602+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2583343970 shutdown_connections 2024-02-26T04:58:29.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.602+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/2583343970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 0x5558e5fdd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.602+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/2583343970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd5c90 0x5558e5fda790 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.602+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/2583343970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd52a0 0x5558e5fd56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.602+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2583343970 >> 172.21.15.60:0/2583343970 conn(0x5558e5fd07c0 msgr2=0x5558e5fd0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:29.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.602+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2583343970 shutdown_connections 2024-02-26T04:58:29.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.602+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2583343970 wait complete. 2024-02-26T04:58:29.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.602+0000 7f8a88fc5740 1 -- start start 2024-02-26T04:58:29.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.602+0000 7f8a88fc5740 1 Processor -- start 2024-02-26T04:58:29.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a88fc5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fd52a0 0x5558e5fdf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a88fc5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 0x5558e5fdf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a88fc5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fdae00 0x5558e5fe67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a88fc5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e5c11430 con 0x5558e5fdae00 2024-02-26T04:58:29.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a88fc5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e5bcde00 con 0x5558e5fd5c90 2024-02-26T04:58:29.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a88fc5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e5ec2fa0 con 0x5558e5fd52a0 2024-02-26T04:58:29.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a88fc5740 10 client.0 _ll_register_callbacks cb 0x5558e5f18fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:58:29.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fdae00 0x5558e5fe67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fdae00 0x5558e5fe67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59050/0 (socket says 172.21.15.60:59050) 2024-02-26T04:58:29.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a7ffff640 1 -- 172.21.15.60:0/3228116752 learned_addr learned my addr 172.21.15.60:0/3228116752 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:29.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a7effd640 1 --2- 172.21.15.60:0/3228116752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 0x5558e5fdf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a7f7fe640 1 --2- 172.21.15.60:0/3228116752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fd52a0 0x5558e5fdf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a7ffff640 1 -- 172.21.15.60:0/3228116752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fd52a0 msgr2=0x5558e5fdf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a7ffff640 1 --2- 172.21.15.60:0/3228116752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fd52a0 0x5558e5fdf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a7ffff640 1 -- 172.21.15.60:0/3228116752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 msgr2=0x5558e5fdf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a7ffff640 1 --2- 172.21.15.60:0/3228116752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 0x5558e5fdf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a7ffff640 1 -- 172.21.15.60:0/3228116752 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a801a1a30 con 0x5558e5fdae00 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a7f7fe640 1 --2- 172.21.15.60:0/3228116752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fd52a0 0x5558e5fdf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a7ffff640 1 --2- 172.21.15.60:0/3228116752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fdae00 0x5558e5fe67d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009ca0 tx=0x7f8a78009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d67a614e9f51da9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.603+0000 7f8a5effd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.604+0000 7f8a5effd640 1 -- 172.21.15.60:0/3228116752 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78032eb0 con 0x5558e5fdae00 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.604+0000 7f8a5effd640 1 -- 172.21.15.60:0/3228116752 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8a78046070 con 0x5558e5fdae00 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.604+0000 7f8a88fc5740 1 -- 172.21.15.60:0/3228116752 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558e5c3d7e0 con 0x5558e5fdae00 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.604+0000 7f8a5effd640 1 -- 172.21.15.60:0/3228116752 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a78034a90 con 0x5558e5fdae00 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.604+0000 7f8a88fc5740 10 client.15612 fetch_fsmap learned FSMap version 23 2024-02-26T04:58:29.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.604+0000 7f8a88fc5740 1 -- 172.21.15.60:0/3228116752 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5558e5f3c5e0 con 0x5558e5fdae00 2024-02-26T04:58:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.604+0000 7f8a5effd640 1 -- 172.21.15.60:0/3228116752 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5558e5c3d7e0 con 0x5558e5fdae00 2024-02-26T04:58:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.605+0000 7f8a5effd640 1 -- 172.21.15.60:0/3228116752 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a780042a0 con 0x5558e5fdae00 2024-02-26T04:58:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.605+0000 7f8a88fc5740 10 client.15612 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:58:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.605+0000 7f8a88fc5740 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:58:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.605+0000 7f8a88fc5740 20 client.15612 populate_metadata read hostname 'smithi060' 2024-02-26T04:58:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.605+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/3228116752 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558e5feb4d0 0x5558e5fed8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.605+0000 7f8a88fc5740 4 client.15612 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:58:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.605+0000 7f8a88fc5740 1 -- 172.21.15.60:0/3228116752 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558e5c7a720 con 0x5558e5feb4d0 2024-02-26T04:58:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.605+0000 7f8a7f7fe640 1 --2- 172.21.15.60:0/3228116752 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558e5feb4d0 0x5558e5fed8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.606+0000 7f8a5effd640 10 client.15612 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.606+0000 7f8a7f7fe640 1 --2- 172.21.15.60:0/3228116752 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558e5feb4d0 0x5558e5fed8f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e571c45d8a65f02 server_cookie=5ad9f247c9ff587 in_seq=0 out_seq=0 2024-02-26T04:58:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.606+0000 7f8a5effd640 1 -- 172.21.15.60:0/3228116752 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5558e5c7a720 con 0x5558e5feb4d0 2024-02-26T04:58:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.606+0000 7f8a5effd640 10 client.15612 handle_command_reply: tid=0 2024-02-26T04:58:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.606+0000 7f8a88fc5740 1 client.15612 shutdown 2024-02-26T04:58:29.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.607+0000 7f8a88fc5740 1 -- 172.21.15.60:0/3228116752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fdae00 msgr2=0x5558e5fe67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:29.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.607+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/3228116752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fdae00 0x5558e5fe67d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009ca0 tx=0x7f8a78009890 comp rx=0 tx=0).stop 2024-02-26T04:58:29.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.607+0000 7f8a88fc5740 1 -- 172.21.15.60:0/3228116752 shutdown_connections 2024-02-26T04:58:29.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.607+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/3228116752 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558e5feb4d0 0x5558e5fed8f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.607+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/3228116752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fdae00 0x5558e5fe67d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.607+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/3228116752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 0x5558e5fdf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.607+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/3228116752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fd52a0 0x5558e5fdf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.607+0000 7f8a88fc5740 1 -- 172.21.15.60:0/3228116752 >> 172.21.15.60:0/3228116752 conn(0x5558e5fd07c0 msgr2=0x5558e5fcfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:29.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.607+0000 7f8a5effd640 0 client.15612 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:29.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.607+0000 7f8a88fc5740 1 -- 172.21.15.60:0/3228116752 shutdown_connections 2024-02-26T04:58:29.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.607+0000 7f8a88fc5740 1 -- 172.21.15.60:0/3228116752 wait complete. 2024-02-26T04:58:29.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.607+0000 7f8a88fc5740 20 client.15612 trim_cache size 0 max 16384 2024-02-26T04:58:29.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.622+0000 7f8a88fc5740 1 Processor -- start 2024-02-26T04:58:29.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.622+0000 7f8a88fc5740 1 -- start start 2024-02-26T04:58:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.622+0000 7f8a88fc5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd52a0 0x5558e5fd0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.622+0000 7f8a88fc5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 0x5558e5fe66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.622+0000 7f8a88fc5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 0x5558e5fd6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.622+0000 7f8a88fc5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e5c3d810 con 0x5558e5fd52a0 2024-02-26T04:58:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.622+0000 7f8a88fc5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e5fd60f0 con 0x5558e5fd5c90 2024-02-26T04:58:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.622+0000 7f8a88fc5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e5ff2b20 con 0x5558e5fdae00 2024-02-26T04:58:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.622+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd52a0 0x5558e5fd0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.622+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd52a0 0x5558e5fd0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59066/0 (socket says 172.21.15.60:59066) 2024-02-26T04:58:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.622+0000 7f8a7f7fe640 1 -- 172.21.15.60:0/309375814 learned_addr learned my addr 172.21.15.60:0/309375814 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.623+0000 7f8a7ffff640 1 --2- 172.21.15.60:0/309375814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 0x5558e5fd6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.623+0000 7f8a7effd640 1 --2- 172.21.15.60:0/309375814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 0x5558e5fe66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.623+0000 7f8a7f7fe640 1 -- 172.21.15.60:0/309375814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 msgr2=0x5558e5fd6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.623+0000 7f8a7f7fe640 1 --2- 172.21.15.60:0/309375814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 0x5558e5fd6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.623+0000 7f8a7f7fe640 1 -- 172.21.15.60:0/309375814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 msgr2=0x5558e5fe66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.623+0000 7f8a7f7fe640 1 --2- 172.21.15.60:0/309375814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 0x5558e5fe66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.623+0000 7f8a7f7fe640 1 -- 172.21.15.60:0/309375814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558e5ed8a90 con 0x5558e5fd52a0 2024-02-26T04:58:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.623+0000 7f8a7f7fe640 1 --2- 172.21.15.60:0/309375814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd52a0 0x5558e5fd0ba0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x5558e5fee240 tx=0x7f8a70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886dce302042815c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.623+0000 7f8a657fa640 1 -- 172.21.15.60:0/309375814 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70005220 con 0x5558e5fd52a0 2024-02-26T04:58:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.623+0000 7f8a657fa640 1 -- 172.21.15.60:0/309375814 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a70023040 con 0x5558e5fd52a0 2024-02-26T04:58:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.623+0000 7f8a657fa640 1 -- 172.21.15.60:0/309375814 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7002a360 con 0x5558e5fd52a0 2024-02-26T04:58:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.624+0000 7f8a88fc5740 1 -- 172.21.15.60:0/309375814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd52a0 msgr2=0x5558e5fd0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.624+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/309375814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd52a0 0x5558e5fd0ba0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x5558e5fee240 tx=0x7f8a70014000 comp rx=0 tx=0).stop 2024-02-26T04:58:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.625+0000 7f8a88fc5740 1 -- 172.21.15.60:0/309375814 shutdown_connections 2024-02-26T04:58:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.625+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/309375814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 0x5558e5fd6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.625+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/309375814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 0x5558e5fe66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.625+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/309375814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd52a0 0x5558e5fd0ba0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.625+0000 7f8a88fc5740 1 -- 172.21.15.60:0/309375814 >> 172.21.15.60:0/309375814 conn(0x5558e5fd07c0 msgr2=0x5558e5fda520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.625+0000 7f8a88fc5740 1 -- 172.21.15.60:0/309375814 shutdown_connections 2024-02-26T04:58:29.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.625+0000 7f8a88fc5740 1 -- 172.21.15.60:0/309375814 wait complete. 2024-02-26T04:58:29.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.626+0000 7f8a88fc5740 1 -- start start 2024-02-26T04:58:29.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.626+0000 7f8a88fc5740 1 Processor -- start 2024-02-26T04:58:29.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a88fc5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd52a0 0x5558e5fe66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a88fc5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 0x5558e5fe4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a88fc5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 0x5558e5fe5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd52a0 0x5558e5fe66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd52a0 0x5558e5fe66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59082/0 (socket says 172.21.15.60:59082) 2024-02-26T04:58:29.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a7effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 0x5558e5fe4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a7effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 0x5558e5fe4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.60:35544/0 (socket says 172.21.15.60:35544) 2024-02-26T04:58:29.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a7f7fe640 1 -- 172.21.15.60:0/2374279455 learned_addr learned my addr 172.21.15.60:0/2374279455 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:29.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2374279455 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e5fe5a10 con 0x5558e5fd52a0 2024-02-26T04:58:29.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2374279455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e5fe5b80 con 0x5558e5fd5c90 2024-02-26T04:58:29.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2374279455 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e6041a30 con 0x5558e5fdae00 2024-02-26T04:58:29.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a7ffff640 1 --2- 172.21.15.60:0/2374279455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 0x5558e5fe5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a7effd640 1 -- 172.21.15.60:0/2374279455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 msgr2=0x5558e5fe5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a7effd640 1 --2- 172.21.15.60:0/2374279455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 0x5558e5fe5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a7effd640 1 -- 172.21.15.60:0/2374279455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd52a0 msgr2=0x5558e5fe66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a7effd640 1 --2- 172.21.15.60:0/2374279455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd52a0 0x5558e5fe66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a7effd640 1 -- 172.21.15.60:0/2374279455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a70017070 con 0x5558e5fd5c90 2024-02-26T04:58:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.627+0000 7f8a7f7fe640 1 --2- 172.21.15.60:0/2374279455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd52a0 0x5558e5fe66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.628+0000 7f8a88fc5740 10 client.0 _ll_register_callbacks cb 0x7f8a74096740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:58:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.628+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2374279455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a74097230 con 0x5558e5fd5c90 2024-02-26T04:58:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.628+0000 7f8a7effd640 1 --2- 172.21.15.60:0/2374279455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 0x5558e5fe4d00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8a74097590 tx=0x7f8a740967f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b4f807433b72d41 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.628+0000 7f8a5effd640 10 client.25232 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-26T04:58:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.628+0000 7f8a5effd640 1 -- 172.21.15.60:0/2374279455 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7409a210 con 0x5558e5fd5c90 2024-02-26T04:58:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.628+0000 7f8a5effd640 1 -- 172.21.15.60:0/2374279455 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8a740b3070 con 0x5558e5fd5c90 2024-02-26T04:58:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.628+0000 7f8a5effd640 1 -- 172.21.15.60:0/2374279455 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a7409a370 con 0x5558e5fd5c90 2024-02-26T04:58:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.628+0000 7f8a88fc5740 10 client.25232 fetch_fsmap learned FSMap version 23 2024-02-26T04:58:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.628+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2374279455 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({fsmap=23}) v3 -- 0x5558e5efe670 con 0x5558e5fd5c90 2024-02-26T04:58:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.629+0000 7f8a5effd640 1 -- 172.21.15.60:0/2374279455 <== mon.2 v2:172.21.15.156:3300/0 4 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a74096910 con 0x5558e5fd5c90 2024-02-26T04:58:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.629+0000 7f8a88fc5740 10 client.25232 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:58:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.629+0000 7f8a88fc5740 10 client.25232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:58:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.629+0000 7f8a88fc5740 20 client.25232 populate_metadata read hostname 'smithi060' 2024-02-26T04:58:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.629+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/2374279455 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558e6042d10 0x5558e6045100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.629+0000 7f8a88fc5740 4 client.25232 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T04:58:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.629+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2374279455 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5558e5cc9890 con 0x5558e6042d10 2024-02-26T04:58:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.629+0000 7f8a7f7fe640 1 --2- 172.21.15.60:0/2374279455 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558e6042d10 0x5558e6045100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.629+0000 7f8a5effd640 10 client.25232 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.629+0000 7f8a7f7fe640 1 --2- 172.21.15.60:0/2374279455 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558e6042d10 0x5558e6045100 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=942ba77d2b55831 server_cookie=1ed61f9c7c129a05 in_seq=0 out_seq=0 2024-02-26T04:58:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a5effd640 1 -- 172.21.15.60:0/2374279455 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5558e5cc9890 con 0x5558e6042d10 2024-02-26T04:58:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a5effd640 10 client.25232 handle_command_reply: tid=0 2024-02-26T04:58:29.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a88fc5740 1 client.25232 shutdown 2024-02-26T04:58:29.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2374279455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 msgr2=0x5558e5fe4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:29.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/2374279455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 0x5558e5fe4d00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8a74097590 tx=0x7f8a740967f0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2374279455 shutdown_connections 2024-02-26T04:58:29.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/2374279455 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558e6042d10 0x5558e6045100 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/2374279455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558e5fdae00 0x5558e5fe5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/2374279455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558e5fd5c90 0x5558e5fe4d00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a88fc5740 1 --2- 172.21.15.60:0/2374279455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558e5fd52a0 0x5558e5fe66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a5effd640 0 client.25232 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:29.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2374279455 >> 172.21.15.60:0/2374279455 conn(0x5558e5fd07c0 msgr2=0x5558e5fda390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:29.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2374279455 shutdown_connections 2024-02-26T04:58:29.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a88fc5740 1 -- 172.21.15.60:0/2374279455 wait complete. 2024-02-26T04:58:29.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.630+0000 7f8a88fc5740 20 client.25232 trim_cache size 0 max 16384 2024-02-26T04:58:29.641 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:58:29.641 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-02-26T04:58:29.641 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-02-26T04:58:29.641 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:58:29.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.631+0000 7f8a86115640 1 -- 172.21.15.60:0/4066190202 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a680a2140 msgr2=0x7f8a680a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:29.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.631+0000 7f8a86115640 1 --2- 172.21.15.60:0/4066190202 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a680a2140 0x7f8a680a4600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8a80102200 tx=0x7f8a70005f90 comp rx=0 tx=0).stop 2024-02-26T04:58:29.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.631+0000 7f8a86115640 1 -- 172.21.15.60:0/4066190202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a801023a0 msgr2=0x7f8a80073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:29.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.631+0000 7f8a86115640 1 --2- 172.21.15.60:0/4066190202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a801023a0 0x7f8a80073570 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400ea20 tx=0x7f8a7400eee0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.631+0000 7f8a86115640 1 -- 172.21.15.60:0/4066190202 shutdown_connections 2024-02-26T04:58:29.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.631+0000 7f8a86115640 1 --2- 172.21.15.60:0/4066190202 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8a680a2140 0x7f8a680a4600 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.631+0000 7f8a86115640 1 --2- 172.21.15.60:0/4066190202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a80102d60 0x7f8a80073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.631+0000 7f8a86115640 1 --2- 172.21.15.60:0/4066190202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a801023a0 0x7f8a80073570 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.631+0000 7f8a86115640 1 --2- 172.21.15.60:0/4066190202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a801011b0 0x7f8a80074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.631+0000 7f8a86115640 1 -- 172.21.15.60:0/4066190202 >> 172.21.15.60:0/4066190202 conn(0x7f8a800fc740 msgr2=0x7f8a80105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:29.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.631+0000 7f8a86115640 1 -- 172.21.15.60:0/4066190202 shutdown_connections 2024-02-26T04:58:29.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.631+0000 7f8a86115640 1 -- 172.21.15.60:0/4066190202 wait complete. 2024-02-26T04:58:29.649 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T04:58:29.649 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:58:29.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.750+0000 7ff675236640 1 -- 172.21.15.60:0/3245254711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff670100d70 msgr2=0x7ff6701011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:29.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.750+0000 7ff675236640 1 --2- 172.21.15.60:0/3245254711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff670100d70 0x7ff6701011f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff668009ca0 tx=0x7ff66802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.750+0000 7ff675236640 1 -- 172.21.15.60:0/3245254711 shutdown_connections 2024-02-26T04:58:29.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.750+0000 7ff675236640 1 --2- 172.21.15.60:0/3245254711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff670100d70 0x7ff6701011f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.750+0000 7ff675236640 1 --2- 172.21.15.60:0/3245254711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6701003d0 0x7ff670100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.750+0000 7ff675236640 1 --2- 172.21.15.60:0/3245254711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6700ffa70 0x7ff6700ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.750+0000 7ff675236640 1 -- 172.21.15.60:0/3245254711 >> 172.21.15.60:0/3245254711 conn(0x7ff6700fb610 msgr2=0x7ff6700fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:29.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.750+0000 7ff675236640 1 -- 172.21.15.60:0/3245254711 shutdown_connections 2024-02-26T04:58:29.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.750+0000 7ff675236640 1 -- 172.21.15.60:0/3245254711 wait complete. 2024-02-26T04:58:29.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.751+0000 7ff675236640 1 Processor -- start 2024-02-26T04:58:29.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.751+0000 7ff675236640 1 -- start start 2024-02-26T04:58:29.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.751+0000 7ff675236640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6700ffa70 0x7ff670199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.751+0000 7ff675236640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6701003d0 0x7ff670199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.751+0000 7ff675236640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff670100d70 0x7ff6701a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff675236640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6701a0ef0 con 0x7ff6700ffa70 2024-02-26T04:58:29.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff675236640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6701a1060 con 0x7ff670100d70 2024-02-26T04:58:29.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff675236640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6701a1220 con 0x7ff6701003d0 2024-02-26T04:58:29.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff66ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6700ffa70 0x7ff670199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff66ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6700ffa70 0x7ff670199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59102/0 (socket says 172.21.15.60:59102) 2024-02-26T04:58:29.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff66ed9d640 1 -- 172.21.15.60:0/4196293050 learned_addr learned my addr 172.21.15.60:0/4196293050 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:29.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff66f59e640 1 --2- 172.21.15.60:0/4196293050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff670100d70 0x7ff6701a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff66e59c640 1 --2- 172.21.15.60:0/4196293050 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6701003d0 0x7ff670199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff66ed9d640 1 -- 172.21.15.60:0/4196293050 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6701003d0 msgr2=0x7ff670199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:29.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff66ed9d640 1 --2- 172.21.15.60:0/4196293050 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6701003d0 0x7ff670199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff66ed9d640 1 -- 172.21.15.60:0/4196293050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff670100d70 msgr2=0x7ff6701a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:29.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff66ed9d640 1 --2- 172.21.15.60:0/4196293050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff670100d70 0x7ff6701a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:29.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff66ed9d640 1 -- 172.21.15.60:0/4196293050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff660009490 con 0x7ff6700ffa70 2024-02-26T04:58:29.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff66ed9d640 1 --2- 172.21.15.60:0/4196293050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6700ffa70 0x7ff670199400 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff66000ea60 tx=0x7ff66000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:29.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.752+0000 7ff657fff640 1 -- 172.21.15.60:0/4196293050 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66000cde0 con 0x7ff6700ffa70 2024-02-26T04:58:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.753+0000 7ff675236640 1 -- 172.21.15.60:0/4196293050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff668009900 con 0x7ff6700ffa70 2024-02-26T04:58:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.753+0000 7ff657fff640 1 -- 172.21.15.60:0/4196293050 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6600035f0 con 0x7ff6700ffa70 2024-02-26T04:58:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.753+0000 7ff657fff640 1 -- 172.21.15.60:0/4196293050 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660010690 con 0x7ff6700ffa70 2024-02-26T04:58:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.753+0000 7ff675236640 1 -- 172.21.15.60:0/4196293050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6701a18d0 con 0x7ff6700ffa70 2024-02-26T04:58:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.754+0000 7ff675236640 1 -- 172.21.15.60:0/4196293050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff670067f70 con 0x7ff6700ffa70 2024-02-26T04:58:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.755+0000 7ff657fff640 1 -- 172.21.15.60:0/4196293050 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff660003af0 con 0x7ff6700ffa70 2024-02-26T04:58:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.756+0000 7ff657fff640 1 --2- 172.21.15.60:0/4196293050 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6580a2140 0x7ff6580a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.756+0000 7ff657fff640 1 -- 172.21.15.60:0/4196293050 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff660014070 con 0x7ff6700ffa70 2024-02-26T04:58:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.756+0000 7ff66e59c640 1 --2- 172.21.15.60:0/4196293050 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6580a2140 0x7ff6580a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:29.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.756+0000 7ff66e59c640 1 --2- 172.21.15.60:0/4196293050 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6580a2140 0x7ff6580a4600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff67019a910 tx=0x7ff664005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:29.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:29.760+0000 7ff657fff640 1 -- 172.21.15.60:0/4196293050 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff6600970e0 con 0x7ff6700ffa70 2024-02-26T04:58:30.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.020+0000 7ff675236640 1 -- 172.21.15.60:0/4196293050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff67010ef70 con 0x7ff6700ffa70 2024-02-26T04:58:30.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.021+0000 7ff657fff640 1 -- 172.21.15.60:0/4196293050 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11347 (secure 0 0 0) 0x7ff66009a7a0 con 0x7ff6700ffa70 2024-02-26T04:58:30.025 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:58:30.025 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:57:52.162616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24476,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T04:58:30.025 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-02-26T04:58:30.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.024+0000 7ff675236640 1 -- 172.21.15.60:0/4196293050 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6580a2140 msgr2=0x7ff6580a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.024+0000 7ff675236640 1 --2- 172.21.15.60:0/4196293050 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6580a2140 0x7ff6580a4600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff67019a910 tx=0x7ff664005f90 comp rx=0 tx=0).stop 2024-02-26T04:58:30.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.024+0000 7ff675236640 1 -- 172.21.15.60:0/4196293050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6700ffa70 msgr2=0x7ff670199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.024+0000 7ff675236640 1 --2- 172.21.15.60:0/4196293050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6700ffa70 0x7ff670199400 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff66000ea60 tx=0x7ff66000ef20 comp rx=0 tx=0).stop 2024-02-26T04:58:30.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.024+0000 7ff675236640 1 -- 172.21.15.60:0/4196293050 shutdown_connections 2024-02-26T04:58:30.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.024+0000 7ff675236640 1 --2- 172.21.15.60:0/4196293050 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff6580a2140 0x7ff6580a4600 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.024+0000 7ff675236640 1 --2- 172.21.15.60:0/4196293050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff670100d70 0x7ff6701a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.024+0000 7ff675236640 1 --2- 172.21.15.60:0/4196293050 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6701003d0 0x7ff670199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.024+0000 7ff675236640 1 --2- 172.21.15.60:0/4196293050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6700ffa70 0x7ff670199400 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.024+0000 7ff675236640 1 -- 172.21.15.60:0/4196293050 >> 172.21.15.60:0/4196293050 conn(0x7ff6700fb610 msgr2=0x7ff6700fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.024+0000 7ff675236640 1 -- 172.21.15.60:0/4196293050 shutdown_connections 2024-02-26T04:58:30.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.024+0000 7ff675236640 1 -- 172.21.15.60:0/4196293050 wait complete. 2024-02-26T04:58:30.039 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T04:58:30.039 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T04:58:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.144+0000 7f9604236640 1 -- 172.21.15.60:0/350219239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc101c30 msgr2=0x7f95fc1084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.144+0000 7f9604236640 1 --2- 172.21.15.60:0/350219239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc101c30 0x7f95fc1084c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009ca0 tx=0x7f95f402c0f0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.144+0000 7f9604236640 1 -- 172.21.15.60:0/350219239 shutdown_connections 2024-02-26T04:58:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.144+0000 7f9604236640 1 --2- 172.21.15.60:0/350219239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc101c30 0x7f95fc1084c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.144+0000 7f9604236640 1 --2- 172.21.15.60:0/350219239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95fc101270 0x7f95fc1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.144+0000 7f9604236640 1 --2- 172.21.15.60:0/350219239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95fc100080 0x7f95fc1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.144+0000 7f9604236640 1 -- 172.21.15.60:0/350219239 >> 172.21.15.60:0/350219239 conn(0x7f95fc0fb610 msgr2=0x7f95fc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.144+0000 7f9604236640 1 -- 172.21.15.60:0/350219239 shutdown_connections 2024-02-26T04:58:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.144+0000 7f9604236640 1 -- 172.21.15.60:0/350219239 wait complete. 2024-02-26T04:58:30.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.145+0000 7f9604236640 1 Processor -- start 2024-02-26T04:58:30.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.145+0000 7f9604236640 1 -- start start 2024-02-26T04:58:30.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.145+0000 7f9604236640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc100080 0x7f95fc1960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.145+0000 7f9604236640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95fc101270 0x7f95fc196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.145+0000 7f9604236640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95fc101c30 0x7f95fc19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.145+0000 7f9604236640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc19dbd0 con 0x7f95fc100080 2024-02-26T04:58:30.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.145+0000 7f9604236640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc19dd40 con 0x7f95fc101c30 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.145+0000 7f9604236640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc19df00 con 0x7f95fc101270 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.145+0000 7f9601fd2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc100080 0x7f95fc1960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.145+0000 7f96017d1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95fc101270 0x7f95fc196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.146+0000 7f9601fd2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc100080 0x7f95fc1960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59118/0 (socket says 172.21.15.60:59118) 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.146+0000 7f96017d1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95fc101270 0x7f95fc196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.60:49964/0 (socket says 172.21.15.60:49964) 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.146+0000 7f9601fd2640 1 -- 172.21.15.60:0/4277531693 learned_addr learned my addr 172.21.15.60:0/4277531693 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.146+0000 7f96027d3640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95fc101c30 0x7f95fc19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.146+0000 7f96017d1640 1 -- 172.21.15.60:0/4277531693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95fc101c30 msgr2=0x7f95fc19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.146+0000 7f96017d1640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95fc101c30 0x7f95fc19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.146+0000 7f96017d1640 1 -- 172.21.15.60:0/4277531693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc100080 msgr2=0x7f95fc1960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.146+0000 7f96017d1640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc100080 0x7f95fc1960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.146+0000 7f96017d1640 1 -- 172.21.15.60:0/4277531693 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f4009900 con 0x7f95fc101270 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.146+0000 7f9601fd2640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc100080 0x7f95fc1960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.146+0000 7f96027d3640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95fc101c30 0x7f95fc19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.146+0000 7f96017d1640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95fc101270 0x7f95fc196620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009d20 tx=0x7f95ec00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.146+0000 7f95f2ffd640 1 -- 172.21.15.60:0/4277531693 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec012250 con 0x7f95fc101270 2024-02-26T04:58:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.147+0000 7f95f2ffd640 1 -- 172.21.15.60:0/4277531693 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95ec011c30 con 0x7f95fc101270 2024-02-26T04:58:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.147+0000 7f9604236640 1 -- 172.21.15.60:0/4277531693 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc068340 con 0x7f95fc101270 2024-02-26T04:58:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.147+0000 7f95f2ffd640 1 -- 172.21.15.60:0/4277531693 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec01a990 con 0x7f95fc101270 2024-02-26T04:58:30.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.147+0000 7f9604236640 1 -- 172.21.15.60:0/4277531693 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc0687f0 con 0x7f95fc101270 2024-02-26T04:58:30.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.150+0000 7f9607013740 1 Processor -- start 2024-02-26T04:58:30.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.151+0000 7f9607013740 1 -- start start 2024-02-26T04:58:30.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.151+0000 7f9607013740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80582a50 0x55ab80582e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.151+0000 7f9607013740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab80583440 0x55ab80587f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.151+0000 7f9607013740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab805885b0 0x55ab8058aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.151+0000 7f9607013740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab80536930 con 0x55ab805885b0 2024-02-26T04:58:30.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.151+0000 7f9607013740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab804ea2e0 con 0x55ab80583440 2024-02-26T04:58:30.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.151+0000 7f9607013740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab804e7810 con 0x55ab80582a50 2024-02-26T04:58:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.151+0000 7f96027d3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80582a50 0x55ab80582e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.151+0000 7f9601fd2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab80583440 0x55ab80587f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.151+0000 7f96027d3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80582a50 0x55ab80582e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.60:49966/0 (socket says 172.21.15.60:49966) 2024-02-26T04:58:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.151+0000 7f96027d3640 1 -- 172.21.15.60:0/2451012566 learned_addr learned my addr 172.21.15.60:0/2451012566 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.151+0000 7f95f2ffd640 1 -- 172.21.15.60:0/4277531693 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f95ec012440 con 0x7f95fc101270 2024-02-26T04:58:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.151+0000 7f9601fd2640 1 -- 172.21.15.60:0/2451012566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80582a50 msgr2=0x55ab80582e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.151+0000 7f9601fd2640 1 --2- 172.21.15.60:0/2451012566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80582a50 0x55ab80582e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.152+0000 7f9601fd2640 1 -- 172.21.15.60:0/2451012566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab805885b0 msgr2=0x55ab8058aa40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-26T04:58:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.152+0000 7f96017d1640 1 --2- 172.21.15.60:0/2451012566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab805885b0 0x55ab8058aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.152+0000 7f9601fd2640 1 --2- 172.21.15.60:0/2451012566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab805885b0 0x55ab8058aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.152+0000 7f9601fd2640 1 -- 172.21.15.60:0/2451012566 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab804a5030 con 0x55ab80583440 2024-02-26T04:58:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.152+0000 7f96017d1640 1 --2- 172.21.15.60:0/2451012566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab805885b0 0x55ab8058aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:58:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.152+0000 7f95f2ffd640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f95e80a20c0 0x7f95e80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.152+0000 7f9601fd2640 1 --2- 172.21.15.60:0/2451012566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab80583440 0x55ab80587f40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f95f80058c0 tx=0x7f95f8011950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e8748bfed569ea6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.152+0000 7f95f2ffd640 1 -- 172.21.15.60:0/4277531693 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f95ec0cdc40 con 0x7f95fc101270 2024-02-26T04:58:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.152+0000 7f95e7fff640 1 -- 172.21.15.60:0/2451012566 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8004030 con 0x55ab80583440 2024-02-26T04:58:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.153+0000 7f96027d3640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f95e80a20c0 0x7f95e80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.153+0000 7f95e7fff640 1 -- 172.21.15.60:0/2451012566 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95f80035f0 con 0x55ab80583440 2024-02-26T04:58:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.153+0000 7f95e7fff640 1 -- 172.21.15.60:0/2451012566 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f800f900 con 0x55ab80583440 2024-02-26T04:58:30.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.153+0000 7f9607013740 1 -- 172.21.15.60:0/2451012566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab80583440 msgr2=0x55ab80587f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.153+0000 7f96027d3640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f95e80a20c0 0x7f95e80a4580 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55ab80583ca0 tx=0x7f95f40326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.153+0000 7f9607013740 1 --2- 172.21.15.60:0/2451012566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab80583440 0x55ab80587f40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f95f80058c0 tx=0x7f95f8011950 comp rx=0 tx=0).stop 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.154+0000 7f9607013740 1 -- 172.21.15.60:0/2451012566 shutdown_connections 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.154+0000 7f9607013740 1 --2- 172.21.15.60:0/2451012566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab805885b0 0x55ab8058aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.154+0000 7f9607013740 1 --2- 172.21.15.60:0/2451012566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab80583440 0x55ab80587f40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.154+0000 7f9607013740 1 --2- 172.21.15.60:0/2451012566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80582a50 0x55ab80582e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.154+0000 7f9607013740 1 -- 172.21.15.60:0/2451012566 >> 172.21.15.60:0/2451012566 conn(0x55ab805757b0 msgr2=0x55ab80575bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.154+0000 7f9607013740 1 -- 172.21.15.60:0/2451012566 shutdown_connections 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.154+0000 7f9607013740 1 -- 172.21.15.60:0/2451012566 wait complete. 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.154+0000 7f9607013740 1 -- start start 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.154+0000 7f9607013740 1 Processor -- start 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9607013740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab8057bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9607013740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80583440 0x55ab8057c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9607013740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab805885b0 0x55ab80581030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9607013740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab804e2ff0 con 0x55ab80582a50 2024-02-26T04:58:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9607013740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab801b6430 con 0x55ab805885b0 2024-02-26T04:58:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9607013740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab80172e00 con 0x55ab80583440 2024-02-26T04:58:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9601fd2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab8057bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9607013740 10 client.0 _ll_register_callbacks cb 0x55ab8048ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:58:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9601fd2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab8057bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59132/0 (socket says 172.21.15.60:59132) 2024-02-26T04:58:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9601fd2640 1 -- 172.21.15.60:0/2272103483 learned_addr learned my addr 172.21.15.60:0/2272103483 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f96027d3640 1 --2- 172.21.15.60:0/2272103483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab805885b0 0x55ab80581030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f96017d1640 1 --2- 172.21.15.60:0/2272103483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80583440 0x55ab8057c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9601fd2640 1 -- 172.21.15.60:0/2272103483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80583440 msgr2=0x55ab8057c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9601fd2640 1 --2- 172.21.15.60:0/2272103483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80583440 0x55ab8057c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9601fd2640 1 -- 172.21.15.60:0/2272103483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab805885b0 msgr2=0x55ab80581030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9601fd2640 1 --2- 172.21.15.60:0/2272103483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab805885b0 0x55ab80581030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9601fd2640 1 -- 172.21.15.60:0/2272103483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f8009cd0 con 0x55ab80582a50 2024-02-26T04:58:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f96027d3640 1 --2- 172.21.15.60:0/2272103483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab805885b0 0x55ab80581030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f9601fd2640 1 --2- 172.21.15.60:0/2272103483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab8057bac0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009600 tx=0x7f95f80041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a26b871ff9d0d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.155+0000 7f95e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:58:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.156+0000 7f95e57fa640 1 -- 172.21.15.60:0/2272103483 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8005040 con 0x55ab80582a50 2024-02-26T04:58:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.156+0000 7f95e57fa640 1 -- 172.21.15.60:0/2272103483 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f95f80051a0 con 0x55ab80582a50 2024-02-26T04:58:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.156+0000 7f9607013740 1 -- 172.21.15.60:0/2272103483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab801e27e0 con 0x55ab80582a50 2024-02-26T04:58:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.156+0000 7f95e57fa640 1 -- 172.21.15.60:0/2272103483 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95f8005360 con 0x55ab80582a50 2024-02-26T04:58:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.156+0000 7f9607013740 10 client.15633 fetch_fsmap learned FSMap version 23 2024-02-26T04:58:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.156+0000 7f9607013740 1 -- 172.21.15.60:0/2272103483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ab804e1670 con 0x55ab80582a50 2024-02-26T04:58:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.156+0000 7f95e57fa640 1 -- 172.21.15.60:0/2272103483 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ab801e27e0 con 0x55ab80582a50 2024-02-26T04:58:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.157+0000 7f95e57fa640 1 -- 172.21.15.60:0/2272103483 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f95f800f800 con 0x55ab80582a50 2024-02-26T04:58:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.157+0000 7f9607013740 10 client.15633 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:58:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.157+0000 7f9607013740 10 client.15633 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-02-26T04:58:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.157+0000 7f9607013740 20 client.15633 populate_metadata read hostname 'smithi060' 2024-02-26T04:58:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.157+0000 7f9607013740 1 --2- 172.21.15.60:0/2272103483 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55ab805a14a0 0x55ab805a3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.157+0000 7f9607013740 4 client.15633 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:58:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.157+0000 7f9607013740 1 -- 172.21.15.60:0/2272103483 --> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab8021f720 con 0x55ab805a14a0 2024-02-26T04:58:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.157+0000 7f96017d1640 1 --2- 172.21.15.60:0/2272103483 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55ab805a14a0 0x55ab805a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.158+0000 7f96017d1640 1 --2- 172.21.15.60:0/2272103483 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55ab805a14a0 0x55ab805a3890 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c2093822ce6738fe server_cookie=77cb2db619211bbb in_seq=0 out_seq=0 2024-02-26T04:58:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.158+0000 7f95e57fa640 10 client.15633 ms_handle_connect on v2:172.21.15.191:6838/640686517 2024-02-26T04:58:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.158+0000 7f95e57fa640 1 -- 172.21.15.60:0/2272103483 <== mds.2 v2:172.21.15.191:6838/640686517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ab8021f720 con 0x55ab805a14a0 2024-02-26T04:58:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.158+0000 7f95e57fa640 10 client.15633 handle_command_reply: tid=0 2024-02-26T04:58:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.158+0000 7f9607013740 1 client.15633 shutdown 2024-02-26T04:58:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.158+0000 7f9607013740 1 -- 172.21.15.60:0/2272103483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 msgr2=0x55ab8057bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.158+0000 7f9607013740 1 --2- 172.21.15.60:0/2272103483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab8057bac0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009600 tx=0x7f95f80041e0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.159+0000 7f9607013740 1 -- 172.21.15.60:0/2272103483 shutdown_connections 2024-02-26T04:58:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.159+0000 7f9607013740 1 --2- 172.21.15.60:0/2272103483 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55ab805a14a0 0x55ab805a3890 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.159+0000 7f9607013740 1 --2- 172.21.15.60:0/2272103483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab805885b0 0x55ab80581030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.159+0000 7f9607013740 1 --2- 172.21.15.60:0/2272103483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80583440 0x55ab8057c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.159+0000 7f9607013740 1 --2- 172.21.15.60:0/2272103483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab8057bac0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.159+0000 7f9607013740 1 -- 172.21.15.60:0/2272103483 >> 172.21.15.60:0/2272103483 conn(0x55ab805757b0 msgr2=0x55ab80574b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.159+0000 7f95e57fa640 0 client.15633 ms_handle_reset on v2:172.21.15.191:6838/640686517 2024-02-26T04:58:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.159+0000 7f9607013740 1 -- 172.21.15.60:0/2272103483 shutdown_connections 2024-02-26T04:58:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.159+0000 7f9607013740 1 -- 172.21.15.60:0/2272103483 wait complete. 2024-02-26T04:58:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.159+0000 7f9607013740 20 client.15633 trim_cache size 0 max 16384 2024-02-26T04:58:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.171+0000 7f9607013740 1 Processor -- start 2024-02-26T04:58:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.171+0000 7f9607013740 1 -- start start 2024-02-26T04:58:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.171+0000 7f9607013740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab805b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.171+0000 7f9607013740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80583440 0x55ab805b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.171+0000 7f9607013740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab805885b0 0x55ab8057a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.171+0000 7f9607013740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab80468100 con 0x55ab80582a50 2024-02-26T04:58:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.171+0000 7f9607013740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab80583cd0 con 0x55ab805885b0 2024-02-26T04:58:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.171+0000 7f9607013740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab805814b0 con 0x55ab80583440 2024-02-26T04:58:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.171+0000 7f9601fd2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab805b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.171+0000 7f9601fd2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab805b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59148/0 (socket says 172.21.15.60:59148) 2024-02-26T04:58:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.171+0000 7f9601fd2640 1 -- 172.21.15.60:0/464282429 learned_addr learned my addr 172.21.15.60:0/464282429 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.171+0000 7f96027d3640 1 --2- 172.21.15.60:0/464282429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab805885b0 0x55ab8057a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.171+0000 7f96017d1640 1 --2- 172.21.15.60:0/464282429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80583440 0x55ab805b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.172+0000 7f96027d3640 1 -- 172.21.15.60:0/464282429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80583440 msgr2=0x55ab805b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.172+0000 7f96027d3640 1 --2- 172.21.15.60:0/464282429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80583440 0x55ab805b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.172+0000 7f96027d3640 1 -- 172.21.15.60:0/464282429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 msgr2=0x55ab805b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.172+0000 7f96027d3640 1 --2- 172.21.15.60:0/464282429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab805b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.172+0000 7f96027d3640 1 -- 172.21.15.60:0/464282429 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab8047dbf0 con 0x55ab805885b0 2024-02-26T04:58:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.172+0000 7f9601fd2640 1 --2- 172.21.15.60:0/464282429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab805b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.172+0000 7f96027d3640 1 --2- 172.21.15.60:0/464282429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab805885b0 0x55ab8057a560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f95f402c700 tx=0x7f95f4042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a525fe582e2e7f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.172+0000 7f95e77fe640 1 -- 172.21.15.60:0/464282429 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4003b70 con 0x55ab805885b0 2024-02-26T04:58:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.172+0000 7f95e77fe640 1 -- 172.21.15.60:0/464282429 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95f4051070 con 0x55ab805885b0 2024-02-26T04:58:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.172+0000 7f95e77fe640 1 -- 172.21.15.60:0/464282429 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f404d690 con 0x55ab805885b0 2024-02-26T04:58:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.173+0000 7f9607013740 1 -- 172.21.15.60:0/464282429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab805885b0 msgr2=0x55ab8057a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.173+0000 7f9607013740 1 --2- 172.21.15.60:0/464282429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab805885b0 0x55ab8057a560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f95f402c700 tx=0x7f95f4042000 comp rx=0 tx=0).stop 2024-02-26T04:58:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.173+0000 7f9607013740 1 -- 172.21.15.60:0/464282429 shutdown_connections 2024-02-26T04:58:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.173+0000 7f9607013740 1 --2- 172.21.15.60:0/464282429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab805885b0 0x55ab8057a560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.173+0000 7f9607013740 1 --2- 172.21.15.60:0/464282429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab80583440 0x55ab805b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.173+0000 7f9607013740 1 --2- 172.21.15.60:0/464282429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab805b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.173+0000 7f9607013740 1 -- 172.21.15.60:0/464282429 >> 172.21.15.60:0/464282429 conn(0x55ab805757b0 msgr2=0x55ab80575b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.173+0000 7f9607013740 1 -- 172.21.15.60:0/464282429 shutdown_connections 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.173+0000 7f9607013740 1 -- 172.21.15.60:0/464282429 wait complete. 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.174+0000 7f9607013740 1 -- start start 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.174+0000 7f9607013740 1 Processor -- start 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.174+0000 7f9607013740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab805b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.174+0000 7f9607013740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab80583440 0x55ab805f4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.174+0000 7f9607013740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab805885b0 0x55ab805f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.174+0000 7f9601fd2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab805b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.174+0000 7f9601fd2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab805b1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59154/0 (socket says 172.21.15.60:59154) 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f9601fd2640 1 -- 172.21.15.60:0/3432607276 learned_addr learned my addr 172.21.15.60:0/3432607276 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f9607013740 1 -- 172.21.15.60:0/3432607276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab805f7140 con 0x55ab80582a50 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f9607013740 1 -- 172.21.15.60:0/3432607276 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab805f72b0 con 0x55ab80583440 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f9607013740 1 -- 172.21.15.60:0/3432607276 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab805f7470 con 0x55ab805885b0 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f96017d1640 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab80583440 0x55ab805f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f9607013740 10 client.0 _ll_register_callbacks cb 0x7f95f8009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f96027d3640 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab805885b0 0x55ab805f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f9601fd2640 1 -- 172.21.15.60:0/3432607276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab805885b0 msgr2=0x55ab805f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f9601fd2640 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab805885b0 0x55ab805f6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f9601fd2640 1 -- 172.21.15.60:0/3432607276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab80583440 msgr2=0x55ab805f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f9601fd2640 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab80583440 0x55ab805f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f9601fd2640 1 -- 172.21.15.60:0/3432607276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab804e1670 con 0x55ab80582a50 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f96027d3640 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab805885b0 0x55ab805f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f96017d1640 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab80583440 0x55ab805f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f9601fd2640 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab805b1c90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009600 tx=0x7f95f800f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d340e317c4fa561f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f95e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.175+0000 7f95e57fa640 1 -- 172.21.15.60:0/3432607276 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8003e50 con 0x55ab80582a50 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.176+0000 7f95e57fa640 1 -- 172.21.15.60:0/3432607276 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f95f8005240 con 0x55ab80582a50 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.176+0000 7f9607013740 1 -- 172.21.15.60:0/3432607276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95f80035f0 con 0x55ab80582a50 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.176+0000 7f95e57fa640 1 -- 172.21.15.60:0/3432607276 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95f8003b20 con 0x55ab80582a50 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.176+0000 7f9607013740 10 client.15639 fetch_fsmap learned FSMap version 23 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.176+0000 7f9607013740 1 -- 172.21.15.60:0/3432607276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f95fc108dc0 con 0x55ab80582a50 2024-02-26T04:58:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.176+0000 7f95e57fa640 1 -- 172.21.15.60:0/3432607276 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95f80035f0 con 0x55ab80582a50 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.176+0000 7f95e57fa640 1 -- 172.21.15.60:0/3432607276 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f95f80195c0 con 0x55ab80582a50 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.176+0000 7f9607013740 10 client.15639 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.176+0000 7f9607013740 10 client.15639 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.176+0000 7f9607013740 20 client.15639 populate_metadata read hostname 'smithi060' 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.176+0000 7f9607013740 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55ab805f8b00 0x55ab805faf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.176+0000 7f9607013740 4 client.15639 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.176+0000 7f9607013740 1 -- 172.21.15.60:0/3432607276 --> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ab8026e890 con 0x55ab805f8b00 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f96017d1640 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55ab805f8b00 0x55ab805faf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f95e57fa640 10 client.15639 ms_handle_connect on v2:172.21.15.191:6838/640686517 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f96017d1640 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55ab805f8b00 0x55ab805faf20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e6eea38cdeba2d1 server_cookie=87597fea6c564faa in_seq=0 out_seq=0 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f95e57fa640 1 -- 172.21.15.60:0/3432607276 <== mds.2 v2:172.21.15.191:6838/640686517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ab8026e890 con 0x55ab805f8b00 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f95e57fa640 10 client.15639 handle_command_reply: tid=0 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f9607013740 1 client.15639 shutdown 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f9607013740 1 -- 172.21.15.60:0/3432607276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 msgr2=0x55ab805b1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f9607013740 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab805b1c90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009600 tx=0x7f95f800f770 comp rx=0 tx=0).stop 2024-02-26T04:58:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f9607013740 1 -- 172.21.15.60:0/3432607276 shutdown_connections 2024-02-26T04:58:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f9607013740 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55ab805f8b00 0x55ab805faf20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f9607013740 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab805885b0 0x55ab805f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f9607013740 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab80583440 0x55ab805f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f95e57fa640 0 client.15639 ms_handle_reset on v2:172.21.15.191:6838/640686517 2024-02-26T04:58:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f9607013740 1 --2- 172.21.15.60:0/3432607276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ab80582a50 0x55ab805b1c90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.177+0000 7f9607013740 1 -- 172.21.15.60:0/3432607276 >> 172.21.15.60:0/3432607276 conn(0x55ab805757b0 msgr2=0x55ab80578a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9607013740 1 -- 172.21.15.60:0/3432607276 shutdown_connections 2024-02-26T04:58:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9607013740 1 -- 172.21.15.60:0/3432607276 wait complete. 2024-02-26T04:58:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9607013740 20 client.15639 trim_cache size 0 max 16384 2024-02-26T04:58:30.185 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T04:58:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9604236640 1 -- 172.21.15.60:0/4277531693 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f95e80a20c0 msgr2=0x7f95e80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9604236640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f95e80a20c0 0x7f95e80a4580 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55ab80583ca0 tx=0x7f95f40326b0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9604236640 1 -- 172.21.15.60:0/4277531693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95fc101270 msgr2=0x7f95fc196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9604236640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95fc101270 0x7f95fc196620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009d20 tx=0x7f95ec00c6a0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9604236640 1 -- 172.21.15.60:0/4277531693 shutdown_connections 2024-02-26T04:58:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9604236640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f95e80a20c0 0x7f95e80a4580 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9604236640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95fc101c30 0x7f95fc19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9604236640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95fc101270 0x7f95fc196620 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9604236640 1 --2- 172.21.15.60:0/4277531693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc100080 0x7f95fc1960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9604236640 1 -- 172.21.15.60:0/4277531693 >> 172.21.15.60:0/4277531693 conn(0x7f95fc0fb610 msgr2=0x7f95fc104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9604236640 1 -- 172.21.15.60:0/4277531693 shutdown_connections 2024-02-26T04:58:30.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.178+0000 7f9604236640 1 -- 172.21.15.60:0/4277531693 wait complete. 2024-02-26T04:58:30.187 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:29 smithi156 ceph-mon[35805]: pgmap v388: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:58:30.193 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T04:58:30.193 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T04:58:30.193 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:29 smithi060 ceph-mon[32382]: pgmap v388: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:58:30.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:29 smithi191 ceph-mon[36352]: pgmap v388: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:58:30.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.297+0000 7f0445a36640 1 -- 172.21.15.60:0/2309480887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0440102d60 msgr2=0x7f04401095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.297+0000 7f0445a36640 1 --2- 172.21.15.60:0/2309480887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0440102d60 0x7f04401095f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0438009ca0 tx=0x7f043802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.297+0000 7f0445a36640 1 -- 172.21.15.60:0/2309480887 shutdown_connections 2024-02-26T04:58:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.297+0000 7f0445a36640 1 --2- 172.21.15.60:0/2309480887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0440102d60 0x7f04401095f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.297+0000 7f0445a36640 1 --2- 172.21.15.60:0/2309480887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04401023a0 0x7f0440102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.297+0000 7f0445a36640 1 --2- 172.21.15.60:0/2309480887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04401011b0 0x7f04401015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.298+0000 7f0445a36640 1 -- 172.21.15.60:0/2309480887 >> 172.21.15.60:0/2309480887 conn(0x7f04400fc720 msgr2=0x7f04400feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.298+0000 7f0445a36640 1 -- 172.21.15.60:0/2309480887 shutdown_connections 2024-02-26T04:58:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.298+0000 7f0445a36640 1 -- 172.21.15.60:0/2309480887 wait complete. 2024-02-26T04:58:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.298+0000 7f0445a36640 1 Processor -- start 2024-02-26T04:58:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.298+0000 7f0445a36640 1 -- start start 2024-02-26T04:58:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.298+0000 7f0445a36640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04401011b0 0x7f0440197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.298+0000 7f0445a36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04401023a0 0x7f0440197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f0445a36640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0440102d60 0x7f044019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f0445a36640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044019ed10 con 0x7f04401023a0 2024-02-26T04:58:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f0445a36640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044019ee80 con 0x7f0440102d60 2024-02-26T04:58:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f0445a36640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044019f040 con 0x7f04401011b0 2024-02-26T04:58:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f043e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04401023a0 0x7f0440197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f043e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04401023a0 0x7f0440197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59162/0 (socket says 172.21.15.60:59162) 2024-02-26T04:58:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f043e7fc640 1 -- 172.21.15.60:0/2606391211 learned_addr learned my addr 172.21.15.60:0/2606391211 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f043effd640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04401011b0 0x7f0440197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f043f7fe640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0440102d60 0x7f044019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f043e7fc640 1 -- 172.21.15.60:0/2606391211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04401011b0 msgr2=0x7f0440197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f043e7fc640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04401011b0 0x7f0440197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f043e7fc640 1 -- 172.21.15.60:0/2606391211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0440102d60 msgr2=0x7f044019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f043e7fc640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0440102d60 0x7f044019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f043e7fc640 1 -- 172.21.15.60:0/2606391211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0438009900 con 0x7f04401023a0 2024-02-26T04:58:30.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f043effd640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04401011b0 0x7f0440197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:30.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.299+0000 7f043e7fc640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04401023a0 0x7f0440197760 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f042c00d950 tx=0x7f042c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.300+0000 7f0444a34640 1 -- 172.21.15.60:0/2606391211 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c009c30 con 0x7f04401023a0 2024-02-26T04:58:30.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.300+0000 7f0445a36640 1 -- 172.21.15.60:0/2606391211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044019f320 con 0x7f04401023a0 2024-02-26T04:58:30.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.300+0000 7f0444a34640 1 -- 172.21.15.60:0/2606391211 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f042c009d90 con 0x7f04401023a0 2024-02-26T04:58:30.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.300+0000 7f0444a34640 1 -- 172.21.15.60:0/2606391211 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c00f7e0 con 0x7f04401023a0 2024-02-26T04:58:30.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.300+0000 7f0445a36640 1 -- 172.21.15.60:0/2606391211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044019f880 con 0x7f04401023a0 2024-02-26T04:58:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.303+0000 7f044880e740 1 Processor -- start 2024-02-26T04:58:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.303+0000 7f044880e740 1 -- start start 2024-02-26T04:58:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.303+0000 7f0444a34640 1 -- 172.21.15.60:0/2606391211 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f042c00f940 con 0x7f04401023a0 2024-02-26T04:58:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.303+0000 7f044880e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 0x55efe514f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.303+0000 7f044880e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe514fc30 0x55efe5154730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.303+0000 7f044880e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe5154da0 0x55efe5157230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.303+0000 7f044880e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe510b930 con 0x55efe5154da0 2024-02-26T04:58:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.303+0000 7f044880e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe50bf2e0 con 0x55efe514fc30 2024-02-26T04:58:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.303+0000 7f044880e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe50bc810 con 0x55efe514f240 2024-02-26T04:58:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.303+0000 7f043e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe5154da0 0x55efe5157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.303+0000 7f043f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 0x55efe514f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.303+0000 7f043e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe5154da0 0x55efe5157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59168/0 (socket says 172.21.15.60:59168) 2024-02-26T04:58:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.303+0000 7f043f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 0x55efe514f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.60:50014/0 (socket says 172.21.15.60:50014) 2024-02-26T04:58:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.303+0000 7f043e7fc640 1 -- 172.21.15.60:0/3104172256 learned_addr learned my addr 172.21.15.60:0/3104172256 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.304+0000 7f043f7fe640 1 -- 172.21.15.60:0/3104172256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe514fc30 msgr2=0x55efe5154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.304+0000 7f043f7fe640 1 --2- 172.21.15.60:0/3104172256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe514fc30 0x55efe5154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.304+0000 7f043f7fe640 1 -- 172.21.15.60:0/3104172256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe5154da0 msgr2=0x55efe5157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.304+0000 7f043f7fe640 1 --2- 172.21.15.60:0/3104172256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe5154da0 0x55efe5157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.304+0000 7f043f7fe640 1 -- 172.21.15.60:0/3104172256 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efe507a030 con 0x55efe514f240 2024-02-26T04:58:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.304+0000 7f0444a34640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f04240a2140 0x7f04240a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.304+0000 7f043f7fe640 1 --2- 172.21.15.60:0/3104172256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 0x55efe514f660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0438037040 tx=0x7f04380093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6e443feacc9ed81 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.304+0000 7f0444a34640 1 -- 172.21.15.60:0/2606391211 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f042c0cdc20 con 0x7f04401023a0 2024-02-26T04:58:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.304+0000 7f0435ffb640 1 -- 172.21.15.60:0/3104172256 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438034440 con 0x55efe514f240 2024-02-26T04:58:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.304+0000 7f0435ffb640 1 -- 172.21.15.60:0/3104172256 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04380345a0 con 0x55efe514f240 2024-02-26T04:58:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.304+0000 7f0435ffb640 1 -- 172.21.15.60:0/3104172256 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438003bd0 con 0x55efe514f240 2024-02-26T04:58:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.304+0000 7f043effd640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f04240a2140 0x7f04240a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.305+0000 7f044880e740 1 -- 172.21.15.60:0/3104172256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 msgr2=0x55efe514f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.305+0000 7f043effd640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f04240a2140 0x7f04240a4600 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f043000b620 tx=0x7f043000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.305+0000 7f044880e740 1 --2- 172.21.15.60:0/3104172256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 0x55efe514f660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0438037040 tx=0x7f04380093f0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.305+0000 7f044880e740 1 -- 172.21.15.60:0/3104172256 shutdown_connections 2024-02-26T04:58:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.305+0000 7f044880e740 1 --2- 172.21.15.60:0/3104172256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe5154da0 0x55efe5157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.305+0000 7f044880e740 1 --2- 172.21.15.60:0/3104172256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe514fc30 0x55efe5154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.305+0000 7f044880e740 1 --2- 172.21.15.60:0/3104172256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 0x55efe514f660 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.305+0000 7f044880e740 1 -- 172.21.15.60:0/3104172256 >> 172.21.15.60:0/3104172256 conn(0x55efe514a7b0 msgr2=0x55efe514abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.305+0000 7f044880e740 1 -- 172.21.15.60:0/3104172256 shutdown_connections 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.305+0000 7f044880e740 1 -- 172.21.15.60:0/3104172256 wait complete. 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.306+0000 7f044880e740 1 -- start start 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.306+0000 7f044880e740 1 Processor -- start 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.306+0000 7f044880e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 0x55efe5159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.306+0000 7f044880e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe514fc30 0x55efe5159960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.306+0000 7f044880e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe5154da0 0x55efe51609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.306+0000 7f044880e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe50b7ff0 con 0x55efe514fc30 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.306+0000 7f044880e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe4d8b430 con 0x55efe5154da0 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.306+0000 7f044880e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe4d47e00 con 0x55efe514f240 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.306+0000 7f044880e740 10 client.0 _ll_register_callbacks cb 0x55efe505fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.306+0000 7f043e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe514fc30 0x55efe5159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.306+0000 7f043e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe514fc30 0x55efe5159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59176/0 (socket says 172.21.15.60:59176) 2024-02-26T04:58:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.306+0000 7f043e7fc640 1 -- 172.21.15.60:0/713189251 learned_addr learned my addr 172.21.15.60:0/713189251 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.306+0000 7f043effd640 1 --2- 172.21.15.60:0/713189251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 0x55efe5159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.307+0000 7f043e7fc640 1 -- 172.21.15.60:0/713189251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 msgr2=0x55efe5159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.307+0000 7f043e7fc640 1 --2- 172.21.15.60:0/713189251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 0x55efe5159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.307+0000 7f043e7fc640 1 -- 172.21.15.60:0/713189251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe5154da0 msgr2=0x55efe51609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.307+0000 7f043e7fc640 1 --2- 172.21.15.60:0/713189251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe5154da0 0x55efe51609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.307+0000 7f043e7fc640 1 -- 172.21.15.60:0/713189251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0440109ef0 con 0x55efe514fc30 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.307+0000 7f043e7fc640 1 --2- 172.21.15.60:0/713189251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe514fc30 0x55efe5159960 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f042c010fd0 tx=0x7f042c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c8042725f38d74 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.307+0000 7f0422ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.307+0000 7f0422ffd640 1 -- 172.21.15.60:0/713189251 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c096040 con 0x55efe514fc30 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.307+0000 7f0422ffd640 1 -- 172.21.15.60:0/713189251 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f042c010ad0 con 0x55efe514fc30 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.307+0000 7f044880e740 1 -- 172.21.15.60:0/713189251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55efe4db77e0 con 0x55efe514fc30 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.307+0000 7f0422ffd640 1 -- 172.21.15.60:0/713189251 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f042c09a390 con 0x55efe514fc30 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.307+0000 7f044880e740 10 client.15651 fetch_fsmap learned FSMap version 23 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.307+0000 7f044880e740 1 -- 172.21.15.60:0/713189251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55efe50b6670 con 0x55efe514fc30 2024-02-26T04:58:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.308+0000 7f0422ffd640 1 -- 172.21.15.60:0/713189251 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f042c09a100 con 0x55efe514fc30 2024-02-26T04:58:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.308+0000 7f0422ffd640 1 -- 172.21.15.60:0/713189251 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f042c010460 con 0x55efe514fc30 2024-02-26T04:58:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.308+0000 7f044880e740 10 client.15651 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:58:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.308+0000 7f044880e740 10 client.15651 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-26T04:58:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.308+0000 7f044880e740 20 client.15651 populate_metadata read hostname 'smithi060' 2024-02-26T04:58:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.308+0000 7f044880e740 1 --2- 172.21.15.60:0/713189251 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x55efe5165620 0x55efe5167a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.308+0000 7f044880e740 4 client.15651 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:58:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.308+0000 7f044880e740 1 -- 172.21.15.60:0/713189251 --> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efe4db77e0 con 0x55efe5165620 2024-02-26T04:58:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.308+0000 7f043effd640 1 --2- 172.21.15.60:0/713189251 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x55efe5165620 0x55efe5167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.309+0000 7f043effd640 1 --2- 172.21.15.60:0/713189251 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x55efe5165620 0x55efe5167a50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfb28355c2d1caaf server_cookie=e2ad33f027a88e06 in_seq=0 out_seq=0 2024-02-26T04:58:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.309+0000 7f0422ffd640 10 client.15651 ms_handle_connect on v2:172.21.15.156:6836/707575792 2024-02-26T04:58:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.309+0000 7f0422ffd640 1 -- 172.21.15.60:0/713189251 <== mds.1 v2:172.21.15.156:6836/707575792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55efe4db77e0 con 0x55efe5165620 2024-02-26T04:58:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.309+0000 7f0422ffd640 10 client.15651 handle_command_reply: tid=0 2024-02-26T04:58:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.309+0000 7f044880e740 1 client.15651 shutdown 2024-02-26T04:58:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.310+0000 7f044880e740 1 -- 172.21.15.60:0/713189251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe514fc30 msgr2=0x55efe5159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.310+0000 7f044880e740 1 --2- 172.21.15.60:0/713189251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe514fc30 0x55efe5159960 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f042c010fd0 tx=0x7f042c00bf60 comp rx=0 tx=0).stop 2024-02-26T04:58:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.310+0000 7f044880e740 1 -- 172.21.15.60:0/713189251 shutdown_connections 2024-02-26T04:58:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.310+0000 7f044880e740 1 --2- 172.21.15.60:0/713189251 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x55efe5165620 0x55efe5167a50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.310+0000 7f044880e740 1 --2- 172.21.15.60:0/713189251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe5154da0 0x55efe51609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.310+0000 7f044880e740 1 --2- 172.21.15.60:0/713189251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe514fc30 0x55efe5159960 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.310+0000 7f044880e740 1 --2- 172.21.15.60:0/713189251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 0x55efe5159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.310+0000 7f044880e740 1 -- 172.21.15.60:0/713189251 >> 172.21.15.60:0/713189251 conn(0x55efe514a7b0 msgr2=0x55efe5149b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.310+0000 7f0422ffd640 0 client.15651 ms_handle_reset on v2:172.21.15.156:6836/707575792 2024-02-26T04:58:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.310+0000 7f044880e740 1 -- 172.21.15.60:0/713189251 shutdown_connections 2024-02-26T04:58:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.310+0000 7f044880e740 1 -- 172.21.15.60:0/713189251 wait complete. 2024-02-26T04:58:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.310+0000 7f044880e740 20 client.15651 trim_cache size 0 max 16384 2024-02-26T04:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.322+0000 7f044880e740 1 Processor -- start 2024-02-26T04:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.322+0000 7f044880e740 1 -- start start 2024-02-26T04:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.322+0000 7f044880e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe514f240 0x55efe514ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.322+0000 7f044880e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe514fc30 0x55efe5160960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.322+0000 7f044880e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe5154da0 0x55efe5159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.322+0000 7f044880e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe503d100 con 0x55efe514fc30 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.322+0000 7f044880e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe51578f0 con 0x55efe514f240 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.322+0000 7f044880e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe4db77e0 con 0x55efe5154da0 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.322+0000 7f043e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe514fc30 0x55efe5160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.322+0000 7f043e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe514fc30 0x55efe5160960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59182/0 (socket says 172.21.15.60:59182) 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.322+0000 7f043e7fc640 1 -- 172.21.15.60:0/4267998187 learned_addr learned my addr 172.21.15.60:0/4267998187 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.322+0000 7f043f7fe640 1 --2- 172.21.15.60:0/4267998187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe5154da0 0x55efe5159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.322+0000 7f043effd640 1 --2- 172.21.15.60:0/4267998187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe514f240 0x55efe514ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.323+0000 7f043e7fc640 1 -- 172.21.15.60:0/4267998187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe5154da0 msgr2=0x55efe5159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.323+0000 7f043e7fc640 1 --2- 172.21.15.60:0/4267998187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe5154da0 0x55efe5159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.323+0000 7f043e7fc640 1 -- 172.21.15.60:0/4267998187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe514f240 msgr2=0x55efe514ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.323+0000 7f043e7fc640 1 --2- 172.21.15.60:0/4267998187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe514f240 0x55efe514ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.323+0000 7f043e7fc640 1 -- 172.21.15.60:0/4267998187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efe5052bf0 con 0x55efe514fc30 2024-02-26T04:58:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.323+0000 7f043effd640 1 --2- 172.21.15.60:0/4267998187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe514f240 0x55efe514ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:58:30.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.323+0000 7f043e7fc640 1 --2- 172.21.15.60:0/4267998187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe514fc30 0x55efe5160960 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f042c096f90 tx=0x7f042c098ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79a3878fd7e0e17 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.323+0000 7f04357fa640 1 -- 172.21.15.60:0/4267998187 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c014070 con 0x55efe514fc30 2024-02-26T04:58:30.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.323+0000 7f04357fa640 1 -- 172.21.15.60:0/4267998187 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f042c010940 con 0x55efe514fc30 2024-02-26T04:58:30.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.323+0000 7f04357fa640 1 -- 172.21.15.60:0/4267998187 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c0d0050 con 0x55efe514fc30 2024-02-26T04:58:30.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.324+0000 7f044880e740 1 -- 172.21.15.60:0/4267998187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe514fc30 msgr2=0x55efe5160960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.324+0000 7f044880e740 1 --2- 172.21.15.60:0/4267998187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe514fc30 0x55efe5160960 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f042c096f90 tx=0x7f042c098ad0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.324+0000 7f044880e740 1 -- 172.21.15.60:0/4267998187 shutdown_connections 2024-02-26T04:58:30.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.324+0000 7f044880e740 1 --2- 172.21.15.60:0/4267998187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe5154da0 0x55efe5159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.324+0000 7f044880e740 1 --2- 172.21.15.60:0/4267998187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe514fc30 0x55efe5160960 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.324+0000 7f044880e740 1 --2- 172.21.15.60:0/4267998187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe514f240 0x55efe514ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.324+0000 7f044880e740 1 -- 172.21.15.60:0/4267998187 >> 172.21.15.60:0/4267998187 conn(0x55efe514a7b0 msgr2=0x55efe51459e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.324+0000 7f044880e740 1 -- 172.21.15.60:0/4267998187 shutdown_connections 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.324+0000 7f044880e740 1 -- 172.21.15.60:0/4267998187 wait complete. 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.324+0000 7f044880e740 1 -- start start 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.324+0000 7f044880e740 1 Processor -- start 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f044880e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 0x55efe51606c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f044880e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe5154da0 0x55efe515edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f044880e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe515f2f0 0x55efe515f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f044880e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe5160d20 con 0x55efe515f2f0 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f044880e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe51bbb30 con 0x55efe5154da0 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f044880e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe51c00b0 con 0x55efe514f240 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f044880e740 10 client.0 _ll_register_callbacks cb 0x7f0438009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:58:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f043f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe515f2f0 0x55efe515f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f043f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe515f2f0 0x55efe515f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59184/0 (socket says 172.21.15.60:59184) 2024-02-26T04:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f043e7fc640 1 --2- 172.21.15.60:0/271157441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe5154da0 0x55efe515edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f043effd640 1 --2- 172.21.15.60:0/271157441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 0x55efe51606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f043f7fe640 1 -- 172.21.15.60:0/271157441 learned_addr learned my addr 172.21.15.60:0/271157441 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f043e7fc640 1 -- 172.21.15.60:0/271157441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 msgr2=0x55efe51606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f043e7fc640 1 --2- 172.21.15.60:0/271157441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 0x55efe51606c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f043e7fc640 1 -- 172.21.15.60:0/271157441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe515f2f0 msgr2=0x55efe515f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f043e7fc640 1 --2- 172.21.15.60:0/271157441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe515f2f0 0x55efe515f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f043e7fc640 1 -- 172.21.15.60:0/271157441 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efe50b6670 con 0x55efe5154da0 2024-02-26T04:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f043e7fc640 1 --2- 172.21.15.60:0/271157441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe5154da0 0x55efe515edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55efe514f8a0 tx=0x7f042c00fe20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3c5da9916113700 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.325+0000 7f0422ffd640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-26T04:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.326+0000 7f0422ffd640 1 -- 172.21.15.60:0/271157441 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c0d0070 con 0x55efe5154da0 2024-02-26T04:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.326+0000 7f044880e740 1 -- 172.21.15.60:0/271157441 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04380345a0 con 0x55efe5154da0 2024-02-26T04:58:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.326+0000 7f0422ffd640 1 -- 172.21.15.60:0/271157441 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f042c010ad0 con 0x55efe5154da0 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.326+0000 7f0422ffd640 1 -- 172.21.15.60:0/271157441 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f042c014070 con 0x55efe5154da0 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.326+0000 7f044880e740 10 client.25268 fetch_fsmap learned FSMap version 23 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.326+0000 7f044880e740 1 -- 172.21.15.60:0/271157441 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55efe5078890 con 0x55efe5154da0 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.326+0000 7f0422ffd640 1 -- 172.21.15.60:0/271157441 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04380345a0 con 0x55efe5154da0 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.326+0000 7f0422ffd640 1 -- 172.21.15.60:0/271157441 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f042c099a10 con 0x55efe5154da0 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.326+0000 7f044880e740 10 client.25268 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.326+0000 7f044880e740 10 client.25268 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.326+0000 7f044880e740 20 client.25268 populate_metadata read hostname 'smithi060' 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.326+0000 7f044880e740 1 --2- 172.21.15.60:0/271157441 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x55efe51c1440 0x55efe51c3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.326+0000 7f044880e740 4 client.25268 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.326+0000 7f044880e740 1 -- 172.21.15.60:0/271157441 --> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55efe4df4720 con 0x55efe51c1440 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.327+0000 7f043effd640 1 --2- 172.21.15.60:0/271157441 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x55efe51c1440 0x55efe51c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.327+0000 7f043effd640 1 --2- 172.21.15.60:0/271157441 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x55efe51c1440 0x55efe51c3830 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd66930afce895c5 server_cookie=237fa9cb16ca9d64 in_seq=0 out_seq=0 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.327+0000 7f0422ffd640 10 client.25268 ms_handle_connect on v2:172.21.15.156:6836/707575792 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.327+0000 7f0422ffd640 1 -- 172.21.15.60:0/271157441 <== mds.1 v2:172.21.15.156:6836/707575792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55efe4df4720 con 0x55efe51c1440 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.327+0000 7f0422ffd640 10 client.25268 handle_command_reply: tid=0 2024-02-26T04:58:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.327+0000 7f044880e740 1 client.25268 shutdown 2024-02-26T04:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.327+0000 7f044880e740 1 -- 172.21.15.60:0/271157441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe5154da0 msgr2=0x55efe515edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.327+0000 7f044880e740 1 --2- 172.21.15.60:0/271157441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe5154da0 0x55efe515edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55efe514f8a0 tx=0x7f042c00fe20 comp rx=0 tx=0).stop 2024-02-26T04:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.327+0000 7f044880e740 1 -- 172.21.15.60:0/271157441 shutdown_connections 2024-02-26T04:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.327+0000 7f044880e740 1 --2- 172.21.15.60:0/271157441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55efe515f2f0 0x55efe515f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.327+0000 7f044880e740 1 --2- 172.21.15.60:0/271157441 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x55efe51c1440 0x55efe51c3830 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f044880e740 1 --2- 172.21.15.60:0/271157441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55efe5154da0 0x55efe515edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f044880e740 1 --2- 172.21.15.60:0/271157441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55efe514f240 0x55efe51606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f0422ffd640 0 client.25268 ms_handle_reset on v2:172.21.15.156:6836/707575792 2024-02-26T04:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f044880e740 1 -- 172.21.15.60:0/271157441 >> 172.21.15.60:0/271157441 conn(0x55efe514a7b0 msgr2=0x55efe514ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f044880e740 1 -- 172.21.15.60:0/271157441 shutdown_connections 2024-02-26T04:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f044880e740 1 -- 172.21.15.60:0/271157441 wait complete. 2024-02-26T04:58:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f044880e740 20 client.25268 trim_cache size 0 max 16384 2024-02-26T04:58:30.335 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T04:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f0445a36640 1 -- 172.21.15.60:0/2606391211 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f04240a2140 msgr2=0x7f04240a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f0445a36640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f04240a2140 0x7f04240a4600 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f043000b620 tx=0x7f043000b5b0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f0445a36640 1 -- 172.21.15.60:0/2606391211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04401023a0 msgr2=0x7f0440197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f0445a36640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04401023a0 0x7f0440197760 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f042c00d950 tx=0x7f042c00de10 comp rx=0 tx=0).stop 2024-02-26T04:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f0445a36640 1 -- 172.21.15.60:0/2606391211 shutdown_connections 2024-02-26T04:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f0445a36640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f04240a2140 0x7f04240a4600 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f0445a36640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0440102d60 0x7f044019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f0445a36640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04401023a0 0x7f0440197760 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f0445a36640 1 --2- 172.21.15.60:0/2606391211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04401011b0 0x7f0440197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f0445a36640 1 -- 172.21.15.60:0/2606391211 >> 172.21.15.60:0/2606391211 conn(0x7f04400fc720 msgr2=0x7f04401056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.328+0000 7f0445a36640 1 -- 172.21.15.60:0/2606391211 shutdown_connections 2024-02-26T04:58:30.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.329+0000 7f0445a36640 1 -- 172.21.15.60:0/2606391211 wait complete. 2024-02-26T04:58:30.344 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T04:58:30.344 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T04:58:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.448+0000 7f511de0f640 1 -- 172.21.15.60:0/2939305020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5118101c30 msgr2=0x7f51181084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.448+0000 7f511de0f640 1 --2- 172.21.15.60:0/2939305020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5118101c30 0x7f51181084c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5110009ca0 tx=0x7f511002c0f0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.448+0000 7f511de0f640 1 -- 172.21.15.60:0/2939305020 shutdown_connections 2024-02-26T04:58:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.448+0000 7f511de0f640 1 --2- 172.21.15.60:0/2939305020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5118101c30 0x7f51181084c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.448+0000 7f511de0f640 1 --2- 172.21.15.60:0/2939305020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5118101270 0x7f51181016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.448+0000 7f511de0f640 1 --2- 172.21.15.60:0/2939305020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5118100080 0x7f51181004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.448+0000 7f511de0f640 1 -- 172.21.15.60:0/2939305020 >> 172.21.15.60:0/2939305020 conn(0x7f51180fb610 msgr2=0x7f51180fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.448+0000 7f511de0f640 1 -- 172.21.15.60:0/2939305020 shutdown_connections 2024-02-26T04:58:30.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.448+0000 7f511de0f640 1 -- 172.21.15.60:0/2939305020 wait complete. 2024-02-26T04:58:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.449+0000 7f511de0f640 1 Processor -- start 2024-02-26T04:58:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.449+0000 7f511de0f640 1 -- start start 2024-02-26T04:58:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.449+0000 7f511de0f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5118100080 0x7f511810a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.449+0000 7f511de0f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5118101270 0x7f5118108f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.449+0000 7f511de0f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5118101c30 0x7f51181094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.449+0000 7f511de0f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118109c20 con 0x7f5118101270 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.449+0000 7f511de0f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118109d90 con 0x7f5118100080 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.449+0000 7f511de0f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51181a1250 con 0x7f5118101c30 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.449+0000 7f5116ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5118101270 0x7f5118108f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.450+0000 7f5116ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5118101270 0x7f5118108f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59210/0 (socket says 172.21.15.60:59210) 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.450+0000 7f5116ffd640 1 -- 172.21.15.60:0/762184983 learned_addr learned my addr 172.21.15.60:0/762184983 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.450+0000 7f5117fff640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5118101c30 0x7f51181094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.450+0000 7f5116ffd640 1 -- 172.21.15.60:0/762184983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5118101c30 msgr2=0x7f51181094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.450+0000 7f5116ffd640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5118101c30 0x7f51181094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.450+0000 7f5116ffd640 1 -- 172.21.15.60:0/762184983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5118100080 msgr2=0x7f511810a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.450+0000 7f51177fe640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5118100080 0x7f511810a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.450+0000 7f5116ffd640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5118100080 0x7f511810a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.450+0000 7f5116ffd640 1 -- 172.21.15.60:0/762184983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5110009900 con 0x7f5118101270 2024-02-26T04:58:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.450+0000 7f51177fe640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5118100080 0x7f511810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:58:30.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.450+0000 7f5116ffd640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5118101270 0x7f5118108f80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f510c00ea20 tx=0x7f510c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.450+0000 7f5114ff9640 1 -- 172.21.15.60:0/762184983 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c009720 con 0x7f5118101270 2024-02-26T04:58:30.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.450+0000 7f511de0f640 1 -- 172.21.15.60:0/762184983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51181a1500 con 0x7f5118101270 2024-02-26T04:58:30.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.450+0000 7f511de0f640 1 -- 172.21.15.60:0/762184983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51181a1a10 con 0x7f5118101270 2024-02-26T04:58:30.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.451+0000 7f5114ff9640 1 -- 172.21.15.60:0/762184983 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f510c00cd80 con 0x7f5118101270 2024-02-26T04:58:30.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.451+0000 7f5114ff9640 1 -- 172.21.15.60:0/762184983 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c010490 con 0x7f5118101270 2024-02-26T04:58:30.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.454+0000 7f5114ff9640 1 -- 172.21.15.60:0/762184983 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f510c010730 con 0x7f5118101270 2024-02-26T04:58:30.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.454+0000 7f5120be7740 1 Processor -- start 2024-02-26T04:58:30.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.454+0000 7f5120be7740 1 -- start start 2024-02-26T04:58:30.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.454+0000 7f5120be7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7bd4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.454+0000 7f5120be7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7bd9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.454+0000 7f5120be7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 0x558cd7bdc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.454+0000 7f5120be7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd7b90930 con 0x558cd7bd4240 2024-02-26T04:58:30.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.454+0000 7f5120be7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd7b442e0 con 0x558cd7bd9da0 2024-02-26T04:58:30.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.454+0000 7f5120be7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd7b41810 con 0x558cd7bd4c30 2024-02-26T04:58:30.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.454+0000 7f5117fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7bd4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.454+0000 7f5117fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7bd4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59226/0 (socket says 172.21.15.60:59226) 2024-02-26T04:58:30.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.454+0000 7f5117fff640 1 -- 172.21.15.60:0/3094910456 learned_addr learned my addr 172.21.15.60:0/3094910456 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.454+0000 7f51177fe640 1 --2- 172.21.15.60:0/3094910456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7bd9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.454+0000 7f5116ffd640 1 --2- 172.21.15.60:0/3094910456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 0x558cd7bdc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.455+0000 7f5114ff9640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f51000a21c0 0x7f51000a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.455+0000 7f51177fe640 1 -- 172.21.15.60:0/3094910456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 msgr2=0x558cd7bdc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.455+0000 7f51177fe640 1 --2- 172.21.15.60:0/3094910456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 0x558cd7bdc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.455+0000 7f51177fe640 1 -- 172.21.15.60:0/3094910456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 msgr2=0x558cd7bd4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.455+0000 7f51177fe640 1 --2- 172.21.15.60:0/3094910456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7bd4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.455+0000 7f51177fe640 1 -- 172.21.15.60:0/3094910456 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cd7aff030 con 0x558cd7bd4c30 2024-02-26T04:58:30.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.455+0000 7f5117fff640 1 --2- 172.21.15.60:0/3094910456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7bd4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:30.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.455+0000 7f5116ffd640 1 --2- 172.21.15.60:0/3094910456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 0x558cd7bdc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:30.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.455+0000 7f5114ff9640 1 -- 172.21.15.60:0/762184983 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f510c014070 con 0x7f5118101270 2024-02-26T04:58:30.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.455+0000 7f5117fff640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f51000a21c0 0x7f51000a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.455+0000 7f51177fe640 1 --2- 172.21.15.60:0/3094910456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7bd9730 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5108009c30 tx=0x7f5108010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35d732d4af3971c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.455+0000 7f50fdffb640 1 -- 172.21.15.60:0/3094910456 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108005020 con 0x558cd7bd4c30 2024-02-26T04:58:30.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.455+0000 7f50fdffb640 1 -- 172.21.15.60:0/3094910456 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5108005660 con 0x558cd7bd4c30 2024-02-26T04:58:30.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.455+0000 7f50fdffb640 1 -- 172.21.15.60:0/3094910456 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51080126b0 con 0x558cd7bd4c30 2024-02-26T04:58:30.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.456+0000 7f5120be7740 1 -- 172.21.15.60:0/3094910456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 msgr2=0x558cd7bd9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.456+0000 7f5120be7740 1 --2- 172.21.15.60:0/3094910456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7bd9730 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5108009c30 tx=0x7f5108010740 comp rx=0 tx=0).stop 2024-02-26T04:58:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.456+0000 7f5117fff640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f51000a21c0 0x7f51000a4680 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5110006010 tx=0x7f51100095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.456+0000 7f5120be7740 1 -- 172.21.15.60:0/3094910456 shutdown_connections 2024-02-26T04:58:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.456+0000 7f5120be7740 1 --2- 172.21.15.60:0/3094910456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 0x558cd7bdc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.456+0000 7f5120be7740 1 --2- 172.21.15.60:0/3094910456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7bd9730 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.456+0000 7f5120be7740 1 --2- 172.21.15.60:0/3094910456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7bd4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.456+0000 7f5120be7740 1 -- 172.21.15.60:0/3094910456 >> 172.21.15.60:0/3094910456 conn(0x558cd7bcf7b0 msgr2=0x558cd7bcfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.456+0000 7f5120be7740 1 -- 172.21.15.60:0/3094910456 shutdown_connections 2024-02-26T04:58:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.456+0000 7f5120be7740 1 -- 172.21.15.60:0/3094910456 wait complete. 2024-02-26T04:58:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.456+0000 7f5120be7740 1 -- start start 2024-02-26T04:58:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5120be7740 1 Processor -- start 2024-02-26T04:58:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5120be7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4240 0x558cd7bde480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5120be7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd4c30 0x558cd7bde9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5120be7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd9da0 0x558cd7be5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5120be7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd7b3cff0 con 0x558cd7bd9da0 2024-02-26T04:58:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5120be7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd7810430 con 0x558cd7bd4c30 2024-02-26T04:58:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5120be7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd77cce00 con 0x558cd7bd4240 2024-02-26T04:58:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5120be7740 10 client.0 _ll_register_callbacks cb 0x558cd7ae4c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:58:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5117fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd9da0 0x558cd7be5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5117fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd9da0 0x558cd7be5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59238/0 (socket says 172.21.15.60:59238) 2024-02-26T04:58:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5117fff640 1 -- 172.21.15.60:0/8249008 learned_addr learned my addr 172.21.15.60:0/8249008 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f51177fe640 1 --2- 172.21.15.60:0/8249008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4240 0x558cd7bde480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5116ffd640 1 --2- 172.21.15.60:0/8249008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd4c30 0x558cd7bde9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5117fff640 1 -- 172.21.15.60:0/8249008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4240 msgr2=0x558cd7bde480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5117fff640 1 --2- 172.21.15.60:0/8249008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4240 0x558cd7bde480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5117fff640 1 -- 172.21.15.60:0/8249008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd4c30 msgr2=0x558cd7bde9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5117fff640 1 --2- 172.21.15.60:0/8249008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd4c30 0x558cd7bde9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.457+0000 7f5117fff640 1 -- 172.21.15.60:0/8249008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5108009890 con 0x558cd7bd9da0 2024-02-26T04:58:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.458+0000 7f51177fe640 1 --2- 172.21.15.60:0/8249008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4240 0x558cd7bde480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.458+0000 7f5117fff640 1 --2- 172.21.15.60:0/8249008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd9da0 0x558cd7be5a00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f51100325e0 tx=0x7f511002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ba3e371facdc55 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.458+0000 7f50f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:58:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.458+0000 7f50f6ffd640 1 -- 172.21.15.60:0/8249008 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110034c90 con 0x558cd7bd9da0 2024-02-26T04:58:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.458+0000 7f5120be7740 1 -- 172.21.15.60:0/8249008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cd783c7e0 con 0x558cd7bd9da0 2024-02-26T04:58:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.458+0000 7f50f6ffd640 1 -- 172.21.15.60:0/8249008 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f5110034df0 con 0x558cd7bd9da0 2024-02-26T04:58:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.458+0000 7f50f6ffd640 1 -- 172.21.15.60:0/8249008 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5110003970 con 0x558cd7bd9da0 2024-02-26T04:58:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.458+0000 7f5120be7740 10 client.15669 fetch_fsmap learned FSMap version 23 2024-02-26T04:58:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.458+0000 7f5120be7740 1 -- 172.21.15.60:0/8249008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558cd7b3b670 con 0x558cd7bd9da0 2024-02-26T04:58:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.458+0000 7f50f6ffd640 1 -- 172.21.15.60:0/8249008 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558cd783c7e0 con 0x558cd7bd9da0 2024-02-26T04:58:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.459+0000 7f50f6ffd640 1 -- 172.21.15.60:0/8249008 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5110049050 con 0x558cd7bd9da0 2024-02-26T04:58:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.459+0000 7f5120be7740 10 client.15669 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:58:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.459+0000 7f5120be7740 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:58:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.459+0000 7f5120be7740 20 client.15669 populate_metadata read hostname 'smithi060' 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.459+0000 7f5120be7740 1 --2- 172.21.15.60:0/8249008 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x558cd7bea6d0 0x558cd7becaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.459+0000 7f5120be7740 4 client.15669 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.459+0000 7f5120be7740 1 -- 172.21.15.60:0/8249008 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cd7879720 con 0x558cd7bea6d0 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.459+0000 7f51177fe640 1 --2- 172.21.15.60:0/8249008 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x558cd7bea6d0 0x558cd7becaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.460+0000 7f50f6ffd640 10 client.15669 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.460+0000 7f51177fe640 1 --2- 172.21.15.60:0/8249008 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x558cd7bea6d0 0x558cd7becaf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebb38b9502575bfa server_cookie=11461f8f606be15a in_seq=0 out_seq=0 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.460+0000 7f50f6ffd640 1 -- 172.21.15.60:0/8249008 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558cd7879720 con 0x558cd7bea6d0 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.460+0000 7f50f6ffd640 10 client.15669 handle_command_reply: tid=0 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.460+0000 7f5120be7740 1 client.15669 shutdown 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.461+0000 7f5120be7740 1 -- 172.21.15.60:0/8249008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd9da0 msgr2=0x558cd7be5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.461+0000 7f5120be7740 1 --2- 172.21.15.60:0/8249008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd9da0 0x558cd7be5a00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f51100325e0 tx=0x7f511002c0f0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.461+0000 7f5120be7740 1 -- 172.21.15.60:0/8249008 shutdown_connections 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.461+0000 7f5120be7740 1 --2- 172.21.15.60:0/8249008 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x558cd7bea6d0 0x558cd7becaf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.461+0000 7f50f6ffd640 0 client.15669 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.461+0000 7f5120be7740 1 --2- 172.21.15.60:0/8249008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd9da0 0x558cd7be5a00 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.461+0000 7f5120be7740 1 --2- 172.21.15.60:0/8249008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd4c30 0x558cd7bde9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.461+0000 7f5120be7740 1 --2- 172.21.15.60:0/8249008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4240 0x558cd7bde480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.461+0000 7f5120be7740 1 -- 172.21.15.60:0/8249008 >> 172.21.15.60:0/8249008 conn(0x558cd7bcf7b0 msgr2=0x558cd7bceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.461+0000 7f5120be7740 1 -- 172.21.15.60:0/8249008 shutdown_connections 2024-02-26T04:58:30.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.461+0000 7f5120be7740 1 -- 172.21.15.60:0/8249008 wait complete. 2024-02-26T04:58:30.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.461+0000 7f5120be7740 20 client.15669 trim_cache size 0 max 16384 2024-02-26T04:58:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.484+0000 7f5120be7740 1 Processor -- start 2024-02-26T04:58:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.484+0000 7f5120be7740 1 -- start start 2024-02-26T04:58:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.484+0000 7f5120be7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7bd60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.484+0000 7f5120be7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7bd65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.484+0000 7f5120be7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 0x558cd7be59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.484+0000 7f5120be7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd7ac2100 con 0x558cd7bd4240 2024-02-26T04:58:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.484+0000 7f51177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7bd60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.484+0000 7f5120be7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd7bd9b80 con 0x558cd7bd9da0 2024-02-26T04:58:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.484+0000 7f5120be7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd7bdf320 con 0x558cd7bd4c30 2024-02-26T04:58:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.485+0000 7f51177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7bd60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59246/0 (socket says 172.21.15.60:59246) 2024-02-26T04:58:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.485+0000 7f51177fe640 1 -- 172.21.15.60:0/3322546997 learned_addr learned my addr 172.21.15.60:0/3322546997 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.485+0000 7f5116ffd640 1 --2- 172.21.15.60:0/3322546997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7bd65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.485+0000 7f5117fff640 1 --2- 172.21.15.60:0/3322546997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 0x558cd7be59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.485+0000 7f51177fe640 1 -- 172.21.15.60:0/3322546997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 msgr2=0x558cd7bd65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.485+0000 7f51177fe640 1 --2- 172.21.15.60:0/3322546997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7bd65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.485+0000 7f51177fe640 1 -- 172.21.15.60:0/3322546997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 msgr2=0x558cd7be59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.485+0000 7f51177fe640 1 --2- 172.21.15.60:0/3322546997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 0x558cd7be59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.485+0000 7f51177fe640 1 -- 172.21.15.60:0/3322546997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cd7ad7bf0 con 0x558cd7bd4240 2024-02-26T04:58:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.485+0000 7f5116ffd640 1 --2- 172.21.15.60:0/3322546997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7bd65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:58:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.485+0000 7f51177fe640 1 --2- 172.21.15.60:0/3322546997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7bd60b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5108009c30 tx=0x7f5108010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28618785948b29cf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.485+0000 7f50fd7fa640 1 -- 172.21.15.60:0/3322546997 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51080047c0 con 0x558cd7bd4240 2024-02-26T04:58:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.485+0000 7f50fd7fa640 1 -- 172.21.15.60:0/3322546997 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f510801f040 con 0x558cd7bd4240 2024-02-26T04:58:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.485+0000 7f50fd7fa640 1 -- 172.21.15.60:0/3322546997 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108004d60 con 0x558cd7bd4240 2024-02-26T04:58:30.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.486+0000 7f5120be7740 1 -- 172.21.15.60:0/3322546997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 msgr2=0x558cd7bd60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.486+0000 7f5120be7740 1 --2- 172.21.15.60:0/3322546997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7bd60b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5108009c30 tx=0x7f5108010eb0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.486+0000 7f5120be7740 1 -- 172.21.15.60:0/3322546997 shutdown_connections 2024-02-26T04:58:30.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.486+0000 7f5120be7740 1 --2- 172.21.15.60:0/3322546997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 0x558cd7be59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.486+0000 7f5120be7740 1 --2- 172.21.15.60:0/3322546997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7bd65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.486+0000 7f5120be7740 1 --2- 172.21.15.60:0/3322546997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7bd60b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.486+0000 7f5120be7740 1 -- 172.21.15.60:0/3322546997 >> 172.21.15.60:0/3322546997 conn(0x558cd7bcf7b0 msgr2=0x558cd7bcfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.487+0000 7f5120be7740 1 -- 172.21.15.60:0/3322546997 shutdown_connections 2024-02-26T04:58:30.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.487+0000 7f5120be7740 1 -- 172.21.15.60:0/3322546997 wait complete. 2024-02-26T04:58:30.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.487+0000 7f5120be7740 1 -- start start 2024-02-26T04:58:30.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.487+0000 7f5120be7740 1 Processor -- start 2024-02-26T04:58:30.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.487+0000 7f5120be7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7be56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f51177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7be56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f51177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7be56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59252/0 (socket says 172.21.15.60:59252) 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f51177fe640 1 -- 172.21.15.60:0/234207215 learned_addr learned my addr 172.21.15.60:0/234207215 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f5120be7740 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7be3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f5120be7740 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 0x558cd7be4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f5120be7740 1 -- 172.21.15.60:0/234207215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd7be4740 con 0x558cd7bd4240 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f5116ffd640 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7be3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f5120be7740 1 -- 172.21.15.60:0/234207215 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd7be48b0 con 0x558cd7bd9da0 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f5120be7740 1 -- 172.21.15.60:0/234207215 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd7be4a70 con 0x558cd7bd4c30 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f5117fff640 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 0x558cd7be4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f5120be7740 10 client.0 _ll_register_callbacks cb 0x7f5108009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f5116ffd640 1 -- 172.21.15.60:0/234207215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 msgr2=0x558cd7be4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f5116ffd640 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 0x558cd7be4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f5116ffd640 1 -- 172.21.15.60:0/234207215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 msgr2=0x558cd7be56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f5116ffd640 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7be56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f5116ffd640 1 -- 172.21.15.60:0/234207215 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cd7aff030 con 0x558cd7bd4c30 2024-02-26T04:58:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.488+0000 7f5117fff640 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 0x558cd7be4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:30.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.489+0000 7f5116ffd640 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7be3cc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f510c00e9f0 tx=0x7f510c097090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cb8ee9c2f9118a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:30.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.489+0000 7f50f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-26T04:58:30.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.489+0000 7f5120be7740 1 -- 172.21.15.60:0/234207215 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5108005660 con 0x558cd7bd4c30 2024-02-26T04:58:30.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.489+0000 7f50f6ffd640 1 -- 172.21.15.60:0/234207215 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c09b3e0 con 0x558cd7bd4c30 2024-02-26T04:58:30.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.489+0000 7f50f6ffd640 1 -- 172.21.15.60:0/234207215 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f510c09ace0 con 0x558cd7bd4c30 2024-02-26T04:58:30.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.489+0000 7f50f6ffd640 1 -- 172.21.15.60:0/234207215 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f510c09ee30 con 0x558cd7bd4c30 2024-02-26T04:58:30.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.489+0000 7f5120be7740 10 client.25237 fetch_fsmap learned FSMap version 23 2024-02-26T04:58:30.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.489+0000 7f5120be7740 1 -- 172.21.15.60:0/234207215 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({fsmap=23}) v3 -- 0x558cd7afd890 con 0x558cd7bd4c30 2024-02-26T04:58:30.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.490+0000 7f50f6ffd640 1 -- 172.21.15.60:0/234207215 <== mon.1 v2:172.21.15.191:3300/0 4 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f510c01f060 con 0x558cd7bd4c30 2024-02-26T04:58:30.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.490+0000 7f5120be7740 10 client.25237 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:58:30.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.490+0000 7f5120be7740 10 client.25237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:58:30.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.490+0000 7f5120be7740 20 client.25237 populate_metadata read hostname 'smithi060' 2024-02-26T04:58:30.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.490+0000 7f5120be7740 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x558cd7bea6d0 0x558cd7c44180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:30.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.490+0000 7f5120be7740 4 client.25237 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T04:58:30.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.490+0000 7f5120be7740 1 -- 172.21.15.60:0/234207215 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f5108005660 con 0x558cd7bea6d0 2024-02-26T04:58:30.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.490+0000 7f51177fe640 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x558cd7bea6d0 0x558cd7c44180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:30.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.490+0000 7f51177fe640 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x558cd7bea6d0 0x558cd7c44180 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8dc47bd88667645 server_cookie=1ea2e24fb4e4bc34 in_seq=0 out_seq=0 2024-02-26T04:58:30.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.490+0000 7f50f6ffd640 10 client.25237 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:30.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f50f6ffd640 1 -- 172.21.15.60:0/234207215 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f5108005660 con 0x558cd7bea6d0 2024-02-26T04:58:30.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f50f6ffd640 10 client.25237 handle_command_reply: tid=0 2024-02-26T04:58:30.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f5120be7740 1 client.25237 shutdown 2024-02-26T04:58:30.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f5120be7740 1 -- 172.21.15.60:0/234207215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 msgr2=0x558cd7be3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f5120be7740 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7be3cc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f510c00e9f0 tx=0x7f510c097090 comp rx=0 tx=0).stop 2024-02-26T04:58:30.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f5120be7740 1 -- 172.21.15.60:0/234207215 shutdown_connections 2024-02-26T04:58:30.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f5120be7740 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x558cd7bea6d0 0x558cd7c44180 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f50f6ffd640 0 client.25237 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:30.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f5120be7740 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cd7bd9da0 0x558cd7be4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f5120be7740 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558cd7bd4c30 0x558cd7be3cc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f5120be7740 1 --2- 172.21.15.60:0/234207215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cd7bd4240 0x558cd7be56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f5120be7740 1 -- 172.21.15.60:0/234207215 >> 172.21.15.60:0/234207215 conn(0x558cd7bcf7b0 msgr2=0x558cd7bd60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f5120be7740 1 -- 172.21.15.60:0/234207215 shutdown_connections 2024-02-26T04:58:30.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f5120be7740 1 -- 172.21.15.60:0/234207215 wait complete. 2024-02-26T04:58:30.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.491+0000 7f5120be7740 20 client.25237 trim_cache size 0 max 16384 2024-02-26T04:58:30.499 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T04:58:30.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.492+0000 7f511de0f640 1 -- 172.21.15.60:0/762184983 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f51000a21c0 msgr2=0x7f51000a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.492+0000 7f511de0f640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f51000a21c0 0x7f51000a4680 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5110006010 tx=0x7f51100095c0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.492+0000 7f511de0f640 1 -- 172.21.15.60:0/762184983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5118101270 msgr2=0x7f5118108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:30.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.492+0000 7f511de0f640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5118101270 0x7f5118108f80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f510c00ea20 tx=0x7f510c00ee20 comp rx=0 tx=0).stop 2024-02-26T04:58:30.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.492+0000 7f511de0f640 1 -- 172.21.15.60:0/762184983 shutdown_connections 2024-02-26T04:58:30.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.492+0000 7f511de0f640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f51000a21c0 0x7f51000a4680 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.492+0000 7f511de0f640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5118101c30 0x7f51181094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.492+0000 7f511de0f640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5118101270 0x7f5118108f80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.492+0000 7f511de0f640 1 --2- 172.21.15.60:0/762184983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5118100080 0x7f511810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:30.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.492+0000 7f511de0f640 1 -- 172.21.15.60:0/762184983 >> 172.21.15.60:0/762184983 conn(0x7f51180fb610 msgr2=0x7f5118104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:30.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.492+0000 7f511de0f640 1 -- 172.21.15.60:0/762184983 shutdown_connections 2024-02-26T04:58:30.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:30.492+0000 7f511de0f640 1 -- 172.21.15.60:0/762184983 wait complete. 2024-02-26T04:58:31.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:30 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4196293050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:58:31.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:30 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4196293050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:58:31.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:30 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4196293050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:58:31.507 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T04:58:31.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:58:31.507 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T04:58:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.614+0000 7ff9d5036640 1 -- 172.21.15.60:0/3168079658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d0100d50 msgr2=0x7ff9d01011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.614+0000 7ff9d5036640 1 --2- 172.21.15.60:0/3168079658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d0100d50 0x7ff9d01011d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009ca0 tx=0x7ff9c802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.614+0000 7ff9d5036640 1 -- 172.21.15.60:0/3168079658 shutdown_connections 2024-02-26T04:58:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.614+0000 7ff9d5036640 1 --2- 172.21.15.60:0/3168079658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d0100d50 0x7ff9d01011d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.614+0000 7ff9d5036640 1 --2- 172.21.15.60:0/3168079658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9d01003b0 0x7ff9d0100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.614+0000 7ff9d5036640 1 --2- 172.21.15.60:0/3168079658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9d00ffa50 0x7ff9d00ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.614+0000 7ff9d5036640 1 -- 172.21.15.60:0/3168079658 >> 172.21.15.60:0/3168079658 conn(0x7ff9d00fb610 msgr2=0x7ff9d00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.614+0000 7ff9d5036640 1 -- 172.21.15.60:0/3168079658 shutdown_connections 2024-02-26T04:58:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.614+0000 7ff9d5036640 1 -- 172.21.15.60:0/3168079658 wait complete. 2024-02-26T04:58:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.615+0000 7ff9d5036640 1 Processor -- start 2024-02-26T04:58:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.615+0000 7ff9d5036640 1 -- start start 2024-02-26T04:58:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.615+0000 7ff9d5036640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9d00ffa50 0x7ff9d01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.615+0000 7ff9d5036640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d01003b0 0x7ff9d0199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.615+0000 7ff9d5036640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9d0100d50 0x7ff9d01a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.615+0000 7ff9ce59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d01003b0 0x7ff9d0199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9ce59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d01003b0 0x7ff9d0199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59262/0 (socket says 172.21.15.60:59262) 2024-02-26T04:58:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9ce59c640 1 -- 172.21.15.60:0/3360767268 learned_addr learned my addr 172.21.15.60:0/3360767268 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9d5036640 1 -- 172.21.15.60:0/3360767268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d01a0ec0 con 0x7ff9d01003b0 2024-02-26T04:58:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9d5036640 1 -- 172.21.15.60:0/3360767268 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d01a1030 con 0x7ff9d0100d50 2024-02-26T04:58:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9d5036640 1 -- 172.21.15.60:0/3360767268 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d01a11f0 con 0x7ff9d00ffa50 2024-02-26T04:58:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9ced9d640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9d00ffa50 0x7ff9d01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9cf59e640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9d0100d50 0x7ff9d01a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9ced9d640 1 -- 172.21.15.60:0/3360767268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9d0100d50 msgr2=0x7ff9d01a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9ced9d640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9d0100d50 0x7ff9d01a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9ced9d640 1 -- 172.21.15.60:0/3360767268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d01003b0 msgr2=0x7ff9d0199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9ced9d640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d01003b0 0x7ff9d0199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9ced9d640 1 -- 172.21.15.60:0/3360767268 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c8009900 con 0x7ff9d00ffa50 2024-02-26T04:58:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9ce59c640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d01003b0 0x7ff9d0199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9ced9d640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9d00ffa50 0x7ff9d01993d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000e9f0 tx=0x7ff9c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.616+0000 7ff9b7fff640 1 -- 172.21.15.60:0/3360767268 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c000ccc0 con 0x7ff9d00ffa50 2024-02-26T04:58:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.617+0000 7ff9d5036640 1 -- 172.21.15.60:0/3360767268 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d01a14d0 con 0x7ff9d00ffa50 2024-02-26T04:58:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.617+0000 7ff9b7fff640 1 -- 172.21.15.60:0/3360767268 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9c000ce20 con 0x7ff9d00ffa50 2024-02-26T04:58:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.617+0000 7ff9b7fff640 1 -- 172.21.15.60:0/3360767268 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0018920 con 0x7ff9d00ffa50 2024-02-26T04:58:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.617+0000 7ff9d5036640 1 -- 172.21.15.60:0/3360767268 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d01a1a30 con 0x7ff9d00ffa50 2024-02-26T04:58:31.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.619+0000 7ff9d7e0e740 1 Processor -- start 2024-02-26T04:58:31.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9d7e0e740 1 -- start start 2024-02-26T04:58:31.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9b7fff640 1 -- 172.21.15.60:0/3360767268 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff9c0018a80 con 0x7ff9d00ffa50 2024-02-26T04:58:31.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9d7e0e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396d9490 0x564b396d98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9d7e0e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396da660 0x564b396daae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9d7e0e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396db020 0x564b396e18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9d7e0e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b39648f60 con 0x564b396d9490 2024-02-26T04:58:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9d7e0e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b39646490 con 0x564b396da660 2024-02-26T04:58:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9d7e0e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b39641c70 con 0x564b396db020 2024-02-26T04:58:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9cf59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396d9490 0x564b396d98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9cf59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396d9490 0x564b396d98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59264/0 (socket says 172.21.15.60:59264) 2024-02-26T04:58:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9cf59e640 1 -- 172.21.15.60:0/160914270 learned_addr learned my addr 172.21.15.60:0/160914270 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9ced9d640 1 --2- 172.21.15.60:0/160914270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396da660 0x564b396daae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9ce59c640 1 --2- 172.21.15.60:0/160914270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396db020 0x564b396e18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9cf59e640 1 -- 172.21.15.60:0/160914270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396db020 msgr2=0x564b396e18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9cf59e640 1 --2- 172.21.15.60:0/160914270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396db020 0x564b396e18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9cf59e640 1 -- 172.21.15.60:0/160914270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396da660 msgr2=0x564b396daae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9cf59e640 1 --2- 172.21.15.60:0/160914270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396da660 0x564b396daae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9cf59e640 1 -- 172.21.15.60:0/160914270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b39604080 con 0x564b396d9490 2024-02-26T04:58:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.620+0000 7ff9b7fff640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff9b80a20c0 0x7ff9b80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.621+0000 7ff9ced9d640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff9b80a20c0 0x7ff9b80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.621+0000 7ff9b7fff640 1 -- 172.21.15.60:0/3360767268 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff9c0014070 con 0x7ff9d00ffa50 2024-02-26T04:58:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.621+0000 7ff9cf59e640 1 --2- 172.21.15.60:0/160914270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396d9490 0x564b396d98b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009ca0 tx=0x7ff9c80093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e3e416ef9b74ac server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.621+0000 7ff9b57fa640 1 -- 172.21.15.60:0/160914270 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8034440 con 0x564b396d9490 2024-02-26T04:58:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.621+0000 7ff9b57fa640 1 -- 172.21.15.60:0/160914270 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9c80345a0 con 0x564b396d9490 2024-02-26T04:58:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.621+0000 7ff9b57fa640 1 -- 172.21.15.60:0/160914270 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8003bb0 con 0x564b396d9490 2024-02-26T04:58:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.622+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/160914270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396d9490 msgr2=0x564b396d98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.622+0000 7ff9ced9d640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff9b80a20c0 0x7ff9b80a4580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0097150 tx=0x7ff9c00d1c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.622+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/160914270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396d9490 0x564b396d98b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009ca0 tx=0x7ff9c80093e0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.622+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/160914270 shutdown_connections 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.622+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/160914270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396db020 0x564b396e18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.622+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/160914270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396da660 0x564b396daae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.622+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/160914270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396d9490 0x564b396d98b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.622+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/160914270 >> 172.21.15.60:0/160914270 conn(0x564b396d4670 msgr2=0x564b396d4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.622+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/160914270 shutdown_connections 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.622+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/160914270 wait complete. 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.622+0000 7ff9d7e0e740 1 -- start start 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.622+0000 7ff9d7e0e740 1 Processor -- start 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9d7e0e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 0x564b396e3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9d7e0e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396da660 0x564b396e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9d7e0e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396db020 0x564b396eac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9d7e0e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b395e6f10 con 0x564b396db020 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9d7e0e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b393154b0 con 0x564b396d9490 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9d7e0e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b395c7640 con 0x564b396da660 2024-02-26T04:58:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9d7e0e740 10 client.0 _ll_register_callbacks cb 0x564b39634f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9cf59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396db020 0x564b396eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9ce59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396da660 0x564b396e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9cf59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396db020 0x564b396eac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59270/0 (socket says 172.21.15.60:59270) 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9cf59e640 1 -- 172.21.15.60:0/538275502 learned_addr learned my addr 172.21.15.60:0/538275502 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9ce59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396da660 0x564b396e3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.60:50130/0 (socket says 172.21.15.60:50130) 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9cf59e640 1 --2- 172.21.15.60:0/538275502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 0x564b396e3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9cf59e640 1 -- 172.21.15.60:0/538275502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396da660 msgr2=0x564b396e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9cf59e640 1 --2- 172.21.15.60:0/538275502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396da660 0x564b396e3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9cf59e640 1 -- 172.21.15.60:0/538275502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 msgr2=0x564b396e3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9cf59e640 1 --2- 172.21.15.60:0/538275502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 0x564b396e3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9cf59e640 1 -- 172.21.15.60:0/538275502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d010c0a0 con 0x564b396db020 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9ce59c640 1 --2- 172.21.15.60:0/538275502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396da660 0x564b396e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9cf59e640 1 --2- 172.21.15.60:0/538275502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396db020 0x564b396eac00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80325e0 tx=0x7ff9c8005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b9abecb6a755fd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.623+0000 7ff9b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:58:31.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.624+0000 7ff9b2ff5640 1 -- 172.21.15.60:0/538275502 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8034640 con 0x564b396db020 2024-02-26T04:58:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.624+0000 7ff9b2ff5640 1 -- 172.21.15.60:0/538275502 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff9c80327b0 con 0x564b396db020 2024-02-26T04:58:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.624+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/538275502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b3961bf10 con 0x564b396db020 2024-02-26T04:58:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.624+0000 7ff9b2ff5640 1 -- 172.21.15.60:0/538275502 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c80347a0 con 0x564b396db020 2024-02-26T04:58:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.624+0000 7ff9d7e0e740 10 client.15687 fetch_fsmap learned FSMap version 23 2024-02-26T04:58:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.624+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/538275502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564b396402f0 con 0x564b396db020 2024-02-26T04:58:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.624+0000 7ff9b2ff5640 1 -- 172.21.15.60:0/538275502 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564b3961bf10 con 0x564b396db020 2024-02-26T04:58:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.624+0000 7ff9b2ff5640 1 -- 172.21.15.60:0/538275502 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff9c804a450 con 0x564b396db020 2024-02-26T04:58:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.624+0000 7ff9d7e0e740 10 client.15687 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:58:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.624+0000 7ff9d7e0e740 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:58:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.624+0000 7ff9d7e0e740 20 client.15687 populate_metadata read hostname 'smithi060' 2024-02-26T04:58:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.624+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/538275502 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564b396ef570 0x564b396f1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.624+0000 7ff9d7e0e740 4 client.15687 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:58:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.624+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/538275502 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b396382e0 con 0x564b396ef570 2024-02-26T04:58:31.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.625+0000 7ff9ce59c640 1 --2- 172.21.15.60:0/538275502 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564b396ef570 0x564b396f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.625+0000 7ff9ce59c640 1 --2- 172.21.15.60:0/538275502 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564b396ef570 0x564b396f1990 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ccb56355fe4ab27 server_cookie=2fb1214d6d3fc22 in_seq=0 out_seq=0 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.625+0000 7ff9b2ff5640 10 client.15687 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9b2ff5640 1 -- 172.21.15.60:0/538275502 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564b396382e0 con 0x564b396ef570 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9b2ff5640 10 client.15687 handle_command_reply: tid=0 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9d7e0e740 1 client.15687 shutdown 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/538275502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396db020 msgr2=0x564b396eac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/538275502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396db020 0x564b396eac00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80325e0 tx=0x7ff9c8005c30 comp rx=0 tx=0).stop 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/538275502 shutdown_connections 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/538275502 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564b396ef570 0x564b396f1990 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/538275502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396db020 0x564b396eac00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/538275502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396da660 0x564b396e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9b2ff5640 0 client.15687 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/538275502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 0x564b396e3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/538275502 >> 172.21.15.60:0/538275502 conn(0x564b396d4670 msgr2=0x564b396d38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/538275502 shutdown_connections 2024-02-26T04:58:31.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/538275502 wait complete. 2024-02-26T04:58:31.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.626+0000 7ff9d7e0e740 20 client.15687 trim_cache size 0 max 16384 2024-02-26T04:58:31.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.641+0000 7ff9d7e0e740 1 Processor -- start 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.641+0000 7ff9d7e0e740 1 -- start start 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.641+0000 7ff9d7e0e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 0x564b396e3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.641+0000 7ff9d7e0e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396da660 0x564b396eabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.641+0000 7ff9d7e0e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396db020 0x564b396e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.641+0000 7ff9d7e0e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b396e34c0 con 0x564b396da660 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.641+0000 7ff9d7e0e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b396382e0 con 0x564b396d9490 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.641+0000 7ff9d7e0e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3961bf10 con 0x564b396db020 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.642+0000 7ff9ce59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396da660 0x564b396eabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.642+0000 7ff9ce59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396da660 0x564b396eabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59272/0 (socket says 172.21.15.60:59272) 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.642+0000 7ff9ce59c640 1 -- 172.21.15.60:0/1207637497 learned_addr learned my addr 172.21.15.60:0/1207637497 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.642+0000 7ff9cf59e640 1 --2- 172.21.15.60:0/1207637497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 0x564b396e3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.642+0000 7ff9cf59e640 1 --2- 172.21.15.60:0/1207637497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396db020 0x564b396e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.642+0000 7ff9ce59c640 1 -- 172.21.15.60:0/1207637497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396db020 msgr2=0x564b396e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.642+0000 7ff9ce59c640 1 --2- 172.21.15.60:0/1207637497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396db020 0x564b396e3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.642+0000 7ff9ce59c640 1 -- 172.21.15.60:0/1207637497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 msgr2=0x564b396e3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.642+0000 7ff9ce59c640 1 --2- 172.21.15.60:0/1207637497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 0x564b396e3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.642+0000 7ff9ce59c640 1 -- 172.21.15.60:0/1207637497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b396ddde0 con 0x564b396da660 2024-02-26T04:58:31.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.642+0000 7ff9ce59c640 1 --2- 172.21.15.60:0/1207637497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396da660 0x564b396eabc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009640 tx=0x7ff9c4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d75bc62c39843a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:31.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.642+0000 7ff9b4ff9640 1 -- 172.21.15.60:0/1207637497 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4017040 con 0x564b396da660 2024-02-26T04:58:31.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.642+0000 7ff9b4ff9640 1 -- 172.21.15.60:0/1207637497 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9c4018040 con 0x564b396da660 2024-02-26T04:58:31.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.642+0000 7ff9b4ff9640 1 -- 172.21.15.60:0/1207637497 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4004dd0 con 0x564b396da660 2024-02-26T04:58:31.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.643+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/1207637497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396da660 msgr2=0x564b396eabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.643+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/1207637497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396da660 0x564b396eabc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009640 tx=0x7ff9c4005f90 comp rx=0 tx=0).stop 2024-02-26T04:58:31.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.643+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/1207637497 shutdown_connections 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.643+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/1207637497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396db020 0x564b396e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.643+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/1207637497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396da660 0x564b396eabc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.643+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/1207637497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 0x564b396e3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.643+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/1207637497 >> 172.21.15.60:0/1207637497 conn(0x564b396d4670 msgr2=0x564b396de270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.643+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/1207637497 shutdown_connections 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.643+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/1207637497 wait complete. 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9d7e0e740 1 -- start start 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9d7e0e740 1 Processor -- start 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9d7e0e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 0x564b396e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9d7e0e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396da660 0x564b396e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9d7e0e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396db020 0x564b39745a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9cf59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396db020 0x564b39745a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9d7e0e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b39745fa0 con 0x564b396db020 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9cf59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396db020 0x564b39745a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59282/0 (socket says 172.21.15.60:59282) 2024-02-26T04:58:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9d7e0e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b397460e0 con 0x564b396d9490 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9cf59e640 1 -- 172.21.15.60:0/1607761974 learned_addr learned my addr 172.21.15.60:0/1607761974 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/1607761974 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b39746220 con 0x564b396da660 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9d7e0e740 10 client.0 _ll_register_callbacks cb 0x7ff9c0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9ce59c640 1 --2- 172.21.15.60:0/1607761974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396da660 0x564b396e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9cf59e640 1 --2- 172.21.15.60:0/1607761974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 0x564b396e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9cf59e640 1 -- 172.21.15.60:0/1607761974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396da660 msgr2=0x564b396e80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9cf59e640 1 --2- 172.21.15.60:0/1607761974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396da660 0x564b396e80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9cf59e640 1 -- 172.21.15.60:0/1607761974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 msgr2=0x564b396e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.644+0000 7ff9cf59e640 1 --2- 172.21.15.60:0/1607761974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 0x564b396e7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9cf59e640 1 -- 172.21.15.60:0/1607761974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b395dcbd0 con 0x564b396db020 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9cf59e640 1 --2- 172.21.15.60:0/1607761974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396db020 0x564b39745a60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009bf0 tx=0x7ff9c8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e5b170e835ed0e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9b2ff5640 1 -- 172.21.15.60:0/1607761974 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c804a9d0 con 0x564b396db020 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/1607761974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b396678d0 con 0x564b396db020 2024-02-26T04:58:31.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9b2ff5640 1 -- 172.21.15.60:0/1607761974 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff9c804ab30 con 0x564b396db020 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9b2ff5640 1 -- 172.21.15.60:0/1607761974 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c8034440 con 0x564b396db020 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9d7e0e740 10 client.15693 fetch_fsmap learned FSMap version 23 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/1607761974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564b396028e0 con 0x564b396db020 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9b2ff5640 1 -- 172.21.15.60:0/1607761974 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564b396678d0 con 0x564b396db020 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9b2ff5640 1 -- 172.21.15.60:0/1607761974 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff9c803f420 con 0x564b396db020 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9d7e0e740 10 client.15693 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9d7e0e740 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9d7e0e740 20 client.15693 populate_metadata read hostname 'smithi060' 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.645+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/1607761974 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564b397478e0 0x564b39749d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.646+0000 7ff9d7e0e740 4 client.15693 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.646+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/1607761974 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564b39341860 con 0x564b397478e0 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.646+0000 7ff9ce59c640 1 --2- 172.21.15.60:0/1607761974 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564b397478e0 0x564b39749d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.646+0000 7ff9ce59c640 1 --2- 172.21.15.60:0/1607761974 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564b397478e0 0x564b39749d00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4d512ea4e483df7 server_cookie=863c04528391cc95 in_seq=0 out_seq=0 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.646+0000 7ff9b2ff5640 10 client.15693 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:31.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.647+0000 7ff9b2ff5640 1 -- 172.21.15.60:0/1607761974 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564b39341860 con 0x564b397478e0 2024-02-26T04:58:31.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.647+0000 7ff9b2ff5640 10 client.15693 handle_command_reply: tid=0 2024-02-26T04:58:31.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.647+0000 7ff9d7e0e740 1 client.15693 shutdown 2024-02-26T04:58:31.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/1607761974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396db020 msgr2=0x564b39745a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/1607761974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396db020 0x564b39745a60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009bf0 tx=0x7ff9c8003c80 comp rx=0 tx=0).stop 2024-02-26T04:58:31.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/1607761974 shutdown_connections 2024-02-26T04:58:31.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/1607761974 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564b397478e0 0x564b39749d00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/1607761974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564b396db020 0x564b39745a60 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/1607761974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564b396da660 0x564b396e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d7e0e740 1 --2- 172.21.15.60:0/1607761974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564b396d9490 0x564b396e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9b2ff5640 0 client.15693 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:31.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/1607761974 >> 172.21.15.60:0/1607761974 conn(0x564b396d4670 msgr2=0x564b396e3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:31.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/1607761974 shutdown_connections 2024-02-26T04:58:31.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d7e0e740 1 -- 172.21.15.60:0/1607761974 wait complete. 2024-02-26T04:58:31.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d7e0e740 20 client.15693 trim_cache size 0 max 16384 2024-02-26T04:58:31.656 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:58:31.656 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-02-26T04:58:31.656 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "d379eb85-f60d-43d7-a39f-305d2280a2a0", 2024-02-26T04:58:31.656 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-02-26T04:58:31.656 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:58:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d5036640 1 -- 172.21.15.60:0/3360767268 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff9b80a20c0 msgr2=0x7ff9b80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d5036640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff9b80a20c0 0x7ff9b80a4580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0097150 tx=0x7ff9c00d1c30 comp rx=0 tx=0).stop 2024-02-26T04:58:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d5036640 1 -- 172.21.15.60:0/3360767268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9d00ffa50 msgr2=0x7ff9d01993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.648+0000 7ff9d5036640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9d00ffa50 0x7ff9d01993d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000e9f0 tx=0x7ff9c000eeb0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.649+0000 7ff9d5036640 1 -- 172.21.15.60:0/3360767268 shutdown_connections 2024-02-26T04:58:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.649+0000 7ff9d5036640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff9b80a20c0 0x7ff9b80a4580 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.649+0000 7ff9d5036640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9d0100d50 0x7ff9d01a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.649+0000 7ff9d5036640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d01003b0 0x7ff9d0199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.649+0000 7ff9d5036640 1 --2- 172.21.15.60:0/3360767268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9d00ffa50 0x7ff9d01993d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.649+0000 7ff9d5036640 1 -- 172.21.15.60:0/3360767268 >> 172.21.15.60:0/3360767268 conn(0x7ff9d00fb610 msgr2=0x7ff9d00fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.649+0000 7ff9d5036640 1 -- 172.21.15.60:0/3360767268 shutdown_connections 2024-02-26T04:58:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.649+0000 7ff9d5036640 1 -- 172.21.15.60:0/3360767268 wait complete. 2024-02-26T04:58:31.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T04:58:31.663 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T04:58:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.766+0000 7f7c26eb0640 1 -- 172.21.15.60:0/671229139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c20101c10 msgr2=0x7f7c20104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.766+0000 7f7c26eb0640 1 --2- 172.21.15.60:0/671229139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c20101c10 0x7f7c20104000 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009900 tx=0x7f7c1402bc30 comp rx=0 tx=0).stop 2024-02-26T04:58:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.766+0000 7f7c26eb0640 1 -- 172.21.15.60:0/671229139 shutdown_connections 2024-02-26T04:58:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.766+0000 7f7c26eb0640 1 --2- 172.21.15.60:0/671229139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c20106e70 0x7f7c201092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.766+0000 7f7c26eb0640 1 --2- 172.21.15.60:0/671229139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c20104540 0x7f7c20106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.766+0000 7f7c26eb0640 1 --2- 172.21.15.60:0/671229139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c20101c10 0x7f7c20104000 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.766+0000 7f7c26eb0640 1 -- 172.21.15.60:0/671229139 >> 172.21.15.60:0/671229139 conn(0x7f7c200fb5e0 msgr2=0x7f7c200fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.766+0000 7f7c26eb0640 1 -- 172.21.15.60:0/671229139 shutdown_connections 2024-02-26T04:58:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.766+0000 7f7c26eb0640 1 -- 172.21.15.60:0/671229139 wait complete. 2024-02-26T04:58:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.767+0000 7f7c26eb0640 1 Processor -- start 2024-02-26T04:58:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.767+0000 7f7c26eb0640 1 -- start start 2024-02-26T04:58:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.767+0000 7f7c26eb0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c20101c10 0x7f7c20101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.767+0000 7f7c26eb0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c20104540 0x7f7c200ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.767+0000 7f7c26eb0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c20106e70 0x7f7c20100100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.767+0000 7f7c26eb0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c20100860 con 0x7f7c20106e70 2024-02-26T04:58:31.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.767+0000 7f7c26eb0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c201009d0 con 0x7f7c20104540 2024-02-26T04:58:31.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.767+0000 7f7c26eb0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c201a12a0 con 0x7f7c20101c10 2024-02-26T04:58:31.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c2544d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c20106e70 0x7f7c20100100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c2544d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c20106e70 0x7f7c20100100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59296/0 (socket says 172.21.15.60:59296) 2024-02-26T04:58:31.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c2544d640 1 -- 172.21.15.60:0/707765726 learned_addr learned my addr 172.21.15.60:0/707765726 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:31.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c24c4c640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c20101c10 0x7f7c20101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c1ffff640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c20104540 0x7f7c200ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c2544d640 1 -- 172.21.15.60:0/707765726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c20101c10 msgr2=0x7f7c20101580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c2544d640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c20101c10 0x7f7c20101580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c2544d640 1 -- 172.21.15.60:0/707765726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c20104540 msgr2=0x7f7c200ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c2544d640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c20104540 0x7f7c200ffbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c2544d640 1 -- 172.21.15.60:0/707765726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c14009560 con 0x7f7c20106e70 2024-02-26T04:58:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c1ffff640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c20104540 0x7f7c200ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:58:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c24c4c640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c20101c10 0x7f7c20101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:58:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c2544d640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c20106e70 0x7f7c20100100 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800eb40 tx=0x7f7c1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c1dffb640 1 -- 172.21.15.60:0/707765726 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c18004b20 con 0x7f7c20106e70 2024-02-26T04:58:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c26eb0640 1 -- 172.21.15.60:0/707765726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c201a1550 con 0x7f7c20106e70 2024-02-26T04:58:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c1dffb640 1 -- 172.21.15.60:0/707765726 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c18004c80 con 0x7f7c20106e70 2024-02-26T04:58:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.768+0000 7f7c1dffb640 1 -- 172.21.15.60:0/707765726 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c18010670 con 0x7f7c20106e70 2024-02-26T04:58:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.769+0000 7f7c26eb0640 1 -- 172.21.15.60:0/707765726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c201a1a60 con 0x7f7c20106e70 2024-02-26T04:58:31.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.771+0000 7f7c29c84740 1 Processor -- start 2024-02-26T04:58:31.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.772+0000 7f7c1dffb640 1 -- 172.21.15.60:0/707765726 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f7c18005020 con 0x7f7c20106e70 2024-02-26T04:58:31.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.772+0000 7f7c1dffb640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7c100a20c0 0x7f7c100a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.772+0000 7f7c29c84740 1 -- start start 2024-02-26T04:58:31.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.772+0000 7f7c1dffb640 1 -- 172.21.15.60:0/707765726 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f7c18014070 con 0x7f7c20106e70 2024-02-26T04:58:31.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.772+0000 7f7c1ffff640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7c100a20c0 0x7f7c100a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c29c84740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2623ab0 0x5615c2623ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c29c84740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c26244a0 0x5615c2628fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c29c84740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c2629610 0x5615c262baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c29c84740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615c258b250 con 0x5615c2623ab0 2024-02-26T04:58:31.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c29c84740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615c2588780 con 0x5615c26244a0 2024-02-26T04:58:31.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c29c84740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615c2583f60 con 0x5615c2629610 2024-02-26T04:58:31.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c24c4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2623ab0 0x5615c2623ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c24c4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2623ab0 0x5615c2623ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59312/0 (socket says 172.21.15.60:59312) 2024-02-26T04:58:31.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c24c4c640 1 -- 172.21.15.60:0/3908486833 learned_addr learned my addr 172.21.15.60:0/3908486833 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:31.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c2544d640 1 --2- 172.21.15.60:0/3908486833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c2629610 0x5615c262baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c1ffff640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7c100a20c0 0x7f7c100a4580 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7c201012f0 tx=0x7f7c0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:31.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c2544d640 1 -- 172.21.15.60:0/3908486833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c26244a0 msgr2=0x5615c2628fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T04:58:31.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c2544d640 1 --2- 172.21.15.60:0/3908486833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c26244a0 0x5615c2628fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c2544d640 1 -- 172.21.15.60:0/3908486833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2623ab0 msgr2=0x5615c2623ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c2544d640 1 --2- 172.21.15.60:0/3908486833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2623ab0 0x5615c2623ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c2544d640 1 -- 172.21.15.60:0/3908486833 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615c2542540 con 0x5615c2629610 2024-02-26T04:58:31.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.773+0000 7f7c24c4c640 1 --2- 172.21.15.60:0/3908486833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2623ab0 0x5615c2623ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:31.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.774+0000 7f7c2544d640 1 --2- 172.21.15.60:0/3908486833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c2629610 0x5615c262baa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7c180cbec0 tx=0x7f7c1800cfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=423da1962f81c1e7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:31.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.774+0000 7f7c0affd640 1 -- 172.21.15.60:0/3908486833 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c180a3840 con 0x5615c2629610 2024-02-26T04:58:31.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.774+0000 7f7c0affd640 1 -- 172.21.15.60:0/3908486833 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c180993a0 con 0x5615c2629610 2024-02-26T04:58:31.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.774+0000 7f7c0affd640 1 -- 172.21.15.60:0/3908486833 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c180a3610 con 0x5615c2629610 2024-02-26T04:58:31.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.775+0000 7f7c29c84740 1 -- 172.21.15.60:0/3908486833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c2629610 msgr2=0x5615c262baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.775+0000 7f7c29c84740 1 --2- 172.21.15.60:0/3908486833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c2629610 0x5615c262baa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7c180cbec0 tx=0x7f7c1800cfa0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.775+0000 7f7c29c84740 1 -- 172.21.15.60:0/3908486833 shutdown_connections 2024-02-26T04:58:31.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.775+0000 7f7c29c84740 1 --2- 172.21.15.60:0/3908486833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c2629610 0x5615c262baa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.775+0000 7f7c29c84740 1 --2- 172.21.15.60:0/3908486833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c26244a0 0x5615c2628fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.775+0000 7f7c29c84740 1 --2- 172.21.15.60:0/3908486833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2623ab0 0x5615c2623ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.775+0000 7f7c29c84740 1 -- 172.21.15.60:0/3908486833 >> 172.21.15.60:0/3908486833 conn(0x5615c261efd0 msgr2=0x5615c261f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:31.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.775+0000 7f7c29c84740 1 -- 172.21.15.60:0/3908486833 shutdown_connections 2024-02-26T04:58:31.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.775+0000 7f7c29c84740 1 -- 172.21.15.60:0/3908486833 wait complete. 2024-02-26T04:58:31.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.775+0000 7f7c29c84740 1 -- start start 2024-02-26T04:58:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.775+0000 7f7c29c84740 1 Processor -- start 2024-02-26T04:58:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c29c84740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2623ab0 0x5615c262db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c29c84740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c26244a0 0x5615c262e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c29c84740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c2629610 0x5615c26350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c29c84740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615c2257430 con 0x5615c26244a0 2024-02-26T04:58:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c29c84740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615c2213e00 con 0x5615c2623ab0 2024-02-26T04:58:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c29c84740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615c2508fa0 con 0x5615c2629610 2024-02-26T04:58:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c29c84740 10 client.0 _ll_register_callbacks cb 0x5615c255efa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:58:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c24c4c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2623ab0 0x5615c262db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c1ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c26244a0 0x5615c262e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c1ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c26244a0 0x5615c262e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59326/0 (socket says 172.21.15.60:59326) 2024-02-26T04:58:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c24c4c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2623ab0 0x5615c262db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.60:35750/0 (socket says 172.21.15.60:35750) 2024-02-26T04:58:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c1ffff640 1 -- 172.21.15.60:0/1501450674 learned_addr learned my addr 172.21.15.60:0/1501450674 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c2544d640 1 --2- 172.21.15.60:0/1501450674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c2629610 0x5615c26350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c1ffff640 1 -- 172.21.15.60:0/1501450674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c2629610 msgr2=0x5615c26350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c1ffff640 1 --2- 172.21.15.60:0/1501450674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c2629610 0x5615c26350a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c1ffff640 1 -- 172.21.15.60:0/1501450674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2623ab0 msgr2=0x5615c262db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c1ffff640 1 --2- 172.21.15.60:0/1501450674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2623ab0 0x5615c262db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.776+0000 7f7c1ffff640 1 -- 172.21.15.60:0/1501450674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c201a1a60 con 0x5615c26244a0 2024-02-26T04:58:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.777+0000 7f7c1ffff640 1 --2- 172.21.15.60:0/1501450674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c26244a0 0x5615c262e060 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c012920 tx=0x7f7c0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abcb7a34c861cd7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.777+0000 7f7bfffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:58:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.777+0000 7f7bfffff640 1 -- 172.21.15.60:0/1501450674 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c012ea0 con 0x5615c26244a0 2024-02-26T04:58:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.777+0000 7f7c29c84740 1 -- 172.21.15.60:0/1501450674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615c22837e0 con 0x5615c26244a0 2024-02-26T04:58:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.777+0000 7f7bfffff640 1 -- 172.21.15.60:0/1501450674 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f7c0c018070 con 0x5615c26244a0 2024-02-26T04:58:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.777+0000 7f7bfffff640 1 -- 172.21.15.60:0/1501450674 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c0c00fce0 con 0x5615c26244a0 2024-02-26T04:58:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.777+0000 7f7c29c84740 10 client.15705 fetch_fsmap learned FSMap version 23 2024-02-26T04:58:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.777+0000 7f7c29c84740 1 -- 172.21.15.60:0/1501450674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5615c25825e0 con 0x5615c26244a0 2024-02-26T04:58:31.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.777+0000 7f7bfffff640 1 -- 172.21.15.60:0/1501450674 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5615c22837e0 con 0x5615c26244a0 2024-02-26T04:58:31.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.778+0000 7f7bfffff640 1 -- 172.21.15.60:0/1501450674 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7c0c025020 con 0x5615c26244a0 2024-02-26T04:58:31.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.778+0000 7f7c29c84740 10 client.15705 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:58:31.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.778+0000 7f7c29c84740 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:58:31.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.778+0000 7f7c29c84740 20 client.15705 populate_metadata read hostname 'smithi060' 2024-02-26T04:58:31.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.778+0000 7f7c29c84740 1 --2- 172.21.15.60:0/1501450674 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5615c2639d70 0x5615c263c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.778+0000 7f7c29c84740 4 client.15705 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:58:31.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.778+0000 7f7c29c84740 1 -- 172.21.15.60:0/1501450674 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615c22c0720 con 0x5615c2639d70 2024-02-26T04:58:31.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.778+0000 7f7c24c4c640 1 --2- 172.21.15.60:0/1501450674 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5615c2639d70 0x5615c263c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.778+0000 7f7bfffff640 10 client.15705 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:31.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.778+0000 7f7c24c4c640 1 --2- 172.21.15.60:0/1501450674 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5615c2639d70 0x5615c263c190 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f56c4c9f5c8a230 server_cookie=1ab11738d6133e82 in_seq=0 out_seq=0 2024-02-26T04:58:31.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.779+0000 7f7bfffff640 1 -- 172.21.15.60:0/1501450674 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5615c22c0720 con 0x5615c2639d70 2024-02-26T04:58:31.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.779+0000 7f7bfffff640 10 client.15705 handle_command_reply: tid=0 2024-02-26T04:58:31.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.779+0000 7f7c29c84740 1 client.15705 shutdown 2024-02-26T04:58:31.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.779+0000 7f7c29c84740 1 -- 172.21.15.60:0/1501450674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c26244a0 msgr2=0x5615c262e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.779+0000 7f7c29c84740 1 --2- 172.21.15.60:0/1501450674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c26244a0 0x5615c262e060 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c012920 tx=0x7f7c0c004730 comp rx=0 tx=0).stop 2024-02-26T04:58:31.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.779+0000 7f7c29c84740 1 -- 172.21.15.60:0/1501450674 shutdown_connections 2024-02-26T04:58:31.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.779+0000 7f7c29c84740 1 --2- 172.21.15.60:0/1501450674 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5615c2639d70 0x5615c263c190 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.779+0000 7f7bfffff640 0 client.15705 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:31.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.779+0000 7f7c29c84740 1 --2- 172.21.15.60:0/1501450674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c2629610 0x5615c26350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.779+0000 7f7c29c84740 1 --2- 172.21.15.60:0/1501450674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c26244a0 0x5615c262e060 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.779+0000 7f7c29c84740 1 --2- 172.21.15.60:0/1501450674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2623ab0 0x5615c262db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.779+0000 7f7c29c84740 1 -- 172.21.15.60:0/1501450674 >> 172.21.15.60:0/1501450674 conn(0x5615c261efd0 msgr2=0x5615c2615a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:31.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.779+0000 7f7c29c84740 1 -- 172.21.15.60:0/1501450674 shutdown_connections 2024-02-26T04:58:31.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.779+0000 7f7c29c84740 1 -- 172.21.15.60:0/1501450674 wait complete. 2024-02-26T04:58:31.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.780+0000 7f7c29c84740 20 client.15705 trim_cache size 0 max 16384 2024-02-26T04:58:31.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c29c84740 1 Processor -- start 2024-02-26T04:58:31.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c29c84740 1 -- start start 2024-02-26T04:58:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c29c84740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2623ab0 0x5615c262db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c29c84740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c26244a0 0x5615c262e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c29c84740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2629610 0x5615c2635060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c29c84740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615c22c0720 con 0x5615c2629610 2024-02-26T04:58:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c29c84740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615c22837e0 con 0x5615c2623ab0 2024-02-26T04:58:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c29c84740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615c2624b90 con 0x5615c26244a0 2024-02-26T04:58:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c2544d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2629610 0x5615c2635060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c2544d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2629610 0x5615c2635060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59336/0 (socket says 172.21.15.60:59336) 2024-02-26T04:58:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c24c4c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2623ab0 0x5615c262db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c2544d640 1 -- 172.21.15.60:0/2096075584 learned_addr learned my addr 172.21.15.60:0/2096075584 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c1ffff640 1 --2- 172.21.15.60:0/2096075584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c26244a0 0x5615c262e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c24c4c640 1 -- 172.21.15.60:0/2096075584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c26244a0 msgr2=0x5615c262e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c24c4c640 1 --2- 172.21.15.60:0/2096075584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c26244a0 0x5615c262e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c24c4c640 1 -- 172.21.15.60:0/2096075584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2629610 msgr2=0x5615c2635060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c24c4c640 1 --2- 172.21.15.60:0/2096075584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2629610 0x5615c2635060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c24c4c640 1 -- 172.21.15.60:0/2096075584 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615c251ea90 con 0x5615c2623ab0 2024-02-26T04:58:31.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c1ffff640 1 --2- 172.21.15.60:0/2096075584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c26244a0 0x5615c262e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:58:31.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.795+0000 7f7c2544d640 1 --2- 172.21.15.60:0/2096075584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2629610 0x5615c2635060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:31.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.796+0000 7f7c24c4c640 1 --2- 172.21.15.60:0/2096075584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2623ab0 0x5615c262db00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x5615c2624d00 tx=0x7f7c14009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca6d3f4aeafb0ccf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:31.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.796+0000 7f7c0a7fc640 1 -- 172.21.15.60:0/2096075584 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c14048de0 con 0x5615c2623ab0 2024-02-26T04:58:31.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.796+0000 7f7c0a7fc640 1 -- 172.21.15.60:0/2096075584 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c1402cc80 con 0x5615c2623ab0 2024-02-26T04:58:31.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.796+0000 7f7c0a7fc640 1 -- 172.21.15.60:0/2096075584 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c14048690 con 0x5615c2623ab0 2024-02-26T04:58:31.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.797+0000 7f7c29c84740 1 -- 172.21.15.60:0/2096075584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2623ab0 msgr2=0x5615c262db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.797+0000 7f7c29c84740 1 --2- 172.21.15.60:0/2096075584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2623ab0 0x5615c262db00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x5615c2624d00 tx=0x7f7c14009260 comp rx=0 tx=0).stop 2024-02-26T04:58:31.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.797+0000 7f7c29c84740 1 -- 172.21.15.60:0/2096075584 shutdown_connections 2024-02-26T04:58:31.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.797+0000 7f7c29c84740 1 --2- 172.21.15.60:0/2096075584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2629610 0x5615c2635060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.797+0000 7f7c29c84740 1 --2- 172.21.15.60:0/2096075584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c26244a0 0x5615c262e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.797+0000 7f7c29c84740 1 --2- 172.21.15.60:0/2096075584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2623ab0 0x5615c262db00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.797+0000 7f7c29c84740 1 -- 172.21.15.60:0/2096075584 >> 172.21.15.60:0/2096075584 conn(0x5615c261efd0 msgr2=0x5615c261f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.797+0000 7f7c29c84740 1 -- 172.21.15.60:0/2096075584 shutdown_connections 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.797+0000 7f7c29c84740 1 -- 172.21.15.60:0/2096075584 wait complete. 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.797+0000 7f7c29c84740 1 -- start start 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.797+0000 7f7c29c84740 1 Processor -- start 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c29c84740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2623ab0 0x5615c2633810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c29c84740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c26244a0 0x5615c2633d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c29c84740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2629610 0x5615c2634290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c29c84740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615c26353f0 con 0x5615c2623ab0 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c29c84740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615c2690260 con 0x5615c2629610 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c29c84740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615c26903a0 con 0x5615c26244a0 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c29c84740 10 client.0 _ll_register_callbacks cb 0x7f7c18097340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c24c4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2623ab0 0x5615c2633810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c24c4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2623ab0 0x5615c2633810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59344/0 (socket says 172.21.15.60:59344) 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c24c4c640 1 -- 172.21.15.60:0/3183111493 learned_addr learned my addr 172.21.15.60:0/3183111493 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c1ffff640 1 --2- 172.21.15.60:0/3183111493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c26244a0 0x5615c2633d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c1ffff640 1 -- 172.21.15.60:0/3183111493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2629610 msgr2=0x5615c2634290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c1ffff640 1 --2- 172.21.15.60:0/3183111493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2629610 0x5615c2634290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c1ffff640 1 -- 172.21.15.60:0/3183111493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2623ab0 msgr2=0x5615c2633810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c1ffff640 1 --2- 172.21.15.60:0/3183111493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2623ab0 0x5615c2633810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c1ffff640 1 -- 172.21.15.60:0/3183111493 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615c2542540 con 0x5615c26244a0 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c24c4c640 1 --2- 172.21.15.60:0/3183111493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2623ab0 0x5615c2633810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7c1ffff640 1 --2- 172.21.15.60:0/3183111493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c26244a0 0x5615c2633d50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009640 tx=0x7f7c0c027b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d98d9bca59ab17ae server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.798+0000 7f7bfffff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.799+0000 7f7bfffff640 1 -- 172.21.15.60:0/3183111493 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c012ea0 con 0x5615c26244a0 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.799+0000 7f7bfffff640 1 -- 172.21.15.60:0/3183111493 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f7c0c018040 con 0x5615c26244a0 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.799+0000 7f7c29c84740 1 -- 172.21.15.60:0/3183111493 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c180993a0 con 0x5615c26244a0 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.799+0000 7f7bfffff640 1 -- 172.21.15.60:0/3183111493 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c0c027400 con 0x5615c26244a0 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.799+0000 7f7c29c84740 10 client.25273 fetch_fsmap learned FSMap version 23 2024-02-26T04:58:31.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.799+0000 7f7c29c84740 1 -- 172.21.15.60:0/3183111493 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f7c14030070 con 0x5615c26244a0 2024-02-26T04:58:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.799+0000 7f7bfffff640 1 -- 172.21.15.60:0/3183111493 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c0c016070 con 0x5615c26244a0 2024-02-26T04:58:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.799+0000 7f7bfffff640 1 -- 172.21.15.60:0/3183111493 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7c0c025050 con 0x5615c26244a0 2024-02-26T04:58:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.799+0000 7f7c29c84740 10 client.25273 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:58:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.799+0000 7f7c29c84740 10 client.25273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:58:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.799+0000 7f7c29c84740 20 client.25273 populate_metadata read hostname 'smithi060' 2024-02-26T04:58:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.800+0000 7f7c29c84740 1 --2- 172.21.15.60:0/3183111493 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5615c2639d70 0x5615c2693d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:58:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.800+0000 7f7c29c84740 4 client.25273 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T04:58:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.800+0000 7f7c29c84740 1 -- 172.21.15.60:0/3183111493 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f7c180993a0 con 0x5615c2639d70 2024-02-26T04:58:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.800+0000 7f7c24c4c640 1 --2- 172.21.15.60:0/3183111493 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5615c2639d70 0x5615c2693d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:58:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.800+0000 7f7bfffff640 10 client.25273 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.800+0000 7f7c24c4c640 1 --2- 172.21.15.60:0/3183111493 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5615c2639d70 0x5615c2693d60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d576537040ea1b78 server_cookie=19fcf06272204048 in_seq=0 out_seq=0 2024-02-26T04:58:31.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.800+0000 7f7bfffff640 1 -- 172.21.15.60:0/3183111493 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f7c180993a0 con 0x5615c2639d70 2024-02-26T04:58:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.800+0000 7f7bfffff640 10 client.25273 handle_command_reply: tid=0 2024-02-26T04:58:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.801+0000 7f7c29c84740 1 client.25273 shutdown 2024-02-26T04:58:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.801+0000 7f7c29c84740 1 -- 172.21.15.60:0/3183111493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c26244a0 msgr2=0x5615c2633d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.801+0000 7f7c29c84740 1 --2- 172.21.15.60:0/3183111493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c26244a0 0x5615c2633d50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009640 tx=0x7f7c0c027b00 comp rx=0 tx=0).stop 2024-02-26T04:58:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.801+0000 7f7c29c84740 1 -- 172.21.15.60:0/3183111493 shutdown_connections 2024-02-26T04:58:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.801+0000 7f7c29c84740 1 --2- 172.21.15.60:0/3183111493 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5615c2639d70 0x5615c2693d60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.801+0000 7f7bfffff640 0 client.25273 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:58:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.801+0000 7f7c29c84740 1 --2- 172.21.15.60:0/3183111493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5615c2629610 0x5615c2634290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.801+0000 7f7c29c84740 1 --2- 172.21.15.60:0/3183111493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5615c26244a0 0x5615c2633d50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.801+0000 7f7c29c84740 1 --2- 172.21.15.60:0/3183111493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5615c2623ab0 0x5615c2633810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.801+0000 7f7c29c84740 1 -- 172.21.15.60:0/3183111493 >> 172.21.15.60:0/3183111493 conn(0x5615c261efd0 msgr2=0x5615c262dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.801+0000 7f7c29c84740 1 -- 172.21.15.60:0/3183111493 shutdown_connections 2024-02-26T04:58:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.801+0000 7f7c29c84740 1 -- 172.21.15.60:0/3183111493 wait complete. 2024-02-26T04:58:31.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.801+0000 7f7c29c84740 20 client.25273 trim_cache size 0 max 16384 2024-02-26T04:58:31.810 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:58:31.810 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T04:58:31.810 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-02-26T04:58:31.810 INFO:teuthology.orchestra.run.smithi060.stdout: "d379eb85-f60d-43d7-a39f-305d2280a2a0": { 2024-02-26T04:58:31.810 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-02-26T04:58:31.810 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "d379eb85-f60d-43d7-a39f-305d2280a2a0", 2024-02-26T04:58:31.810 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-02-26T04:58:31.810 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:58:31.810 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:58:31.810 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:58:31.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.802+0000 7f7c26eb0640 1 -- 172.21.15.60:0/707765726 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7c100a20c0 msgr2=0x7f7c100a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.802+0000 7f7c26eb0640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7c100a20c0 0x7f7c100a4580 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7c201012f0 tx=0x7f7c0c005f90 comp rx=0 tx=0).stop 2024-02-26T04:58:31.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.802+0000 7f7c26eb0640 1 -- 172.21.15.60:0/707765726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c20106e70 msgr2=0x7f7c20100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:58:31.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.802+0000 7f7c26eb0640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c20106e70 0x7f7c20100100 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800eb40 tx=0x7f7c1800c2d0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.802+0000 7f7c26eb0640 1 -- 172.21.15.60:0/707765726 shutdown_connections 2024-02-26T04:58:31.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.802+0000 7f7c26eb0640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7c100a20c0 0x7f7c100a4580 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.802+0000 7f7c26eb0640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c20106e70 0x7f7c20100100 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.802+0000 7f7c26eb0640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c20104540 0x7f7c200ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.802+0000 7f7c26eb0640 1 --2- 172.21.15.60:0/707765726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c20101c10 0x7f7c20101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:58:31.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.802+0000 7f7c26eb0640 1 -- 172.21.15.60:0/707765726 >> 172.21.15.60:0/707765726 conn(0x7f7c200fb5e0 msgr2=0x7f7c201026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:58:31.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.802+0000 7f7c26eb0640 1 -- 172.21.15.60:0/707765726 shutdown_connections 2024-02-26T04:58:31.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:58:31.802+0000 7f7c26eb0640 1 -- 172.21.15.60:0/707765726 wait complete. 2024-02-26T04:58:31.817 INFO:tasks.cephfs.filesystem:scrub status for tag:d379eb85-f60d-43d7-a39f-305d2280a2a0 - {'path': '/', 'tag': 'd379eb85-f60d-43d7-a39f-305d2280a2a0', 'options': 'recursive,force'} 2024-02-26T04:58:32.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:31 smithi156 ceph-mon[35805]: pgmap v389: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:58:32.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:31 smithi060 ceph-mon[32382]: pgmap v389: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:58:32.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:31 smithi191 ceph-mon[36352]: pgmap v389: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-26T04:58:33.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:32 smithi156 ceph-mon[35805]: scrub queued for path: / 2024-02-26T04:58:33.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:32 smithi156 ceph-mon[35805]: scrub summary: idle+waiting paths [/] 2024-02-26T04:58:33.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:32 smithi156 ceph-mon[35805]: scrub summary: active paths [/] 2024-02-26T04:58:33.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:32 smithi156 ceph-mon[35805]: scrub summary: idle+waiting paths [/] 2024-02-26T04:58:33.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:32 smithi156 ceph-mon[35805]: scrub summary: idle 2024-02-26T04:58:33.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:32 smithi060 ceph-mon[32382]: scrub queued for path: / 2024-02-26T04:58:33.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:32 smithi060 ceph-mon[32382]: scrub summary: idle+waiting paths [/] 2024-02-26T04:58:33.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:32 smithi060 ceph-mon[32382]: scrub summary: active paths [/] 2024-02-26T04:58:33.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:32 smithi060 ceph-mon[32382]: scrub summary: idle+waiting paths [/] 2024-02-26T04:58:33.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:32 smithi060 ceph-mon[32382]: scrub summary: idle 2024-02-26T04:58:33.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:32 smithi191 ceph-mon[36352]: scrub queued for path: / 2024-02-26T04:58:33.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:32 smithi191 ceph-mon[36352]: scrub summary: idle+waiting paths [/] 2024-02-26T04:58:33.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:32 smithi191 ceph-mon[36352]: scrub summary: active paths [/] 2024-02-26T04:58:33.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:32 smithi191 ceph-mon[36352]: scrub summary: idle+waiting paths [/] 2024-02-26T04:58:33.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:32 smithi191 ceph-mon[36352]: scrub summary: idle 2024-02-26T04:58:34.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:33 smithi156 ceph-mon[35805]: pgmap v390: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:58:34.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:33 smithi060 ceph-mon[32382]: pgmap v390: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:58:34.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:33 smithi191 ceph-mon[36352]: pgmap v390: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-26T04:58:36.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:35 smithi156 ceph-mon[35805]: pgmap v391: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:58:36.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:35 smithi060 ceph-mon[32382]: pgmap v391: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:58:36.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:35 smithi191 ceph-mon[36352]: pgmap v391: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-26T04:58:38.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:37 smithi156 ceph-mon[35805]: pgmap v392: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-26T04:58:38.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:37 smithi060 ceph-mon[32382]: pgmap v392: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-26T04:58:38.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:37 smithi191 ceph-mon[36352]: pgmap v392: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-02-26T04:58:39.777 INFO:tasks.workunit.client.0.smithi060.stderr:Note: switching to 'bc0b84a094b787eb15ce0eff5f7a342217d95e39'. 2024-02-26T04:58:39.777 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-02-26T04:58:39.777 INFO:tasks.workunit.client.0.smithi060.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-26T04:58:39.777 INFO:tasks.workunit.client.0.smithi060.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-26T04:58:39.777 INFO:tasks.workunit.client.0.smithi060.stderr:state without impacting any branches by switching back to a branch. 2024-02-26T04:58:39.777 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-02-26T04:58:39.777 INFO:tasks.workunit.client.0.smithi060.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-26T04:58:39.777 INFO:tasks.workunit.client.0.smithi060.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-26T04:58:39.777 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-02-26T04:58:39.777 INFO:tasks.workunit.client.0.smithi060.stderr: git switch -c 2024-02-26T04:58:39.778 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-02-26T04:58:39.778 INFO:tasks.workunit.client.0.smithi060.stderr:Or undo this operation with: 2024-02-26T04:58:39.778 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-02-26T04:58:39.778 INFO:tasks.workunit.client.0.smithi060.stderr: git switch - 2024-02-26T04:58:39.778 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-02-26T04:58:39.778 INFO:tasks.workunit.client.0.smithi060.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-26T04:58:39.778 INFO:tasks.workunit.client.0.smithi060.stderr: 2024-02-26T04:58:39.778 INFO:tasks.workunit.client.0.smithi060.stderr:HEAD is now at bc0b84a094b client: check mds down status bofore getting mds_git_t from mdsmap 2024-02-26T04:58:39.781 DEBUG:teuthology.orchestra.run.smithi060:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-26T04:58:39.806 INFO:tasks.workunit.client.0.smithi060.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-26T04:58:39.808 INFO:tasks.workunit.client.0.smithi060.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-26T04:58:39.808 INFO:tasks.workunit.client.0.smithi060.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-26T04:58:40.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:39 smithi156 ceph-mon[35805]: pgmap v393: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T04:58:40.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:39 smithi060 ceph-mon[32382]: pgmap v393: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T04:58:40.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:39 smithi191 ceph-mon[36352]: pgmap v393: 129 pgs: 129 active+clean; 654 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-26T04:58:40.599 INFO:tasks.workunit.client.0.smithi060.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-26T04:58:40.657 INFO:tasks.workunit.client.0.smithi060.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-26T04:58:40.696 INFO:tasks.workunit.client.0.smithi060.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-26T04:58:40.698 INFO:tasks.workunit.client.0.smithi060.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-26T04:58:40.698 INFO:tasks.workunit.client.0.smithi060.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-26T04:58:40.736 INFO:tasks.workunit.client.0.smithi060.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-26T04:58:40.740 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T04:58:40.740 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-26T04:58:40.795 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-02-26T04:58:40.796 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-02-26T04:58:40.796 DEBUG:teuthology.orchestra.run.smithi060:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc0b84a094b787eb15ce0eff5f7a342217d95e39 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-02-26T04:58:40.855 INFO:tasks.workunit.client.0.smithi060.stdout:test large (16MB) dio write 2024-02-26T04:58:40.855 INFO:tasks.workunit.client.0.smithi060.stderr:+ echo 'test large (16MB) dio write' 2024-02-26T04:58:40.855 INFO:tasks.workunit.client.0.smithi060.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-02-26T04:58:40.922 INFO:tasks.workunit.client.0.smithi060.stderr:1+0 records in 2024-02-26T04:58:40.922 INFO:tasks.workunit.client.0.smithi060.stderr:1+0 records out 2024-02-26T04:58:40.922 INFO:tasks.workunit.client.0.smithi060.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.0651698 s, 257 MB/s 2024-02-26T04:58:40.922 INFO:tasks.workunit.client.0.smithi060.stdout:OK 2024-02-26T04:58:40.923 INFO:tasks.workunit.client.0.smithi060.stderr:+ echo OK 2024-02-26T04:58:40.923 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-26T04:58:40.923 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-26T04:58:40.998 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-02-26T04:58:40.998 DEBUG:teuthology.orchestra.run.smithi060:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc0b84a094b787eb15ce0eff5f7a342217d95e39 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-02-26T04:58:42.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:41 smithi156 ceph-mon[35805]: pgmap v394: 129 pgs: 129 active+clean; 659 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-26T04:58:42.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:41 smithi060 ceph-mon[32382]: pgmap v394: 129 pgs: 129 active+clean; 659 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-26T04:58:42.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:41 smithi191 ceph-mon[36352]: pgmap v394: 129 pgs: 129 active+clean; 659 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-02-26T04:58:44.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:43 smithi156 ceph-mon[35805]: pgmap v395: 129 pgs: 129 active+clean; 4.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 342 KiB/s wr, 6 op/s 2024-02-26T04:58:44.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:43 smithi060 ceph-mon[32382]: pgmap v395: 129 pgs: 129 active+clean; 4.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 342 KiB/s wr, 6 op/s 2024-02-26T04:58:44.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:43 smithi191 ceph-mon[36352]: pgmap v395: 129 pgs: 129 active+clean; 4.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 342 KiB/s wr, 6 op/s 2024-02-26T04:58:46.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:45 smithi156 ceph-mon[35805]: pgmap v396: 129 pgs: 129 active+clean; 4.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 342 KiB/s wr, 6 op/s 2024-02-26T04:58:46.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:45 smithi060 ceph-mon[32382]: pgmap v396: 129 pgs: 129 active+clean; 4.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 342 KiB/s wr, 6 op/s 2024-02-26T04:58:46.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:45 smithi191 ceph-mon[36352]: pgmap v396: 129 pgs: 129 active+clean; 4.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 342 KiB/s wr, 6 op/s 2024-02-26T04:58:48.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:47 smithi156 ceph-mon[35805]: pgmap v397: 129 pgs: 129 active+clean; 17 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 571 op/s 2024-02-26T04:58:48.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:47 smithi060 ceph-mon[32382]: pgmap v397: 129 pgs: 129 active+clean; 17 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 571 op/s 2024-02-26T04:58:48.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:47 smithi191 ceph-mon[36352]: pgmap v397: 129 pgs: 129 active+clean; 17 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 571 op/s 2024-02-26T04:58:50.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:49 smithi156 ceph-mon[35805]: pgmap v398: 129 pgs: 129 active+clean; 17 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 569 op/s 2024-02-26T04:58:50.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:49 smithi060 ceph-mon[32382]: pgmap v398: 129 pgs: 129 active+clean; 17 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 569 op/s 2024-02-26T04:58:50.258 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:49 smithi191 ceph-mon[36352]: pgmap v398: 129 pgs: 129 active+clean; 17 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 569 op/s 2024-02-26T04:58:51.002 INFO:tasks.workunit.client.0.smithi060.stdout:.......... 2024-02-26T04:58:51.003 INFO:tasks.workunit.client.0.smithi060.stdout:do_o_direct_reads: SUCCESS 2024-02-26T04:58:51.006 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-26T04:58:51.006 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-26T04:58:51.049 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-02-26T04:58:51.049 DEBUG:teuthology.orchestra.run.smithi060:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc0b84a094b787eb15ce0eff5f7a342217d95e39 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-02-26T04:58:51.113 INFO:tasks.workunit.client.0.smithi060.stdout:test read from hole 2024-02-26T04:58:51.113 INFO:tasks.workunit.client.0.smithi060.stderr:+ echo 'test read from hole' 2024-02-26T04:58:51.113 INFO:tasks.workunit.client.0.smithi060.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-02-26T04:58:51.115 INFO:tasks.workunit.client.0.smithi060.stderr:0+0 records in 2024-02-26T04:58:51.115 INFO:tasks.workunit.client.0.smithi060.stderr:0+0 records out 2024-02-26T04:58:51.115 INFO:tasks.workunit.client.0.smithi060.stderr:0 bytes copied, 3.7528e-05 s, 0.0 kB/s 2024-02-26T04:58:51.115 INFO:tasks.workunit.client.0.smithi060.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-02-26T04:58:51.117 INFO:tasks.workunit.client.0.smithi060.stderr:2+0 records in 2024-02-26T04:58:51.117 INFO:tasks.workunit.client.0.smithi060.stderr:2+0 records out 2024-02-26T04:58:51.117 INFO:tasks.workunit.client.0.smithi060.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0012333 s, 830 kB/s 2024-02-26T04:58:51.117 INFO:tasks.workunit.client.0.smithi060.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-02-26T04:58:51.118 INFO:tasks.workunit.client.0.smithi060.stderr:2+0 records in 2024-02-26T04:58:51.118 INFO:tasks.workunit.client.0.smithi060.stderr:2+0 records out 2024-02-26T04:58:51.118 INFO:tasks.workunit.client.0.smithi060.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 4.4019e-05 s, 23.3 MB/s 2024-02-26T04:58:51.118 INFO:tasks.workunit.client.0.smithi060.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-02-26T04:58:51.119 INFO:tasks.workunit.client.0.smithi060.stdout:other thing 2024-02-26T04:58:51.119 INFO:tasks.workunit.client.0.smithi060.stderr:+ echo 'other thing' 2024-02-26T04:58:51.119 INFO:tasks.workunit.client.0.smithi060.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-02-26T04:58:51.120 INFO:tasks.workunit.client.0.smithi060.stderr:1+0 records in 2024-02-26T04:58:51.120 INFO:tasks.workunit.client.0.smithi060.stderr:1+0 records out 2024-02-26T04:58:51.120 INFO:tasks.workunit.client.0.smithi060.stderr:500 bytes copied, 4.4129e-05 s, 11.3 MB/s 2024-02-26T04:58:51.120 INFO:tasks.workunit.client.0.smithi060.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-02-26T04:58:51.128 INFO:tasks.workunit.client.0.smithi060.stderr:0+1 records in 2024-02-26T04:58:51.128 INFO:tasks.workunit.client.0.smithi060.stderr:0+1 records out 2024-02-26T04:58:51.128 INFO:tasks.workunit.client.0.smithi060.stderr:500 bytes copied, 0.00700277 s, 71.4 kB/s 2024-02-26T04:58:51.128 INFO:tasks.workunit.client.0.smithi060.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-02-26T04:58:51.133 INFO:tasks.workunit.client.0.smithi060.stderr:0+1 records in 2024-02-26T04:58:51.133 INFO:tasks.workunit.client.0.smithi060.stderr:0+1 records out 2024-02-26T04:58:51.133 INFO:tasks.workunit.client.0.smithi060.stderr:500 bytes copied, 0.00477334 s, 105 kB/s 2024-02-26T04:58:51.133 INFO:tasks.workunit.client.0.smithi060.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-02-26T04:58:51.134 INFO:tasks.workunit.client.0.smithi060.stdout:OK 2024-02-26T04:58:51.134 INFO:tasks.workunit.client.0.smithi060.stderr:+ echo OK 2024-02-26T04:58:51.135 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-26T04:58:51.135 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-26T04:58:51.222 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-02-26T04:58:51.222 DEBUG:teuthology.orchestra.run.smithi060:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc0b84a094b787eb15ce0eff5f7a342217d95e39 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-02-26T04:58:51.253 INFO:tasks.workunit.client.0.smithi060.stdout:writing first 3 bytes of 10k file 2024-02-26T04:58:51.254 INFO:tasks.workunit.client.0.smithi060.stdout:reading O_DIRECT 2024-02-26T04:58:51.254 INFO:tasks.workunit.client.0.smithi060.stdout:got 10000 2024-02-26T04:58:51.254 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-26T04:58:51.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-26T04:58:51.333 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-02-26T04:58:51.334 DEBUG:teuthology.orchestra.run.smithi060:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc0b84a094b787eb15ce0eff5f7a342217d95e39 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-02-26T04:58:52.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:51 smithi156 ceph-mon[35805]: pgmap v399: 129 pgs: 129 active+clean; 17 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 571 op/s 2024-02-26T04:58:52.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:51 smithi060 ceph-mon[32382]: pgmap v399: 129 pgs: 129 active+clean; 17 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 571 op/s 2024-02-26T04:58:52.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:51 smithi191 ceph-mon[36352]: pgmap v399: 129 pgs: 129 active+clean; 17 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 571 op/s 2024-02-26T04:58:52.631 INFO:tasks.workunit.client.0.smithi060.stdout:writing pattern 2024-02-26T04:58:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-02-26T04:58:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-02-26T04:58:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-02-26T04:58:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-02-26T04:58:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-02-26T04:58:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-02-26T04:58:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-02-26T04:58:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-02-26T04:58:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-02-26T04:58:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-02-26T04:58:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-02-26T04:58:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-02-26T04:58:52.632 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-02-26T04:58:52.633 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-02-26T04:58:52.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-02-26T04:58:52.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-02-26T04:58:52.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-02-26T04:58:52.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-02-26T04:58:52.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-02-26T04:58:52.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-02-26T04:58:52.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-02-26T04:58:52.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-02-26T04:58:52.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-02-26T04:58:52.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-02-26T04:58:52.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-02-26T04:58:52.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-02-26T04:58:52.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-02-26T04:58:52.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-02-26T04:58:52.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-02-26T04:58:52.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-02-26T04:58:52.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-02-26T04:58:52.682 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-02-26T04:58:52.682 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-02-26T04:58:52.682 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-02-26T04:58:52.682 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-02-26T04:58:52.682 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-02-26T04:58:52.682 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-02-26T04:58:52.682 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-02-26T04:58:52.683 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-02-26T04:58:52.683 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-02-26T04:58:52.683 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-02-26T04:58:52.683 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-02-26T04:58:52.683 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-02-26T04:58:52.683 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-02-26T04:58:52.683 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-02-26T04:58:52.683 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-02-26T04:58:52.683 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-02-26T04:58:52.683 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-02-26T04:58:52.683 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-02-26T04:58:52.683 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-02-26T04:58:52.683 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-02-26T04:58:52.683 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-02-26T04:58:52.684 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-02-26T04:58:52.685 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-02-26T04:58:52.686 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-02-26T04:58:52.687 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-02-26T04:58:52.688 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-02-26T04:58:52.688 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-02-26T04:58:52.688 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-02-26T04:58:52.688 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-02-26T04:58:52.688 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-02-26T04:58:52.688 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-02-26T04:58:52.688 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-02-26T04:58:52.706 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-02-26T04:58:52.707 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-02-26T04:58:52.707 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-02-26T04:58:52.707 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-02-26T04:58:52.707 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-02-26T04:58:52.707 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-02-26T04:58:52.707 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-02-26T04:58:52.707 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-02-26T04:58:52.707 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-02-26T04:58:52.707 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-02-26T04:58:52.707 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-02-26T04:58:52.707 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-02-26T04:58:52.707 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-02-26T04:58:52.707 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-02-26T04:58:52.708 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-02-26T04:58:52.708 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-02-26T04:58:52.708 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-02-26T04:58:52.708 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-02-26T04:58:52.708 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-02-26T04:58:52.708 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-02-26T04:58:52.708 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-02-26T04:58:52.708 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-02-26T04:58:52.708 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-02-26T04:58:52.708 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-02-26T04:58:52.708 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-02-26T04:58:52.708 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-02-26T04:58:52.708 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-02-26T04:58:52.708 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-02-26T04:58:52.709 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-02-26T04:58:52.710 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-02-26T04:58:52.711 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-02-26T04:58:52.712 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-02-26T04:58:52.712 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-02-26T04:58:52.712 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-02-26T04:58:52.712 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-02-26T04:58:52.712 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-02-26T04:58:52.712 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-02-26T04:58:52.712 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-02-26T04:58:52.712 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-02-26T04:58:52.712 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-02-26T04:58:52.712 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-02-26T04:58:52.712 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-02-26T04:58:52.712 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-02-26T04:58:52.712 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-02-26T04:58:52.712 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-02-26T04:58:52.735 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-02-26T04:58:52.735 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-02-26T04:58:52.736 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-02-26T04:58:52.736 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-02-26T04:58:52.736 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-02-26T04:58:52.736 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-02-26T04:58:52.736 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-02-26T04:58:52.736 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-02-26T04:58:52.736 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-02-26T04:58:52.736 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-02-26T04:58:52.736 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-02-26T04:58:52.736 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-02-26T04:58:52.736 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-02-26T04:58:52.736 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-02-26T04:58:52.736 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-02-26T04:58:52.736 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-02-26T04:58:52.737 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-02-26T04:58:52.738 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-02-26T04:58:52.738 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-02-26T04:58:52.738 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-02-26T04:58:52.738 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-02-26T04:58:52.738 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-02-26T04:58:52.738 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-02-26T04:58:52.738 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-02-26T04:58:52.738 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-02-26T04:58:52.738 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-02-26T04:58:52.738 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-02-26T04:58:52.738 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-02-26T04:58:52.738 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-02-26T04:58:52.738 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-02-26T04:58:52.738 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-02-26T04:58:52.739 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-02-26T04:58:52.740 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-02-26T04:58:52.740 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-02-26T04:58:52.740 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-02-26T04:58:52.740 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-02-26T04:58:52.740 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-02-26T04:58:52.740 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-02-26T04:58:52.740 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-02-26T04:58:52.740 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-02-26T04:58:52.740 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-02-26T04:58:52.740 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-02-26T04:58:52.740 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-02-26T04:58:52.740 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-02-26T04:58:52.740 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-02-26T04:58:52.743 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-02-26T04:58:52.743 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-02-26T04:58:52.744 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-02-26T04:58:52.744 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-02-26T04:58:52.744 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-02-26T04:58:52.744 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-02-26T04:58:52.744 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-02-26T04:58:52.744 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-02-26T04:58:52.744 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-02-26T04:58:52.744 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-02-26T04:58:52.744 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-02-26T04:58:52.744 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-02-26T04:58:52.744 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-02-26T04:58:52.744 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-02-26T04:58:52.782 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-02-26T04:58:52.782 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-02-26T04:58:52.782 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-02-26T04:58:52.782 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-02-26T04:58:52.782 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-02-26T04:58:52.783 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-02-26T04:58:52.783 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-02-26T04:58:52.783 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-02-26T04:58:52.783 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-02-26T04:58:52.783 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-02-26T04:58:52.783 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-02-26T04:58:52.783 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-02-26T04:58:52.783 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-02-26T04:58:52.783 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-02-26T04:58:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-02-26T04:58:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-02-26T04:58:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-02-26T04:58:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-02-26T04:58:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-02-26T04:58:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-02-26T04:58:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-02-26T04:58:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-02-26T04:58:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-02-26T04:58:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-02-26T04:58:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-02-26T04:58:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-02-26T04:58:52.784 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-02-26T04:58:52.785 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-02-26T04:58:52.786 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-02-26T04:58:52.787 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-02-26T04:58:52.788 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-02-26T04:58:52.788 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-02-26T04:58:52.788 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-02-26T04:58:52.788 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-02-26T04:58:52.788 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-02-26T04:58:52.788 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-02-26T04:58:52.788 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-02-26T04:58:52.788 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-02-26T04:58:52.788 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-02-26T04:58:52.788 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-02-26T04:58:52.789 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-02-26T04:58:52.789 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-02-26T04:58:52.822 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-02-26T04:58:52.822 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-02-26T04:58:52.822 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-02-26T04:58:52.822 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-02-26T04:58:52.823 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-02-26T04:58:52.823 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-02-26T04:58:52.823 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-02-26T04:58:52.823 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-02-26T04:58:52.823 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-02-26T04:58:52.823 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-02-26T04:58:52.823 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-02-26T04:58:52.823 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-02-26T04:58:52.823 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-02-26T04:58:52.823 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-02-26T04:58:52.823 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-02-26T04:58:52.823 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-02-26T04:58:52.823 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-02-26T04:58:52.823 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-02-26T04:58:52.824 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-02-26T04:58:52.824 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-02-26T04:58:52.824 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-02-26T04:58:52.824 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-02-26T04:58:52.824 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-02-26T04:58:52.824 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-02-26T04:58:52.824 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-02-26T04:58:52.824 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-02-26T04:58:52.824 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-02-26T04:58:52.824 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-02-26T04:58:52.824 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-02-26T04:58:52.825 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-02-26T04:58:52.825 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-02-26T04:58:52.825 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-02-26T04:58:52.825 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-02-26T04:58:52.825 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-02-26T04:58:52.825 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-02-26T04:58:52.825 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-02-26T04:58:52.825 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-02-26T04:58:52.825 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-02-26T04:58:52.825 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-02-26T04:58:52.825 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-02-26T04:58:52.825 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-02-26T04:58:52.826 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-02-26T04:58:52.826 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-02-26T04:58:52.826 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-02-26T04:58:52.826 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-02-26T04:58:52.826 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-02-26T04:58:52.826 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-02-26T04:58:52.826 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-02-26T04:58:52.826 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-02-26T04:58:52.826 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-02-26T04:58:52.826 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-02-26T04:58:52.826 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-02-26T04:58:52.826 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-02-26T04:58:52.826 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-02-26T04:58:52.827 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-02-26T04:58:52.827 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-02-26T04:58:52.827 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-02-26T04:58:52.827 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-02-26T04:58:52.827 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-02-26T04:58:52.827 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-02-26T04:58:52.827 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-02-26T04:58:52.827 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-02-26T04:58:52.827 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-02-26T04:58:52.827 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-02-26T04:58:52.828 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-02-26T04:58:52.829 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-02-26T04:58:52.829 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-02-26T04:58:52.829 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-02-26T04:58:52.829 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-02-26T04:58:52.829 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-02-26T04:58:52.846 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-02-26T04:58:52.846 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-02-26T04:58:52.846 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-02-26T04:58:52.846 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-02-26T04:58:52.846 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-02-26T04:58:52.846 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-02-26T04:58:52.847 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-02-26T04:58:52.848 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-02-26T04:58:52.848 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-02-26T04:58:52.848 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-02-26T04:58:52.848 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-02-26T04:58:52.848 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-02-26T04:58:52.848 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-02-26T04:58:52.848 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-02-26T04:58:52.848 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-02-26T04:58:52.848 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-02-26T04:58:52.848 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-02-26T04:58:52.848 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-02-26T04:58:52.848 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-02-26T04:58:52.848 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-02-26T04:58:52.849 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-02-26T04:58:52.850 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-02-26T04:58:52.851 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-02-26T04:58:52.852 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-02-26T04:58:52.852 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-02-26T04:58:52.852 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-02-26T04:58:52.852 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-02-26T04:58:52.878 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-02-26T04:58:52.878 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-02-26T04:58:52.878 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-02-26T04:58:52.879 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-02-26T04:58:52.879 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-02-26T04:58:52.879 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-02-26T04:58:52.879 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-02-26T04:58:52.879 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-02-26T04:58:52.879 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-02-26T04:58:52.879 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-02-26T04:58:52.880 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-02-26T04:58:52.880 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-02-26T04:58:52.880 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-02-26T04:58:52.880 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-02-26T04:58:52.880 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-02-26T04:58:52.880 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-02-26T04:58:52.880 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-02-26T04:58:52.880 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-02-26T04:58:52.880 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-02-26T04:58:52.880 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-02-26T04:58:52.880 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-02-26T04:58:52.880 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-02-26T04:58:52.881 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-02-26T04:58:52.882 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-02-26T04:58:52.883 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-02-26T04:58:52.883 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-02-26T04:58:52.883 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-02-26T04:58:52.883 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-02-26T04:58:52.883 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-02-26T04:58:52.883 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-02-26T04:58:52.883 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-02-26T04:58:52.883 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-02-26T04:58:52.883 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-02-26T04:58:52.883 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-02-26T04:58:52.883 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-02-26T04:58:52.883 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-02-26T04:58:52.883 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-02-26T04:58:52.884 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-02-26T04:58:52.885 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-02-26T04:58:52.885 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-02-26T04:58:52.885 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-02-26T04:58:52.923 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-02-26T04:58:52.923 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-02-26T04:58:52.923 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-02-26T04:58:52.923 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-02-26T04:58:52.923 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-02-26T04:58:52.923 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-02-26T04:58:52.923 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-02-26T04:58:52.923 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-02-26T04:58:52.924 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-02-26T04:58:52.924 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-02-26T04:58:52.924 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-02-26T04:58:52.924 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-02-26T04:58:52.924 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-02-26T04:58:52.924 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-02-26T04:58:52.924 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-02-26T04:58:52.924 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-02-26T04:58:52.924 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-02-26T04:58:52.924 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-02-26T04:58:52.924 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-02-26T04:58:52.924 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-02-26T04:58:52.924 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-02-26T04:58:52.924 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-02-26T04:58:52.925 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-02-26T04:58:52.925 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-02-26T04:58:52.925 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-02-26T04:58:52.925 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-02-26T04:58:52.925 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-02-26T04:58:52.925 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-02-26T04:58:52.925 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-02-26T04:58:52.925 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-02-26T04:58:52.925 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-02-26T04:58:52.925 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-02-26T04:58:52.925 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-02-26T04:58:52.925 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-02-26T04:58:52.925 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-02-26T04:58:52.926 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-02-26T04:58:52.927 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-02-26T04:58:52.928 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-02-26T04:58:52.928 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-02-26T04:58:52.928 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-02-26T04:58:52.928 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-02-26T04:58:52.928 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-02-26T04:58:52.928 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-02-26T04:58:52.928 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-02-26T04:58:52.928 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-02-26T04:58:52.928 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-02-26T04:58:52.928 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-02-26T04:58:52.928 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-02-26T04:58:52.928 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-02-26T04:58:52.928 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-02-26T04:58:52.928 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-02-26T04:58:52.929 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-02-26T04:58:52.929 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-02-26T04:58:52.929 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-02-26T04:58:52.962 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-02-26T04:58:52.962 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-02-26T04:58:52.962 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-02-26T04:58:52.962 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-02-26T04:58:52.962 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-02-26T04:58:52.962 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-02-26T04:58:52.962 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-02-26T04:58:52.962 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-02-26T04:58:52.962 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-02-26T04:58:52.962 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-02-26T04:58:52.962 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-02-26T04:58:52.962 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-02-26T04:58:52.962 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-02-26T04:58:52.963 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-02-26T04:58:52.963 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-02-26T04:58:52.963 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-02-26T04:58:52.963 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-02-26T04:58:52.963 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-02-26T04:58:52.963 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-02-26T04:58:52.963 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-02-26T04:58:52.963 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-02-26T04:58:52.963 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-02-26T04:58:52.963 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-02-26T04:58:52.963 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-02-26T04:58:52.963 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-02-26T04:58:52.963 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-02-26T04:58:52.963 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-02-26T04:58:52.964 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-02-26T04:58:52.964 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-02-26T04:58:52.964 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-02-26T04:58:52.964 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-02-26T04:58:52.964 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-02-26T04:58:52.964 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-02-26T04:58:52.964 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-02-26T04:58:52.964 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-02-26T04:58:52.964 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-02-26T04:58:52.964 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-02-26T04:58:52.964 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-02-26T04:58:52.964 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-02-26T04:58:52.964 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-02-26T04:58:52.964 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-02-26T04:58:52.965 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-02-26T04:58:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-02-26T04:58:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-02-26T04:58:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-02-26T04:58:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-02-26T04:58:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-02-26T04:58:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-02-26T04:58:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-02-26T04:58:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-02-26T04:58:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-02-26T04:58:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-02-26T04:58:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-02-26T04:58:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-02-26T04:58:52.966 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-02-26T04:58:52.967 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-02-26T04:58:52.967 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-02-26T04:58:52.967 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-02-26T04:58:52.967 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-02-26T04:58:52.967 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-02-26T04:58:52.967 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-02-26T04:58:52.967 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-02-26T04:58:52.967 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-02-26T04:58:52.967 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-02-26T04:58:52.967 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-02-26T04:58:52.968 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-02-26T04:58:52.968 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-02-26T04:58:52.968 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-02-26T04:58:52.985 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-02-26T04:58:52.985 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-02-26T04:58:52.985 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-02-26T04:58:52.985 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-02-26T04:58:52.985 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-02-26T04:58:52.985 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-02-26T04:58:52.985 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-02-26T04:58:52.985 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-02-26T04:58:52.985 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-02-26T04:58:52.985 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-02-26T04:58:52.985 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-02-26T04:58:52.986 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-02-26T04:58:52.987 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-02-26T04:58:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-02-26T04:58:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-02-26T04:58:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-02-26T04:58:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-02-26T04:58:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-02-26T04:58:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-02-26T04:58:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-02-26T04:58:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-02-26T04:58:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-02-26T04:58:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-02-26T04:58:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-02-26T04:58:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-02-26T04:58:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-02-26T04:58:52.988 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-02-26T04:58:52.989 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-02-26T04:58:52.990 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-02-26T04:58:52.990 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-02-26T04:58:52.990 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-02-26T04:58:52.990 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-02-26T04:58:52.990 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-02-26T04:58:52.990 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-02-26T04:58:52.990 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-02-26T04:58:52.990 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-02-26T04:58:52.990 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-02-26T04:58:52.990 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-02-26T04:58:52.990 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-02-26T04:58:52.990 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-02-26T04:58:52.990 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-02-26T04:58:53.020 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-02-26T04:58:53.021 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-02-26T04:58:53.021 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-02-26T04:58:53.021 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-02-26T04:58:53.021 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-02-26T04:58:53.021 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-02-26T04:58:53.021 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-02-26T04:58:53.021 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-02-26T04:58:53.021 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-02-26T04:58:53.021 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-02-26T04:58:53.021 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-02-26T04:58:53.021 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-02-26T04:58:53.021 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-02-26T04:58:53.021 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-02-26T04:58:53.022 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-02-26T04:58:53.022 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-02-26T04:58:53.022 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-02-26T04:58:53.022 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-02-26T04:58:53.022 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-02-26T04:58:53.022 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-02-26T04:58:53.022 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-02-26T04:58:53.022 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-02-26T04:58:53.022 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-02-26T04:58:53.022 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-02-26T04:58:53.022 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-02-26T04:58:53.022 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-02-26T04:58:53.022 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-02-26T04:58:53.023 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-02-26T04:58:53.023 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-02-26T04:58:53.023 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-02-26T04:58:53.023 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-02-26T04:58:53.023 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-02-26T04:58:53.023 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-02-26T04:58:53.023 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-02-26T04:58:53.023 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-02-26T04:58:53.023 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-02-26T04:58:53.023 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-02-26T04:58:53.023 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-02-26T04:58:53.023 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-02-26T04:58:53.023 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-02-26T04:58:53.023 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-02-26T04:58:53.024 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-02-26T04:58:53.025 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-02-26T04:58:53.025 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-02-26T04:58:53.025 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-02-26T04:58:53.025 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-02-26T04:58:53.025 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-02-26T04:58:53.025 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-02-26T04:58:53.025 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-02-26T04:58:53.025 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-02-26T04:58:53.025 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-02-26T04:58:53.025 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-02-26T04:58:53.025 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-02-26T04:58:53.025 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-02-26T04:58:53.025 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-02-26T04:58:53.025 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-02-26T04:58:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-02-26T04:58:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-02-26T04:58:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-02-26T04:58:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-02-26T04:58:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-02-26T04:58:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-02-26T04:58:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-02-26T04:58:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-02-26T04:58:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-02-26T04:58:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-02-26T04:58:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-02-26T04:58:53.026 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-02-26T04:58:53.071 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-02-26T04:58:53.071 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-02-26T04:58:53.071 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-02-26T04:58:53.071 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-02-26T04:58:53.071 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-02-26T04:58:53.071 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-02-26T04:58:53.071 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-02-26T04:58:53.071 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-02-26T04:58:53.071 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-02-26T04:58:53.071 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-02-26T04:58:53.071 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-02-26T04:58:53.071 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-02-26T04:58:53.071 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-02-26T04:58:53.072 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-02-26T04:58:53.072 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-02-26T04:58:53.072 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-02-26T04:58:53.072 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-02-26T04:58:53.072 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-02-26T04:58:53.072 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-02-26T04:58:53.072 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-02-26T04:58:53.072 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-02-26T04:58:53.072 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-02-26T04:58:53.072 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-02-26T04:58:53.072 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-02-26T04:58:53.072 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-02-26T04:58:53.072 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-02-26T04:58:53.073 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-02-26T04:58:53.073 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-02-26T04:58:53.073 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-02-26T04:58:53.073 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-02-26T04:58:53.073 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-02-26T04:58:53.073 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-02-26T04:58:53.073 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-02-26T04:58:53.073 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-02-26T04:58:53.073 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-02-26T04:58:53.073 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-02-26T04:58:53.073 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-02-26T04:58:53.073 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-02-26T04:58:53.073 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-02-26T04:58:53.073 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-02-26T04:58:53.074 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-02-26T04:58:53.075 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-02-26T04:58:53.076 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-02-26T04:58:53.076 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-02-26T04:58:53.076 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-02-26T04:58:53.076 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-02-26T04:58:53.076 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-02-26T04:58:53.076 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-02-26T04:58:53.076 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-02-26T04:58:53.076 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-02-26T04:58:53.076 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-02-26T04:58:53.076 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-02-26T04:58:53.076 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-02-26T04:58:53.103 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-02-26T04:58:53.103 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-02-26T04:58:53.103 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-02-26T04:58:53.103 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-02-26T04:58:53.103 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-02-26T04:58:53.103 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-02-26T04:58:53.103 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-02-26T04:58:53.103 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-02-26T04:58:53.104 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-02-26T04:58:53.104 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-02-26T04:58:53.104 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-02-26T04:58:53.104 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-02-26T04:58:53.104 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-02-26T04:58:53.104 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-02-26T04:58:53.104 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-02-26T04:58:53.104 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-02-26T04:58:53.104 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-02-26T04:58:53.104 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-02-26T04:58:53.104 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-02-26T04:58:53.104 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-02-26T04:58:53.104 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-02-26T04:58:53.104 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-02-26T04:58:53.105 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-02-26T04:58:53.106 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-02-26T04:58:53.106 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-02-26T04:58:53.106 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-02-26T04:58:53.106 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-02-26T04:58:53.106 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-02-26T04:58:53.106 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-02-26T04:58:53.106 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-02-26T04:58:53.107 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-02-26T04:58:53.107 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-02-26T04:58:53.107 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-02-26T04:58:53.107 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-02-26T04:58:53.107 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-02-26T04:58:53.107 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-02-26T04:58:53.107 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-02-26T04:58:53.107 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-02-26T04:58:53.107 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-02-26T04:58:53.108 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-02-26T04:58:53.109 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-02-26T04:58:53.109 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-02-26T04:58:53.109 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-02-26T04:58:53.109 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-02-26T04:58:53.109 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-02-26T04:58:53.109 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-02-26T04:58:53.109 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-02-26T04:58:53.109 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-02-26T04:58:53.109 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-02-26T04:58:53.109 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-02-26T04:58:53.109 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-02-26T04:58:53.109 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-02-26T04:58:53.109 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-02-26T04:58:53.109 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-02-26T04:58:53.110 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-02-26T04:58:53.126 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-02-26T04:58:53.126 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-02-26T04:58:53.126 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-02-26T04:58:53.126 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-02-26T04:58:53.126 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-02-26T04:58:53.126 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-02-26T04:58:53.127 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-02-26T04:58:53.128 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-02-26T04:58:53.128 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-02-26T04:58:53.128 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-02-26T04:58:53.128 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-02-26T04:58:53.128 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-02-26T04:58:53.128 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-02-26T04:58:53.128 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-02-26T04:58:53.128 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-02-26T04:58:53.128 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-02-26T04:58:53.128 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-02-26T04:58:53.128 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-02-26T04:58:53.128 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-02-26T04:58:53.128 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-02-26T04:58:53.128 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-02-26T04:58:53.129 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-02-26T04:58:53.130 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-02-26T04:58:53.131 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-02-26T04:58:53.132 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-02-26T04:58:53.132 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-02-26T04:58:53.166 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-02-26T04:58:53.166 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-02-26T04:58:53.166 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-02-26T04:58:53.166 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-02-26T04:58:53.166 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-02-26T04:58:53.166 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-02-26T04:58:53.166 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-02-26T04:58:53.166 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-02-26T04:58:53.166 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-02-26T04:58:53.166 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-02-26T04:58:53.166 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-02-26T04:58:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-02-26T04:58:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-02-26T04:58:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-02-26T04:58:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-02-26T04:58:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-02-26T04:58:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-02-26T04:58:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-02-26T04:58:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-02-26T04:58:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-02-26T04:58:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-02-26T04:58:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-02-26T04:58:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-02-26T04:58:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-02-26T04:58:53.167 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-02-26T04:58:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-02-26T04:58:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-02-26T04:58:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-02-26T04:58:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-02-26T04:58:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-02-26T04:58:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-02-26T04:58:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-02-26T04:58:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-02-26T04:58:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-02-26T04:58:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-02-26T04:58:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-02-26T04:58:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-02-26T04:58:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-02-26T04:58:53.168 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-02-26T04:58:53.169 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-02-26T04:58:53.170 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-02-26T04:58:53.171 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-02-26T04:58:53.171 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-02-26T04:58:53.171 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-02-26T04:58:53.171 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-02-26T04:58:53.171 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-02-26T04:58:53.171 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-02-26T04:58:53.171 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-02-26T04:58:53.171 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-02-26T04:58:53.171 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-02-26T04:58:53.171 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-02-26T04:58:53.171 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-02-26T04:58:53.171 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-02-26T04:58:53.171 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-02-26T04:58:53.213 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-02-26T04:58:53.214 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-02-26T04:58:53.214 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-02-26T04:58:53.214 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-02-26T04:58:53.214 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-02-26T04:58:53.214 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-02-26T04:58:53.214 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-02-26T04:58:53.214 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-02-26T04:58:53.214 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-02-26T04:58:53.214 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-02-26T04:58:53.214 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-02-26T04:58:53.215 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-02-26T04:58:53.215 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-02-26T04:58:53.215 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-02-26T04:58:53.215 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-02-26T04:58:53.215 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-02-26T04:58:53.215 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-02-26T04:58:53.215 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-02-26T04:58:53.216 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-02-26T04:58:53.216 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-02-26T04:58:53.216 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-02-26T04:58:53.216 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-02-26T04:58:53.216 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-02-26T04:58:53.216 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-02-26T04:58:53.216 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-02-26T04:58:53.216 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-02-26T04:58:53.216 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-02-26T04:58:53.216 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-02-26T04:58:53.217 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-02-26T04:58:53.217 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-02-26T04:58:53.217 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-02-26T04:58:53.217 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-02-26T04:58:53.217 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-02-26T04:58:53.217 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-02-26T04:58:53.217 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-02-26T04:58:53.217 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-02-26T04:58:53.217 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-02-26T04:58:53.217 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-02-26T04:58:53.217 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-02-26T04:58:53.217 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-02-26T04:58:53.217 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-02-26T04:58:53.218 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-02-26T04:58:53.218 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-02-26T04:58:53.218 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-02-26T04:58:53.218 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-02-26T04:58:53.218 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-02-26T04:58:53.218 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-02-26T04:58:53.218 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-02-26T04:58:53.218 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-02-26T04:58:53.218 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-02-26T04:58:53.218 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-02-26T04:58:53.218 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-02-26T04:58:53.218 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-02-26T04:58:53.218 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-02-26T04:58:53.219 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-02-26T04:58:53.219 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-02-26T04:58:53.219 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-02-26T04:58:53.219 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-02-26T04:58:53.219 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-02-26T04:58:53.219 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-02-26T04:58:53.219 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-02-26T04:58:53.219 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-02-26T04:58:53.219 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-02-26T04:58:53.219 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-02-26T04:58:53.219 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-02-26T04:58:53.220 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-02-26T04:58:53.220 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-02-26T04:58:53.220 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-02-26T04:58:53.220 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-02-26T04:58:53.220 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-02-26T04:58:53.220 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-02-26T04:58:53.220 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-02-26T04:58:53.220 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-02-26T04:58:53.220 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-02-26T04:58:53.220 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-02-26T04:58:53.220 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-02-26T04:58:53.220 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-02-26T04:58:53.220 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-02-26T04:58:53.221 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-02-26T04:58:53.221 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-02-26T04:58:53.221 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-02-26T04:58:53.221 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-02-26T04:58:53.242 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-02-26T04:58:53.242 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-02-26T04:58:53.242 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-02-26T04:58:53.242 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-02-26T04:58:53.242 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-02-26T04:58:53.242 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-02-26T04:58:53.242 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-02-26T04:58:53.242 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-02-26T04:58:53.242 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-02-26T04:58:53.243 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-02-26T04:58:53.243 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-02-26T04:58:53.243 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-02-26T04:58:53.243 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-02-26T04:58:53.243 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-02-26T04:58:53.243 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-02-26T04:58:53.243 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-02-26T04:58:53.243 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-02-26T04:58:53.243 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-02-26T04:58:53.243 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-02-26T04:58:53.243 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-02-26T04:58:53.243 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-02-26T04:58:53.243 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-02-26T04:58:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-02-26T04:58:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-02-26T04:58:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-02-26T04:58:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-02-26T04:58:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-02-26T04:58:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-02-26T04:58:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-02-26T04:58:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-02-26T04:58:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-02-26T04:58:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-02-26T04:58:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-02-26T04:58:53.244 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-02-26T04:58:53.245 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-02-26T04:58:53.245 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-02-26T04:58:53.245 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-02-26T04:58:53.245 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-02-26T04:58:53.245 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-02-26T04:58:53.245 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-02-26T04:58:53.245 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-02-26T04:58:53.245 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-02-26T04:58:53.245 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-02-26T04:58:53.245 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-02-26T04:58:53.245 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-02-26T04:58:53.245 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-02-26T04:58:53.246 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-02-26T04:58:53.246 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-02-26T04:58:53.246 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-02-26T04:58:53.246 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-02-26T04:58:53.246 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-02-26T04:58:53.246 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-02-26T04:58:53.246 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-02-26T04:58:53.246 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-02-26T04:58:53.246 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-02-26T04:58:53.246 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-02-26T04:58:53.246 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-02-26T04:58:53.246 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-02-26T04:58:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-02-26T04:58:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-02-26T04:58:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-02-26T04:58:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-02-26T04:58:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-02-26T04:58:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-02-26T04:58:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-02-26T04:58:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-02-26T04:58:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-02-26T04:58:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-02-26T04:58:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-02-26T04:58:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-02-26T04:58:53.247 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-02-26T04:58:53.248 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-02-26T04:58:53.248 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-02-26T04:58:53.248 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-02-26T04:58:53.248 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-02-26T04:58:53.248 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-02-26T04:58:53.248 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-02-26T04:58:53.248 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-02-26T04:58:53.248 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-02-26T04:58:53.248 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-02-26T04:58:53.248 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-02-26T04:58:53.248 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-02-26T04:58:53.265 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-02-26T04:58:53.265 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-02-26T04:58:53.265 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-02-26T04:58:53.265 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-02-26T04:58:53.265 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-02-26T04:58:53.265 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-02-26T04:58:53.265 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-02-26T04:58:53.265 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-02-26T04:58:53.266 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-02-26T04:58:53.267 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-02-26T04:58:53.267 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-02-26T04:58:53.267 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-02-26T04:58:53.267 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-02-26T04:58:53.267 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-02-26T04:58:53.267 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-02-26T04:58:53.267 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-02-26T04:58:53.267 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-02-26T04:58:53.267 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-02-26T04:58:53.267 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-02-26T04:58:53.267 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-02-26T04:58:53.267 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-02-26T04:58:53.267 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-02-26T04:58:53.267 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-02-26T04:58:53.268 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-02-26T04:58:53.268 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-02-26T04:58:53.268 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-02-26T04:58:53.268 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-02-26T04:58:53.268 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-02-26T04:58:53.268 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-02-26T04:58:53.268 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-02-26T04:58:53.268 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-02-26T04:58:53.268 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-02-26T04:58:53.268 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-02-26T04:58:53.268 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-02-26T04:58:53.268 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-02-26T04:58:53.268 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-02-26T04:58:53.269 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-02-26T04:58:53.269 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-02-26T04:58:53.269 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-02-26T04:58:53.269 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-02-26T04:58:53.269 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-02-26T04:58:53.269 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-02-26T04:58:53.269 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-02-26T04:58:53.269 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-02-26T04:58:53.269 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-02-26T04:58:53.270 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-02-26T04:58:53.270 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-02-26T04:58:53.270 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-02-26T04:58:53.270 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-02-26T04:58:53.270 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-02-26T04:58:53.270 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-02-26T04:58:53.270 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-02-26T04:58:53.270 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-02-26T04:58:53.270 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-02-26T04:58:53.270 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-02-26T04:58:53.270 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-02-26T04:58:53.271 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-02-26T04:58:53.271 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-02-26T04:58:53.271 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-02-26T04:58:53.271 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-02-26T04:58:53.271 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-02-26T04:58:53.271 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-02-26T04:58:53.271 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-02-26T04:58:53.271 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-02-26T04:58:53.271 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-02-26T04:58:53.271 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-02-26T04:58:53.271 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-02-26T04:58:53.271 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-02-26T04:58:53.271 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-02-26T04:58:53.304 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-02-26T04:58:53.304 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-02-26T04:58:53.304 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-02-26T04:58:53.304 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-02-26T04:58:53.304 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-02-26T04:58:53.304 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-02-26T04:58:53.304 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-02-26T04:58:53.304 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-02-26T04:58:53.304 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-02-26T04:58:53.305 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-02-26T04:58:53.305 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-02-26T04:58:53.305 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-02-26T04:58:53.305 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-02-26T04:58:53.305 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-02-26T04:58:53.305 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-02-26T04:58:53.305 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-02-26T04:58:53.305 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-02-26T04:58:53.305 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-02-26T04:58:53.305 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-02-26T04:58:53.306 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-02-26T04:58:53.306 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-02-26T04:58:53.306 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-02-26T04:58:53.306 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-02-26T04:58:53.306 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-02-26T04:58:53.306 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-02-26T04:58:53.306 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-02-26T04:58:53.306 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-02-26T04:58:53.306 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-02-26T04:58:53.306 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-02-26T04:58:53.306 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-02-26T04:58:53.307 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-02-26T04:58:53.307 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-02-26T04:58:53.307 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-02-26T04:58:53.307 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-02-26T04:58:53.307 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-02-26T04:58:53.307 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-02-26T04:58:53.307 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-02-26T04:58:53.307 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-02-26T04:58:53.307 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-02-26T04:58:53.307 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-02-26T04:58:53.307 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-02-26T04:58:53.307 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-02-26T04:58:53.308 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-02-26T04:58:53.308 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-02-26T04:58:53.308 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-02-26T04:58:53.308 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-02-26T04:58:53.308 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-02-26T04:58:53.308 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-02-26T04:58:53.308 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-02-26T04:58:53.308 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-02-26T04:58:53.308 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-02-26T04:58:53.308 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-02-26T04:58:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-02-26T04:58:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-02-26T04:58:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-02-26T04:58:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-02-26T04:58:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-02-26T04:58:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-02-26T04:58:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-02-26T04:58:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-02-26T04:58:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-02-26T04:58:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-02-26T04:58:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-02-26T04:58:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-02-26T04:58:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-02-26T04:58:53.309 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-02-26T04:58:53.310 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-02-26T04:58:53.311 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-02-26T04:58:53.356 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-02-26T04:58:53.356 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-02-26T04:58:53.356 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-02-26T04:58:53.356 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-02-26T04:58:53.356 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-02-26T04:58:53.357 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-02-26T04:58:53.357 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-02-26T04:58:53.357 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-02-26T04:58:53.357 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-02-26T04:58:53.357 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-02-26T04:58:53.357 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-02-26T04:58:53.357 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-02-26T04:58:53.357 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-02-26T04:58:53.357 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-02-26T04:58:53.357 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-02-26T04:58:53.358 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-02-26T04:58:53.358 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-02-26T04:58:53.358 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-02-26T04:58:53.358 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-02-26T04:58:53.358 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-02-26T04:58:53.358 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-02-26T04:58:53.358 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-02-26T04:58:53.359 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-02-26T04:58:53.359 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-02-26T04:58:53.359 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-02-26T04:58:53.359 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-02-26T04:58:53.359 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-02-26T04:58:53.359 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-02-26T04:58:53.359 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-02-26T04:58:53.359 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-02-26T04:58:53.359 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-02-26T04:58:53.359 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-02-26T04:58:53.359 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-02-26T04:58:53.359 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-02-26T04:58:53.360 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-02-26T04:58:53.360 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-02-26T04:58:53.360 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-02-26T04:58:53.360 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-02-26T04:58:53.360 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-02-26T04:58:53.360 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-02-26T04:58:53.360 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-02-26T04:58:53.360 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-02-26T04:58:53.360 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-02-26T04:58:53.360 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-02-26T04:58:53.360 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-02-26T04:58:53.360 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-02-26T04:58:53.360 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-02-26T04:58:53.360 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-02-26T04:58:53.361 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-02-26T04:58:53.362 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-02-26T04:58:53.362 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-02-26T04:58:53.362 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-02-26T04:58:53.362 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-02-26T04:58:53.362 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-02-26T04:58:53.362 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-02-26T04:58:53.362 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-02-26T04:58:53.362 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-02-26T04:58:53.362 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-02-26T04:58:53.362 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-02-26T04:58:53.362 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-02-26T04:58:53.362 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-02-26T04:58:53.362 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-02-26T04:58:53.362 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-02-26T04:58:53.363 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-02-26T04:58:53.363 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-02-26T04:58:53.363 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-02-26T04:58:53.363 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-02-26T04:58:53.363 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-02-26T04:58:53.363 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-02-26T04:58:53.381 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-02-26T04:58:53.381 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-02-26T04:58:53.381 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-02-26T04:58:53.382 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-02-26T04:58:53.382 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-02-26T04:58:53.382 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-02-26T04:58:53.382 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-02-26T04:58:53.382 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-02-26T04:58:53.382 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-02-26T04:58:53.382 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-02-26T04:58:53.382 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-02-26T04:58:53.382 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-02-26T04:58:53.382 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-02-26T04:58:53.382 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-02-26T04:58:53.382 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-02-26T04:58:53.383 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-02-26T04:58:53.383 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-02-26T04:58:53.383 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-02-26T04:58:53.383 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-02-26T04:58:53.383 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-02-26T04:58:53.383 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-02-26T04:58:53.383 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-02-26T04:58:53.383 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-02-26T04:58:53.383 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-02-26T04:58:53.383 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-02-26T04:58:53.383 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-02-26T04:58:53.383 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-02-26T04:58:53.383 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-02-26T04:58:53.384 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-02-26T04:58:53.384 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-02-26T04:58:53.384 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-02-26T04:58:53.384 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-02-26T04:58:53.384 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-02-26T04:58:53.384 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-02-26T04:58:53.384 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-02-26T04:58:53.384 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-02-26T04:58:53.384 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-02-26T04:58:53.384 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-02-26T04:58:53.384 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-02-26T04:58:53.384 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-02-26T04:58:53.384 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-02-26T04:58:53.385 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-02-26T04:58:53.385 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-02-26T04:58:53.385 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-02-26T04:58:53.385 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-02-26T04:58:53.385 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-02-26T04:58:53.385 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-02-26T04:58:53.385 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-02-26T04:58:53.385 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-02-26T04:58:53.385 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-02-26T04:58:53.385 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-02-26T04:58:53.385 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-02-26T04:58:53.385 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-02-26T04:58:53.385 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-02-26T04:58:53.386 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-02-26T04:58:53.386 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-02-26T04:58:53.386 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-02-26T04:58:53.386 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-02-26T04:58:53.386 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-02-26T04:58:53.386 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-02-26T04:58:53.386 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-02-26T04:58:53.386 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-02-26T04:58:53.386 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-02-26T04:58:53.386 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-02-26T04:58:53.386 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-02-26T04:58:53.386 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-02-26T04:58:53.386 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-02-26T04:58:53.386 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-02-26T04:58:53.387 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-02-26T04:58:53.387 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-02-26T04:58:53.387 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-02-26T04:58:53.387 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-02-26T04:58:53.387 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-02-26T04:58:53.387 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-02-26T04:58:53.387 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-02-26T04:58:53.387 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-02-26T04:58:53.387 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-02-26T04:58:53.387 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-02-26T04:58:53.387 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-02-26T04:58:53.387 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-02-26T04:58:53.387 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-02-26T04:58:53.387 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-02-26T04:58:53.406 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-02-26T04:58:53.406 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-02-26T04:58:53.406 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-02-26T04:58:53.406 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-02-26T04:58:53.406 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-02-26T04:58:53.406 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-02-26T04:58:53.406 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-02-26T04:58:53.406 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-02-26T04:58:53.406 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-02-26T04:58:53.406 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-02-26T04:58:53.407 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-02-26T04:58:53.408 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-02-26T04:58:53.408 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-02-26T04:58:53.408 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-02-26T04:58:53.408 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-02-26T04:58:53.408 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-02-26T04:58:53.408 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-02-26T04:58:53.408 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-02-26T04:58:53.408 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-02-26T04:58:53.408 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-02-26T04:58:53.408 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-02-26T04:58:53.408 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-02-26T04:58:53.408 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-02-26T04:58:53.408 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-02-26T04:58:53.409 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-02-26T04:58:53.410 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-02-26T04:58:53.410 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-02-26T04:58:53.410 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-02-26T04:58:53.410 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-02-26T04:58:53.410 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-02-26T04:58:53.410 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-02-26T04:58:53.410 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-02-26T04:58:53.410 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-02-26T04:58:53.410 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-02-26T04:58:53.410 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-02-26T04:58:53.410 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-02-26T04:58:53.410 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-02-26T04:58:53.410 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-02-26T04:58:53.410 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-02-26T04:58:53.411 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-02-26T04:58:53.447 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-02-26T04:58:53.448 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-02-26T04:58:53.449 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-02-26T04:58:53.449 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-02-26T04:58:53.449 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-02-26T04:58:53.449 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-02-26T04:58:53.449 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-02-26T04:58:53.449 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-02-26T04:58:53.449 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-02-26T04:58:53.449 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-02-26T04:58:53.449 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-02-26T04:58:53.449 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-02-26T04:58:53.449 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-02-26T04:58:53.450 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-02-26T04:58:53.450 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-02-26T04:58:53.450 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-02-26T04:58:53.450 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-02-26T04:58:53.450 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-02-26T04:58:53.450 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-02-26T04:58:53.450 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-02-26T04:58:53.450 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-02-26T04:58:53.450 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-02-26T04:58:53.450 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-02-26T04:58:53.450 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-02-26T04:58:53.451 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-02-26T04:58:53.452 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-02-26T04:58:53.452 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-02-26T04:58:53.452 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-02-26T04:58:53.452 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-02-26T04:58:53.452 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-02-26T04:58:53.452 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-02-26T04:58:53.452 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-02-26T04:58:53.452 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-02-26T04:58:53.452 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-02-26T04:58:53.452 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-02-26T04:58:53.452 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-02-26T04:58:53.452 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-02-26T04:58:53.452 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-02-26T04:58:53.453 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-02-26T04:58:53.453 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-02-26T04:58:53.453 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-02-26T04:58:53.495 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-02-26T04:58:53.495 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-02-26T04:58:53.495 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-02-26T04:58:53.495 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-02-26T04:58:53.495 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-02-26T04:58:53.495 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-02-26T04:58:53.495 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-02-26T04:58:53.496 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-02-26T04:58:53.496 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-02-26T04:58:53.496 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-02-26T04:58:53.496 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-02-26T04:58:53.496 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-02-26T04:58:53.496 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-02-26T04:58:53.496 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-02-26T04:58:53.496 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-02-26T04:58:53.496 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-02-26T04:58:53.496 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-02-26T04:58:53.496 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-02-26T04:58:53.496 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-02-26T04:58:53.496 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-02-26T04:58:53.497 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-02-26T04:58:53.498 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-02-26T04:58:53.499 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-02-26T04:58:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-02-26T04:58:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-02-26T04:58:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-02-26T04:58:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-02-26T04:58:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-02-26T04:58:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-02-26T04:58:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-02-26T04:58:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-02-26T04:58:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-02-26T04:58:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-02-26T04:58:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-02-26T04:58:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-02-26T04:58:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-02-26T04:58:53.500 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-02-26T04:58:53.501 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-02-26T04:58:53.501 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-02-26T04:58:53.501 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-02-26T04:58:53.519 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-02-26T04:58:53.519 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-02-26T04:58:53.519 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-02-26T04:58:53.519 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-02-26T04:58:53.519 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-02-26T04:58:53.519 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-02-26T04:58:53.520 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-02-26T04:58:53.520 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-02-26T04:58:53.520 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-02-26T04:58:53.520 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-02-26T04:58:53.520 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-02-26T04:58:53.520 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-02-26T04:58:53.520 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-02-26T04:58:53.520 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-02-26T04:58:53.520 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-02-26T04:58:53.520 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-02-26T04:58:53.520 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-02-26T04:58:53.520 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-02-26T04:58:53.520 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-02-26T04:58:53.521 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-02-26T04:58:53.522 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-02-26T04:58:53.523 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-02-26T04:58:53.524 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-02-26T04:58:53.524 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-02-26T04:58:53.524 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-02-26T04:58:53.524 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-02-26T04:58:53.524 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-02-26T04:58:53.524 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-02-26T04:58:53.524 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-02-26T04:58:53.524 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-02-26T04:58:53.524 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-02-26T04:58:53.524 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-02-26T04:58:53.524 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-02-26T04:58:53.524 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-02-26T04:58:53.524 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-02-26T04:58:53.524 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-02-26T04:58:53.525 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-02-26T04:58:53.525 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-02-26T04:58:53.525 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-02-26T04:58:53.525 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-02-26T04:58:53.525 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-02-26T04:58:53.543 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-02-26T04:58:53.543 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-02-26T04:58:53.543 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-02-26T04:58:53.543 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-02-26T04:58:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-02-26T04:58:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-02-26T04:58:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-02-26T04:58:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-02-26T04:58:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-02-26T04:58:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-02-26T04:58:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-02-26T04:58:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-02-26T04:58:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-02-26T04:58:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-02-26T04:58:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-02-26T04:58:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-02-26T04:58:53.544 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-02-26T04:58:53.545 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-02-26T04:58:53.546 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-02-26T04:58:53.547 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-02-26T04:58:53.548 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-02-26T04:58:53.548 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-02-26T04:58:53.548 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-02-26T04:58:53.548 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-02-26T04:58:53.548 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-02-26T04:58:53.548 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-02-26T04:58:53.548 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-02-26T04:58:53.548 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-02-26T04:58:53.548 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-02-26T04:58:53.548 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-02-26T04:58:53.548 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-02-26T04:58:53.548 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-02-26T04:58:53.548 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-02-26T04:58:53.548 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-02-26T04:58:53.549 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-02-26T04:58:53.549 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-02-26T04:58:53.549 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-02-26T04:58:53.549 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-02-26T04:58:53.549 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-02-26T04:58:53.549 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-02-26T04:58:53.587 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-02-26T04:58:53.587 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-02-26T04:58:53.587 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-02-26T04:58:53.587 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-02-26T04:58:53.587 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-02-26T04:58:53.587 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-02-26T04:58:53.587 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-02-26T04:58:53.587 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-02-26T04:58:53.587 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-02-26T04:58:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-02-26T04:58:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-02-26T04:58:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-02-26T04:58:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-02-26T04:58:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-02-26T04:58:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-02-26T04:58:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-02-26T04:58:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-02-26T04:58:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-02-26T04:58:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-02-26T04:58:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-02-26T04:58:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-02-26T04:58:53.588 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-02-26T04:58:53.589 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-02-26T04:58:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-02-26T04:58:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-02-26T04:58:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-02-26T04:58:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-02-26T04:58:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-02-26T04:58:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-02-26T04:58:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-02-26T04:58:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-02-26T04:58:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-02-26T04:58:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-02-26T04:58:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-02-26T04:58:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-02-26T04:58:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-02-26T04:58:53.590 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-02-26T04:58:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-02-26T04:58:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-02-26T04:58:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-02-26T04:58:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-02-26T04:58:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-02-26T04:58:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-02-26T04:58:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-02-26T04:58:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-02-26T04:58:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-02-26T04:58:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-02-26T04:58:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-02-26T04:58:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-02-26T04:58:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-02-26T04:58:53.591 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-02-26T04:58:53.592 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-02-26T04:58:53.592 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-02-26T04:58:53.592 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-02-26T04:58:53.592 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-02-26T04:58:53.592 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-02-26T04:58:53.592 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-02-26T04:58:53.592 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-02-26T04:58:53.592 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-02-26T04:58:53.592 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-02-26T04:58:53.592 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-02-26T04:58:53.592 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-02-26T04:58:53.592 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-02-26T04:58:53.592 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-02-26T04:58:53.592 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-02-26T04:58:53.593 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-02-26T04:58:53.593 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-02-26T04:58:53.593 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-02-26T04:58:53.593 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-02-26T04:58:53.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-02-26T04:58:53.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-02-26T04:58:53.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-02-26T04:58:53.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-02-26T04:58:53.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-02-26T04:58:53.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-02-26T04:58:53.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-02-26T04:58:53.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-02-26T04:58:53.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-02-26T04:58:53.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-02-26T04:58:53.634 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-02-26T04:58:53.635 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-02-26T04:58:53.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-02-26T04:58:53.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-02-26T04:58:53.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-02-26T04:58:53.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-02-26T04:58:53.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-02-26T04:58:53.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-02-26T04:58:53.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-02-26T04:58:53.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-02-26T04:58:53.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-02-26T04:58:53.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-02-26T04:58:53.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-02-26T04:58:53.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-02-26T04:58:53.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-02-26T04:58:53.636 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-02-26T04:58:53.637 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-02-26T04:58:53.638 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-02-26T04:58:53.639 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-02-26T04:58:53.639 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-02-26T04:58:53.639 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-02-26T04:58:53.639 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-02-26T04:58:53.639 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-02-26T04:58:53.639 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-02-26T04:58:53.639 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-02-26T04:58:53.639 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-02-26T04:58:53.639 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-02-26T04:58:53.656 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-02-26T04:58:53.656 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-02-26T04:58:53.656 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-02-26T04:58:53.656 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-02-26T04:58:53.656 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-02-26T04:58:53.656 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-02-26T04:58:53.656 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-02-26T04:58:53.656 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-02-26T04:58:53.657 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-02-26T04:58:53.658 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-02-26T04:58:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-02-26T04:58:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-02-26T04:58:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-02-26T04:58:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-02-26T04:58:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-02-26T04:58:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-02-26T04:58:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-02-26T04:58:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-02-26T04:58:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-02-26T04:58:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-02-26T04:58:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-02-26T04:58:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-02-26T04:58:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-02-26T04:58:53.659 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-02-26T04:58:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-02-26T04:58:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-02-26T04:58:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-02-26T04:58:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-02-26T04:58:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-02-26T04:58:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-02-26T04:58:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-02-26T04:58:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-02-26T04:58:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-02-26T04:58:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-02-26T04:58:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-02-26T04:58:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-02-26T04:58:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-02-26T04:58:53.660 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-02-26T04:58:53.661 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-02-26T04:58:53.945 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-02-26T04:58:53.945 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-02-26T04:58:53.945 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-02-26T04:58:53.945 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-02-26T04:58:53.945 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-02-26T04:58:53.945 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-02-26T04:58:53.946 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-02-26T04:58:53.947 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-02-26T04:58:53.948 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-02-26T04:58:53.949 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-02-26T04:58:53.949 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-02-26T04:58:53.949 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-02-26T04:58:53.949 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-02-26T04:58:53.949 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-02-26T04:58:53.949 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-02-26T04:58:53.949 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-02-26T04:58:53.949 INFO:tasks.workunit.client.0.smithi060.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-02-26T04:58:53.949 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-02-26T04:58:53.949 INFO:tasks.workunit.client.0.smithi060.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-02-26T04:58:53.949 INFO:tasks.workunit.client.0.smithi060.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-02-26T04:58:53.949 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-02-26T04:58:53.950 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-02-26T04:58:53.951 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-02-26T04:58:53.951 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-02-26T04:58:53.951 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-02-26T04:58:53.951 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-02-26T04:58:54.179 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:53 smithi156 ceph-mon[35805]: pgmap v400: 129 pgs: 129 active+clean; 17 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 1.17k op/s 2024-02-26T04:58:54.284 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:53 smithi191 ceph-mon[36352]: pgmap v400: 129 pgs: 129 active+clean; 17 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 1.17k op/s 2024-02-26T04:58:54.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:53 smithi060 ceph-mon[32382]: pgmap v400: 129 pgs: 129 active+clean; 17 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 1.17k op/s 2024-02-26T04:58:55.024 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-02-26T04:58:55.024 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-02-26T04:58:55.024 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-02-26T04:58:55.025 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-02-26T04:58:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-02-26T04:58:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-02-26T04:58:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-02-26T04:58:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-02-26T04:58:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-02-26T04:58:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-02-26T04:58:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-02-26T04:58:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-02-26T04:58:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-02-26T04:58:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-02-26T04:58:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-02-26T04:58:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-02-26T04:58:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-02-26T04:58:55.026 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-02-26T04:58:55.027 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-02-26T04:58:55.028 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-02-26T04:58:55.029 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-02-26T04:58:55.030 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-02-26T04:58:55.030 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-02-26T04:58:55.030 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-02-26T04:58:55.030 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-02-26T04:58:55.030 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-02-26T04:58:55.030 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-02-26T04:58:55.030 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-02-26T04:58:55.687 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-02-26T04:58:55.687 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-02-26T04:58:55.687 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-02-26T04:58:55.687 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-02-26T04:58:55.687 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-02-26T04:58:55.687 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-02-26T04:58:55.687 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-02-26T04:58:55.688 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-02-26T04:58:55.688 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-02-26T04:58:55.688 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-02-26T04:58:55.688 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-02-26T04:58:55.688 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-02-26T04:58:55.688 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-02-26T04:58:55.688 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-02-26T04:58:55.688 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-02-26T04:58:55.688 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-02-26T04:58:55.688 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-02-26T04:58:55.688 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-02-26T04:58:55.688 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-02-26T04:58:55.688 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-02-26T04:58:55.689 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-02-26T04:58:55.690 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-02-26T04:58:55.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-02-26T04:58:55.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-02-26T04:58:55.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-02-26T04:58:55.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-02-26T04:58:55.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-02-26T04:58:55.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-02-26T04:58:55.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-02-26T04:58:55.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-02-26T04:58:55.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-02-26T04:58:55.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-02-26T04:58:55.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-02-26T04:58:55.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-02-26T04:58:55.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-02-26T04:58:55.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-02-26T04:58:55.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-02-26T04:58:55.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-02-26T04:58:55.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-02-26T04:58:56.051 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-02-26T04:58:56.051 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-02-26T04:58:56.051 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-02-26T04:58:56.051 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-02-26T04:58:56.051 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-02-26T04:58:56.051 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-02-26T04:58:56.051 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-02-26T04:58:56.051 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-02-26T04:58:56.051 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-02-26T04:58:56.051 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-02-26T04:58:56.052 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-02-26T04:58:56.052 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-02-26T04:58:56.052 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-02-26T04:58:56.052 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-02-26T04:58:56.052 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-02-26T04:58:56.052 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-02-26T04:58:56.052 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-02-26T04:58:56.052 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-02-26T04:58:56.052 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-02-26T04:58:56.052 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-02-26T04:58:56.052 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-02-26T04:58:56.053 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-02-26T04:58:56.053 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-02-26T04:58:56.053 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-02-26T04:58:56.053 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-02-26T04:58:56.053 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-02-26T04:58:56.053 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-02-26T04:58:56.053 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-02-26T04:58:56.053 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-02-26T04:58:56.053 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-02-26T04:58:56.053 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-02-26T04:58:56.053 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-02-26T04:58:56.053 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-02-26T04:58:56.054 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-02-26T04:58:56.054 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-02-26T04:58:56.054 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-02-26T04:58:56.054 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-02-26T04:58:56.054 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-02-26T04:58:56.054 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-02-26T04:58:56.054 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-02-26T04:58:56.054 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-02-26T04:58:56.054 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-02-26T04:58:56.054 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-02-26T04:58:56.054 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-02-26T04:58:56.054 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-02-26T04:58:56.054 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-02-26T04:58:56.055 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-02-26T04:58:56.055 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-02-26T04:58:56.055 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-02-26T04:58:56.055 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-02-26T04:58:56.055 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-02-26T04:58:56.055 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-02-26T04:58:56.055 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-02-26T04:58:56.055 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-02-26T04:58:56.055 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-02-26T04:58:56.055 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-02-26T04:58:56.055 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-02-26T04:58:56.055 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-02-26T04:58:56.055 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-02-26T04:58:56.056 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-02-26T04:58:56.056 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-02-26T04:58:56.056 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-02-26T04:58:56.056 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-02-26T04:58:56.056 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-02-26T04:58:56.056 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-02-26T04:58:56.056 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-02-26T04:58:56.057 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-02-26T04:58:56.057 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-02-26T04:58:56.057 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-02-26T04:58:56.057 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-02-26T04:58:56.057 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-02-26T04:58:56.057 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-02-26T04:58:56.057 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-02-26T04:58:56.057 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-02-26T04:58:56.057 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-02-26T04:58:56.058 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-02-26T04:58:56.058 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-02-26T04:58:56.058 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-02-26T04:58:56.058 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-02-26T04:58:56.058 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-02-26T04:58:56.058 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-02-26T04:58:56.058 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-02-26T04:58:56.058 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-02-26T04:58:56.058 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-02-26T04:58:56.058 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-02-26T04:58:56.058 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-02-26T04:58:56.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:55 smithi156 ceph-mon[35805]: pgmap v401: 129 pgs: 129 active+clean; 41 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.4 MiB/s wr, 1.17k op/s 2024-02-26T04:58:56.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:55 smithi060 ceph-mon[32382]: pgmap v401: 129 pgs: 129 active+clean; 41 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.4 MiB/s wr, 1.17k op/s 2024-02-26T04:58:56.258 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:55 smithi191 ceph-mon[36352]: pgmap v401: 129 pgs: 129 active+clean; 41 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.4 MiB/s wr, 1.17k op/s 2024-02-26T04:58:56.888 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-02-26T04:58:56.888 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-02-26T04:58:56.888 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-02-26T04:58:56.889 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-02-26T04:58:56.890 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-02-26T04:58:56.890 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-02-26T04:58:56.890 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-02-26T04:58:56.890 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-02-26T04:58:56.890 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-02-26T04:58:56.890 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-02-26T04:58:56.890 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-02-26T04:58:56.890 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-02-26T04:58:56.890 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-02-26T04:58:56.890 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-02-26T04:58:56.890 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-02-26T04:58:56.890 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-02-26T04:58:56.890 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-02-26T04:58:56.890 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-02-26T04:58:56.891 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-02-26T04:58:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-02-26T04:58:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-02-26T04:58:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-02-26T04:58:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-02-26T04:58:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-02-26T04:58:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-02-26T04:58:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-02-26T04:58:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-02-26T04:58:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-02-26T04:58:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-02-26T04:58:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-02-26T04:58:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-02-26T04:58:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-02-26T04:58:56.892 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-02-26T04:58:56.893 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-02-26T04:58:56.894 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-02-26T04:58:56.894 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-02-26T04:58:56.894 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-02-26T04:58:56.894 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-02-26T04:58:56.894 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-02-26T04:58:56.894 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-02-26T04:58:56.894 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-02-26T04:58:56.894 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-02-26T04:58:57.994 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-02-26T04:58:57.994 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-02-26T04:58:57.994 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-02-26T04:58:57.995 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-02-26T04:58:57.996 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-02-26T04:58:57.996 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-02-26T04:58:57.996 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-02-26T04:58:57.996 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-02-26T04:58:57.996 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-02-26T04:58:57.996 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-02-26T04:58:57.996 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-02-26T04:58:57.996 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-02-26T04:58:57.996 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-02-26T04:58:57.996 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-02-26T04:58:57.996 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-02-26T04:58:57.996 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-02-26T04:58:57.996 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-02-26T04:58:57.996 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-02-26T04:58:57.997 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-02-26T04:58:57.998 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-02-26T04:58:57.999 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-02-26T04:58:58.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:57 smithi156 ceph-mon[35805]: pgmap v402: 129 pgs: 129 active+clean; 169 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 1.23k op/s 2024-02-26T04:58:58.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:57 smithi060 ceph-mon[32382]: pgmap v402: 129 pgs: 129 active+clean; 169 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 1.23k op/s 2024-02-26T04:58:58.258 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:57 smithi191 ceph-mon[36352]: pgmap v402: 129 pgs: 129 active+clean; 169 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 1.23k op/s 2024-02-26T04:58:58.383 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-02-26T04:58:58.383 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-02-26T04:58:58.383 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-02-26T04:58:58.383 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-02-26T04:58:58.383 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-02-26T04:58:58.383 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-02-26T04:58:58.383 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-02-26T04:58:58.383 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-02-26T04:58:58.384 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-02-26T04:58:58.384 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-02-26T04:58:58.384 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-02-26T04:58:58.384 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-02-26T04:58:58.384 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-02-26T04:58:58.384 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-02-26T04:58:58.384 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-02-26T04:58:58.384 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-02-26T04:58:58.384 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-02-26T04:58:58.384 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-02-26T04:58:58.384 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-02-26T04:58:58.384 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-02-26T04:58:58.384 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-02-26T04:58:58.384 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-02-26T04:58:58.385 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-02-26T04:58:58.386 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-02-26T04:58:58.387 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-02-26T04:58:58.388 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-02-26T04:58:58.388 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-02-26T04:58:58.388 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-02-26T04:58:58.388 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-02-26T04:58:58.388 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-02-26T04:58:58.388 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-02-26T04:58:58.388 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-02-26T04:58:58.388 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-02-26T04:58:58.388 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-02-26T04:58:58.388 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-02-26T04:58:58.388 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-02-26T04:58:58.822 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-02-26T04:58:58.822 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-02-26T04:58:58.822 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-02-26T04:58:58.823 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-02-26T04:58:58.824 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-02-26T04:58:58.824 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-02-26T04:58:58.824 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-02-26T04:58:58.824 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-02-26T04:58:58.824 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-02-26T04:58:58.824 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-02-26T04:58:58.824 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-02-26T04:58:58.824 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-02-26T04:58:58.824 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-02-26T04:58:58.824 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-02-26T04:58:58.824 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-02-26T04:58:58.824 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-02-26T04:58:58.824 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-02-26T04:58:58.825 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-02-26T04:58:58.826 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-02-26T04:58:58.827 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-02-26T04:58:58.828 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-02-26T04:58:58.828 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-02-26T04:58:58.828 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-02-26T04:58:59.792 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-02-26T04:58:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-02-26T04:58:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-02-26T04:58:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-02-26T04:58:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-02-26T04:58:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-02-26T04:58:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-02-26T04:58:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-02-26T04:58:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-02-26T04:58:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-02-26T04:58:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-02-26T04:58:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-02-26T04:58:59.793 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-02-26T04:58:59.794 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-02-26T04:58:59.794 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-02-26T04:58:59.794 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-02-26T04:58:59.794 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-02-26T04:58:59.794 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-02-26T04:58:59.794 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-02-26T04:58:59.794 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-02-26T04:58:59.794 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-02-26T04:58:59.794 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-02-26T04:58:59.794 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-02-26T04:58:59.794 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-02-26T04:58:59.794 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-02-26T04:58:59.794 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-02-26T04:58:59.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-02-26T04:58:59.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-02-26T04:58:59.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-02-26T04:58:59.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-02-26T04:58:59.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-02-26T04:58:59.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-02-26T04:58:59.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-02-26T04:58:59.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-02-26T04:58:59.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-02-26T04:58:59.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-02-26T04:58:59.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-02-26T04:58:59.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-02-26T04:58:59.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-02-26T04:58:59.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-02-26T04:58:59.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-02-26T04:58:59.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-02-26T04:58:59.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-02-26T04:58:59.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-02-26T04:58:59.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-02-26T04:58:59.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-02-26T04:58:59.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-02-26T04:58:59.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-02-26T04:58:59.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-02-26T04:58:59.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-02-26T04:58:59.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-02-26T04:58:59.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-02-26T04:58:59.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-02-26T04:58:59.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-02-26T04:58:59.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-02-26T04:58:59.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-02-26T04:58:59.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-02-26T04:58:59.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-02-26T04:58:59.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-02-26T04:58:59.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-02-26T04:58:59.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-02-26T04:58:59.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-02-26T04:58:59.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-02-26T04:58:59.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-02-26T04:58:59.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-02-26T04:58:59.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-02-26T04:58:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-02-26T04:58:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-02-26T04:58:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-02-26T04:58:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-02-26T04:58:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-02-26T04:58:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-02-26T04:58:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-02-26T04:58:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-02-26T04:58:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-02-26T04:58:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-02-26T04:58:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-02-26T04:58:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-02-26T04:58:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-02-26T04:58:59.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-02-26T04:59:00.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:58:59 smithi156 ceph-mon[35805]: pgmap v403: 129 pgs: 129 active+clean; 302 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 783 KiB/s wr, 814 op/s 2024-02-26T04:59:00.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:58:59 smithi060 ceph-mon[32382]: pgmap v403: 129 pgs: 129 active+clean; 302 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 783 KiB/s wr, 814 op/s 2024-02-26T04:59:00.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:58:59 smithi191 ceph-mon[36352]: pgmap v403: 129 pgs: 129 active+clean; 302 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 783 KiB/s wr, 814 op/s 2024-02-26T04:59:00.634 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-02-26T04:59:00.635 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-02-26T04:59:00.635 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-02-26T04:59:00.635 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-02-26T04:59:00.635 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-02-26T04:59:00.635 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-02-26T04:59:00.635 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-02-26T04:59:00.635 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-02-26T04:59:00.635 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-02-26T04:59:00.635 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-02-26T04:59:00.635 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-02-26T04:59:00.635 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-02-26T04:59:00.635 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-02-26T04:59:00.635 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-02-26T04:59:00.636 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-02-26T04:59:00.636 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-02-26T04:59:00.636 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-02-26T04:59:00.636 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-02-26T04:59:00.636 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-02-26T04:59:00.636 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-02-26T04:59:00.636 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-02-26T04:59:00.636 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-02-26T04:59:00.636 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-02-26T04:59:00.636 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-02-26T04:59:00.636 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-02-26T04:59:00.636 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-02-26T04:59:00.636 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-02-26T04:59:00.637 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-02-26T04:59:00.638 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-02-26T04:59:00.638 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-02-26T04:59:00.638 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-02-26T04:59:00.638 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-02-26T04:59:00.638 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-02-26T04:59:00.638 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-02-26T04:59:00.638 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-02-26T04:59:00.638 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-02-26T04:59:00.638 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-02-26T04:59:00.638 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-02-26T04:59:00.638 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-02-26T04:59:00.638 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-02-26T04:59:00.638 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-02-26T04:59:00.639 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-02-26T04:59:00.639 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-02-26T04:59:00.639 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-02-26T04:59:00.639 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-02-26T04:59:00.639 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-02-26T04:59:00.639 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-02-26T04:59:00.639 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-02-26T04:59:00.639 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-02-26T04:59:00.639 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-02-26T04:59:00.639 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-02-26T04:59:00.639 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-02-26T04:59:00.640 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-02-26T04:59:00.640 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-02-26T04:59:00.640 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-02-26T04:59:00.640 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-02-26T04:59:00.640 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-02-26T04:59:00.640 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-02-26T04:59:00.640 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-02-26T04:59:00.640 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-02-26T04:59:00.640 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-02-26T04:59:00.640 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-02-26T04:59:00.640 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-02-26T04:59:00.640 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-02-26T04:59:00.640 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-02-26T04:59:00.640 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-02-26T04:59:01.039 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-02-26T04:59:01.039 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-02-26T04:59:01.039 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-02-26T04:59:01.039 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-02-26T04:59:01.039 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-02-26T04:59:01.039 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-02-26T04:59:01.039 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-02-26T04:59:01.039 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-02-26T04:59:01.040 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-02-26T04:59:01.041 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-02-26T04:59:01.041 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-02-26T04:59:01.041 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-02-26T04:59:01.041 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-02-26T04:59:01.041 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-02-26T04:59:01.041 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-02-26T04:59:01.041 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-02-26T04:59:01.041 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-02-26T04:59:01.041 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-02-26T04:59:01.041 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-02-26T04:59:01.041 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-02-26T04:59:01.041 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-02-26T04:59:01.041 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-02-26T04:59:01.041 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-02-26T04:59:01.042 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-02-26T04:59:01.043 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-02-26T04:59:01.044 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-02-26T04:59:01.044 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-02-26T04:59:01.044 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-02-26T04:59:01.044 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-02-26T04:59:01.044 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-02-26T04:59:01.044 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-02-26T04:59:01.044 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-02-26T04:59:01.044 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-02-26T04:59:01.044 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-02-26T04:59:01.044 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-02-26T04:59:01.044 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-02-26T04:59:01.044 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-02-26T04:59:01.044 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-02-26T04:59:01.618 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-02-26T04:59:01.618 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-02-26T04:59:01.618 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-02-26T04:59:01.619 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-02-26T04:59:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-02-26T04:59:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-02-26T04:59:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-02-26T04:59:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-02-26T04:59:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-02-26T04:59:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-02-26T04:59:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-02-26T04:59:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-02-26T04:59:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-02-26T04:59:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-02-26T04:59:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-02-26T04:59:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-02-26T04:59:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-02-26T04:59:01.620 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-02-26T04:59:01.621 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-02-26T04:59:01.622 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-02-26T04:59:01.623 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-02-26T04:59:01.623 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-02-26T04:59:01.623 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-02-26T04:59:01.623 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-02-26T04:59:01.623 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-02-26T04:59:01.623 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-02-26T04:59:01.623 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-02-26T04:59:01.623 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-02-26T04:59:01.623 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-02-26T04:59:01.623 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-02-26T04:59:01.623 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-02-26T04:59:01.623 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-02-26T04:59:01.623 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-02-26T04:59:01.624 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-02-26T04:59:01.624 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-02-26T04:59:01.624 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-02-26T04:59:01.624 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-02-26T04:59:01.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T04:59:01.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T04:59:01.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.935+0000 7f29f8108640 1 -- 172.21.15.60:0/3828068083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29f0106ed0 msgr2=0x7f29f01092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:01.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.935+0000 7f29f8108640 1 --2- 172.21.15.60:0/3828068083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29f0106ed0 0x7f29f01092c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009ca0 tx=0x7f29e802c0f0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.935+0000 7f29f8108640 1 -- 172.21.15.60:0/3828068083 shutdown_connections 2024-02-26T04:59:01.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.935+0000 7f29f8108640 1 --2- 172.21.15.60:0/3828068083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29f0106ed0 0x7f29f01092c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.935+0000 7f29f8108640 1 --2- 172.21.15.60:0/3828068083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29f01045a0 0x7f29f0106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.935+0000 7f29f8108640 1 --2- 172.21.15.60:0/3828068083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29f0101c70 0x7f29f0104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.935+0000 7f29f8108640 1 -- 172.21.15.60:0/3828068083 >> 172.21.15.60:0/3828068083 conn(0x7f29f00fb620 msgr2=0x7f29f00fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:01.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.936+0000 7f29f8108640 1 -- 172.21.15.60:0/3828068083 shutdown_connections 2024-02-26T04:59:01.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.936+0000 7f29f8108640 1 -- 172.21.15.60:0/3828068083 wait complete. 2024-02-26T04:59:01.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.936+0000 7f29f8108640 1 Processor -- start 2024-02-26T04:59:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.936+0000 7f29f8108640 1 -- start start 2024-02-26T04:59:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.936+0000 7f29f8108640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29f0101c70 0x7f29f01015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.936+0000 7f29f8108640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29f01045a0 0x7f29f00ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.936+0000 7f29f8108640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29f0106ed0 0x7f29f0100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.936+0000 7f29f8108640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f01008c0 con 0x7f29f01045a0 2024-02-26T04:59:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.936+0000 7f29f8108640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f0100a30 con 0x7f29f0101c70 2024-02-26T04:59:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.936+0000 7f29f8108640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f01a1220 con 0x7f29f0106ed0 2024-02-26T04:59:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29f56a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29f01045a0 0x7f29f00ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29f5ea4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29f0101c70 0x7f29f01015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29f56a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29f01045a0 0x7f29f00ffc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33712/0 (socket says 172.21.15.60:33712) 2024-02-26T04:59:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29f56a3640 1 -- 172.21.15.60:0/370385820 learned_addr learned my addr 172.21.15.60:0/370385820 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29f66a5640 1 --2- 172.21.15.60:0/370385820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29f0106ed0 0x7f29f0100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29f56a3640 1 -- 172.21.15.60:0/370385820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29f0106ed0 msgr2=0x7f29f0100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29f56a3640 1 --2- 172.21.15.60:0/370385820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29f0106ed0 0x7f29f0100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29f56a3640 1 -- 172.21.15.60:0/370385820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29f0101c70 msgr2=0x7f29f01015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29f56a3640 1 --2- 172.21.15.60:0/370385820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29f0101c70 0x7f29f01015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29f56a3640 1 -- 172.21.15.60:0/370385820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e8009900 con 0x7f29f01045a0 2024-02-26T04:59:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29f5ea4640 1 --2- 172.21.15.60:0/370385820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29f0101c70 0x7f29f01015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:59:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29f56a3640 1 --2- 172.21.15.60:0/370385820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29f01045a0 0x7f29f00ffc20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f29e000e9f0 tx=0x7f29e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29e6ffd640 1 -- 172.21.15.60:0/370385820 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e0009700 con 0x7f29f01045a0 2024-02-26T04:59:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29f8108640 1 -- 172.21.15.60:0/370385820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f01a14d0 con 0x7f29f01045a0 2024-02-26T04:59:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29e6ffd640 1 -- 172.21.15.60:0/370385820 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29e000cd80 con 0x7f29f01045a0 2024-02-26T04:59:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29e6ffd640 1 -- 172.21.15.60:0/370385820 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e0010690 con 0x7f29f01045a0 2024-02-26T04:59:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.937+0000 7f29f8108640 1 -- 172.21.15.60:0/370385820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f01a1a30 con 0x7f29f01045a0 2024-02-26T04:59:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.940+0000 7f29e6ffd640 1 -- 172.21.15.60:0/370385820 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f29e00046c0 con 0x7f29f01045a0 2024-02-26T04:59:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.940+0000 7f29fafbb740 1 Processor -- start 2024-02-26T04:59:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.940+0000 7f29e6ffd640 1 --2- 172.21.15.60:0/370385820 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f29dc0a20c0 0x7f29dc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.940+0000 7f29e6ffd640 1 -- 172.21.15.60:0/370385820 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f29e0014070 con 0x7f29f01045a0 2024-02-26T04:59:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.940+0000 7f29f66a5640 1 --2- 172.21.15.60:0/370385820 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f29dc0a20c0 0x7f29dc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.940+0000 7f29fafbb740 1 -- start start 2024-02-26T04:59:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29fafbb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7f62a0 0x5558af7f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29fafbb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af7fb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29fafbb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7fbe00 0x5558af7fe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29fafbb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558af766250 con 0x5558af7f6c90 2024-02-26T04:59:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29fafbb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558af763780 con 0x5558af7f62a0 2024-02-26T04:59:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29fafbb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558af75ef60 con 0x5558af7fbe00 2024-02-26T04:59:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29f56a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af7fb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29f56a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af7fb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33720/0 (socket says 172.21.15.60:33720) 2024-02-26T04:59:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29f56a3640 1 -- 172.21.15.60:0/1972395765 learned_addr learned my addr 172.21.15.60:0/1972395765 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29f66a5640 1 --2- 172.21.15.60:0/370385820 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f29dc0a20c0 0x7f29dc0a4580 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f29e8005e30 tx=0x7f29e80326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29f5ea4640 1 --2- 172.21.15.60:0/1972395765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7f62a0 0x5558af7f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29f66a5640 1 --2- 172.21.15.60:0/1972395765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7fbe00 0x5558af7fe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29f56a3640 1 -- 172.21.15.60:0/1972395765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7fbe00 msgr2=0x5558af7fe290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29f56a3640 1 --2- 172.21.15.60:0/1972395765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7fbe00 0x5558af7fe290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29f56a3640 1 -- 172.21.15.60:0/1972395765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7f62a0 msgr2=0x5558af7f66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29f56a3640 1 --2- 172.21.15.60:0/1972395765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7f62a0 0x5558af7f66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29f56a3640 1 -- 172.21.15.60:0/1972395765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558af71d540 con 0x5558af7f6c90 2024-02-26T04:59:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29f5ea4640 1 --2- 172.21.15.60:0/1972395765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7f62a0 0x5558af7f66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:59:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29f56a3640 1 --2- 172.21.15.60:0/1972395765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af7fb790 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f29e00a2f20 tx=0x7f29e009a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777be50b7006daa1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29dbfff640 1 -- 172.21.15.60:0/1972395765 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e009ee00 con 0x5558af7f6c90 2024-02-26T04:59:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29dbfff640 1 -- 172.21.15.60:0/1972395765 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29e00a3450 con 0x5558af7f6c90 2024-02-26T04:59:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.941+0000 7f29dbfff640 1 -- 172.21.15.60:0/1972395765 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e00a3220 con 0x5558af7f6c90 2024-02-26T04:59:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.942+0000 7f29fafbb740 1 -- 172.21.15.60:0/1972395765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 msgr2=0x5558af7fb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.942+0000 7f29fafbb740 1 --2- 172.21.15.60:0/1972395765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af7fb790 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f29e00a2f20 tx=0x7f29e009a6d0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.942+0000 7f29fafbb740 1 -- 172.21.15.60:0/1972395765 shutdown_connections 2024-02-26T04:59:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.942+0000 7f29fafbb740 1 --2- 172.21.15.60:0/1972395765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7fbe00 0x5558af7fe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.942+0000 7f29fafbb740 1 --2- 172.21.15.60:0/1972395765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af7fb790 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.942+0000 7f29fafbb740 1 --2- 172.21.15.60:0/1972395765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7f62a0 0x5558af7f66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.942+0000 7f29fafbb740 1 -- 172.21.15.60:0/1972395765 >> 172.21.15.60:0/1972395765 conn(0x5558af7f17c0 msgr2=0x5558af7f1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.942+0000 7f29fafbb740 1 -- 172.21.15.60:0/1972395765 shutdown_connections 2024-02-26T04:59:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.942+0000 7f29fafbb740 1 -- 172.21.15.60:0/1972395765 wait complete. 2024-02-26T04:59:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.942+0000 7f29fafbb740 1 -- start start 2024-02-26T04:59:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29fafbb740 1 Processor -- start 2024-02-26T04:59:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29fafbb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7f62a0 0x5558af800270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29fafbb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af8007b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29fafbb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7fbe00 0x5558af8077f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29fafbb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558af432430 con 0x5558af7f6c90 2024-02-26T04:59:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29fafbb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558af3eee00 con 0x5558af7fbe00 2024-02-26T04:59:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29fafbb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558af6e3fa0 con 0x5558af7f62a0 2024-02-26T04:59:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29fafbb740 10 client.0 _ll_register_callbacks cb 0x5558af739fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29f56a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af8007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29f56a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af8007b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33734/0 (socket says 172.21.15.60:33734) 2024-02-26T04:59:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29f56a3640 1 -- 172.21.15.60:0/3765266238 learned_addr learned my addr 172.21.15.60:0/3765266238 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29f5ea4640 1 --2- 172.21.15.60:0/3765266238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7f62a0 0x5558af800270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29f66a5640 1 --2- 172.21.15.60:0/3765266238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7fbe00 0x5558af8077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29f56a3640 1 -- 172.21.15.60:0/3765266238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7f62a0 msgr2=0x5558af800270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29f56a3640 1 --2- 172.21.15.60:0/3765266238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7f62a0 0x5558af800270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29f56a3640 1 -- 172.21.15.60:0/3765266238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7fbe00 msgr2=0x5558af8077f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29f56a3640 1 --2- 172.21.15.60:0/3765266238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7fbe00 0x5558af8077f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29f56a3640 1 -- 172.21.15.60:0/3765266238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f01a1a30 con 0x5558af7f6c90 2024-02-26T04:59:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.943+0000 7f29f66a5640 1 --2- 172.21.15.60:0/3765266238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7fbe00 0x5558af8077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:59:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29f56a3640 1 --2- 172.21.15.60:0/3765266238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af8007b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f29e0099200 tx=0x7f29e0004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa3018dbceb4f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29d97fa640 1 -- 172.21.15.60:0/3765266238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e00b7070 con 0x5558af7f6c90 2024-02-26T04:59:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29d97fa640 1 -- 172.21.15.60:0/3765266238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f29e0097470 con 0x5558af7f6c90 2024-02-26T04:59:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29fafbb740 1 -- 172.21.15.60:0/3765266238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558af45e7e0 con 0x5558af7f6c90 2024-02-26T04:59:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29d97fa640 1 -- 172.21.15.60:0/3765266238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29e009ee00 con 0x5558af7f6c90 2024-02-26T04:59:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29fafbb740 10 client.15723 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29fafbb740 1 -- 172.21.15.60:0/3765266238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5558af75d5e0 con 0x5558af7f6c90 2024-02-26T04:59:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29d97fa640 1 -- 172.21.15.60:0/3765266238 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5558af45e7e0 con 0x5558af7f6c90 2024-02-26T04:59:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29d97fa640 1 -- 172.21.15.60:0/3765266238 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f29e001f060 con 0x5558af7f6c90 2024-02-26T04:59:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29fafbb740 10 client.15723 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29fafbb740 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:59:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29fafbb740 20 client.15723 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29fafbb740 1 --2- 172.21.15.60:0/3765266238 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558af80c020 0x5558af80e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29fafbb740 4 client.15723 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:59:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.944+0000 7f29fafbb740 1 -- 172.21.15.60:0/3765266238 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558af49b720 con 0x5558af80c020 2024-02-26T04:59:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.945+0000 7f29f5ea4640 1 --2- 172.21.15.60:0/3765266238 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558af80c020 0x5558af80e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.945+0000 7f29d97fa640 10 client.15723 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.945+0000 7f29f5ea4640 1 --2- 172.21.15.60:0/3765266238 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558af80c020 0x5558af80e410 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1724fa941cea842c server_cookie=c35246e69013caf8 in_seq=0 out_seq=0 2024-02-26T04:59:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.945+0000 7f29d97fa640 1 -- 172.21.15.60:0/3765266238 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5558af49b720 con 0x5558af80c020 2024-02-26T04:59:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.945+0000 7f29d97fa640 10 client.15723 handle_command_reply: tid=0 2024-02-26T04:59:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.945+0000 7f29fafbb740 1 client.15723 shutdown 2024-02-26T04:59:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.946+0000 7f29fafbb740 1 -- 172.21.15.60:0/3765266238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 msgr2=0x5558af8007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.946+0000 7f29fafbb740 1 --2- 172.21.15.60:0/3765266238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af8007b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f29e0099200 tx=0x7f29e0004a30 comp rx=0 tx=0).stop 2024-02-26T04:59:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.946+0000 7f29fafbb740 1 -- 172.21.15.60:0/3765266238 shutdown_connections 2024-02-26T04:59:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.946+0000 7f29fafbb740 1 --2- 172.21.15.60:0/3765266238 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558af80c020 0x5558af80e410 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.946+0000 7f29fafbb740 1 --2- 172.21.15.60:0/3765266238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7fbe00 0x5558af8077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.946+0000 7f29fafbb740 1 --2- 172.21.15.60:0/3765266238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af8007b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.946+0000 7f29d97fa640 0 client.15723 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.946+0000 7f29fafbb740 1 --2- 172.21.15.60:0/3765266238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7f62a0 0x5558af800270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.946+0000 7f29fafbb740 1 -- 172.21.15.60:0/3765266238 >> 172.21.15.60:0/3765266238 conn(0x5558af7f17c0 msgr2=0x5558af7f0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:01.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.946+0000 7f29fafbb740 1 -- 172.21.15.60:0/3765266238 shutdown_connections 2024-02-26T04:59:01.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.946+0000 7f29fafbb740 1 -- 172.21.15.60:0/3765266238 wait complete. 2024-02-26T04:59:01.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.946+0000 7f29fafbb740 20 client.15723 trim_cache size 0 max 16384 2024-02-26T04:59:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.963+0000 7f29fafbb740 1 Processor -- start 2024-02-26T04:59:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.963+0000 7f29fafbb740 1 -- start start 2024-02-26T04:59:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.963+0000 7f29fafbb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7f62a0 0x5558af800270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.963+0000 7f29fafbb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af8007b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.963+0000 7f29fafbb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7fbe00 0x5558af8077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.963+0000 7f29fafbb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558af49b720 con 0x5558af7f6c90 2024-02-26T04:59:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.963+0000 7f29fafbb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558af45e7e0 con 0x5558af7f62a0 2024-02-26T04:59:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.963+0000 7f29fafbb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558af808ce0 con 0x5558af7fbe00 2024-02-26T04:59:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.963+0000 7f29f56a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af8007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.963+0000 7f29f5ea4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7f62a0 0x5558af800270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.963+0000 7f29f56a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af8007b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33748/0 (socket says 172.21.15.60:33748) 2024-02-26T04:59:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.964+0000 7f29f56a3640 1 -- 172.21.15.60:0/4037551964 learned_addr learned my addr 172.21.15.60:0/4037551964 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.964+0000 7f29f66a5640 1 --2- 172.21.15.60:0/4037551964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7fbe00 0x5558af8077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.964+0000 7f29f5ea4640 1 -- 172.21.15.60:0/4037551964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7fbe00 msgr2=0x5558af8077b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.964+0000 7f29f5ea4640 1 --2- 172.21.15.60:0/4037551964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7fbe00 0x5558af8077b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.964+0000 7f29f5ea4640 1 -- 172.21.15.60:0/4037551964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 msgr2=0x5558af8007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.964+0000 7f29f5ea4640 1 --2- 172.21.15.60:0/4037551964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af8007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.964+0000 7f29f5ea4640 1 -- 172.21.15.60:0/4037551964 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558af6f9a90 con 0x5558af7f62a0 2024-02-26T04:59:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.964+0000 7f29f66a5640 1 --2- 172.21.15.60:0/4037551964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7fbe00 0x5558af8077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:59:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.964+0000 7f29f5ea4640 1 --2- 172.21.15.60:0/4037551964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7f62a0 0x5558af800270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5558af807db0 tx=0x7f29ec00b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d28d1a0313de2f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.964+0000 7f29db7fe640 1 -- 172.21.15.60:0/4037551964 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec016040 con 0x5558af7f62a0 2024-02-26T04:59:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.964+0000 7f29db7fe640 1 -- 172.21.15.60:0/4037551964 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29ec017040 con 0x5558af7f62a0 2024-02-26T04:59:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.964+0000 7f29db7fe640 1 -- 172.21.15.60:0/4037551964 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec004a00 con 0x5558af7f62a0 2024-02-26T04:59:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.965+0000 7f29fafbb740 1 -- 172.21.15.60:0/4037551964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7f62a0 msgr2=0x5558af800270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.965+0000 7f29fafbb740 1 --2- 172.21.15.60:0/4037551964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7f62a0 0x5558af800270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5558af807db0 tx=0x7f29ec00b390 comp rx=0 tx=0).stop 2024-02-26T04:59:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.965+0000 7f29fafbb740 1 -- 172.21.15.60:0/4037551964 shutdown_connections 2024-02-26T04:59:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.965+0000 7f29fafbb740 1 --2- 172.21.15.60:0/4037551964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7fbe00 0x5558af8077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.965+0000 7f29fafbb740 1 --2- 172.21.15.60:0/4037551964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af8007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.965+0000 7f29fafbb740 1 --2- 172.21.15.60:0/4037551964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7f62a0 0x5558af800270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.965+0000 7f29fafbb740 1 -- 172.21.15.60:0/4037551964 >> 172.21.15.60:0/4037551964 conn(0x5558af7f17c0 msgr2=0x5558af812d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.965+0000 7f29fafbb740 1 -- 172.21.15.60:0/4037551964 shutdown_connections 2024-02-26T04:59:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.965+0000 7f29fafbb740 1 -- 172.21.15.60:0/4037551964 wait complete. 2024-02-26T04:59:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.965+0000 7f29fafbb740 1 -- start start 2024-02-26T04:59:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.965+0000 7f29fafbb740 1 Processor -- start 2024-02-26T04:59:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29fafbb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7f62a0 0x5558af807330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29fafbb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af807870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29fafbb740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7fbe00 0x5558af805a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29fafbb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558af806130 con 0x5558af7f6c90 2024-02-26T04:59:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29fafbb740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558af8062a0 con 0x5558af7fbe00 2024-02-26T04:59:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29fafbb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558af806460 con 0x5558af7f62a0 2024-02-26T04:59:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29fafbb740 10 client.0 _ll_register_callbacks cb 0x7f29e0096f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29f56a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af807870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29f56a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af807870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33764/0 (socket says 172.21.15.60:33764) 2024-02-26T04:59:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29f56a3640 1 -- 172.21.15.60:0/271009331 learned_addr learned my addr 172.21.15.60:0/271009331 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29f66a5640 1 --2- 172.21.15.60:0/271009331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7fbe00 0x5558af805a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29f5ea4640 1 --2- 172.21.15.60:0/271009331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7f62a0 0x5558af807330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29f56a3640 1 -- 172.21.15.60:0/271009331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7f62a0 msgr2=0x5558af807330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29f56a3640 1 --2- 172.21.15.60:0/271009331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7f62a0 0x5558af807330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29f56a3640 1 -- 172.21.15.60:0/271009331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7fbe00 msgr2=0x5558af805a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29f56a3640 1 --2- 172.21.15.60:0/271009331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7fbe00 0x5558af805a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29f56a3640 1 -- 172.21.15.60:0/271009331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec00ba40 con 0x5558af7f6c90 2024-02-26T04:59:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.966+0000 7f29f66a5640 1 --2- 172.21.15.60:0/271009331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7fbe00 0x5558af805a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:59:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29f56a3640 1 --2- 172.21.15.60:0/271009331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af807870 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f29e0098f60 tx=0x7f29e0004b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e995eccb3ea50b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29d97fa640 1 -- 172.21.15.60:0/271009331 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e00b7060 con 0x5558af7f6c90 2024-02-26T04:59:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29fafbb740 1 -- 172.21.15.60:0/271009331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29e00a3450 con 0x5558af7f6c90 2024-02-26T04:59:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29d97fa640 1 -- 172.21.15.60:0/271009331 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f29e00a00a0 con 0x5558af7f6c90 2024-02-26T04:59:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29d97fa640 1 -- 172.21.15.60:0/271009331 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29e009a7f0 con 0x5558af7f6c90 2024-02-26T04:59:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29fafbb740 10 client.15729 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29fafbb740 1 -- 172.21.15.60:0/271009331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5558af71f670 con 0x5558af7f6c90 2024-02-26T04:59:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29d97fa640 1 -- 172.21.15.60:0/271009331 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29e00a3450 con 0x5558af7f6c90 2024-02-26T04:59:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29d97fa640 1 -- 172.21.15.60:0/271009331 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f29e001f070 con 0x5558af7f6c90 2024-02-26T04:59:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29fafbb740 10 client.15729 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29fafbb740 10 client.15729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:59:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29fafbb740 20 client.15729 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29fafbb740 1 --2- 172.21.15.60:0/271009331 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558af863450 0x5558af865870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29fafbb740 4 client.15729 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T04:59:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29fafbb740 1 -- 172.21.15.60:0/271009331 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5558af4ea890 con 0x5558af863450 2024-02-26T04:59:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.967+0000 7f29f5ea4640 1 --2- 172.21.15.60:0/271009331 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558af863450 0x5558af865870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29d97fa640 10 client.15729 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29f5ea4640 1 --2- 172.21.15.60:0/271009331 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558af863450 0x5558af865870 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef58937f834b6f7 server_cookie=ae52d42f46882296 in_seq=0 out_seq=0 2024-02-26T04:59:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29d97fa640 1 -- 172.21.15.60:0/271009331 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5558af4ea890 con 0x5558af863450 2024-02-26T04:59:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29d97fa640 10 client.15729 handle_command_reply: tid=0 2024-02-26T04:59:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29fafbb740 1 client.15729 shutdown 2024-02-26T04:59:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29fafbb740 1 -- 172.21.15.60:0/271009331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 msgr2=0x5558af807870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29fafbb740 1 --2- 172.21.15.60:0/271009331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af807870 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f29e0098f60 tx=0x7f29e0004b30 comp rx=0 tx=0).stop 2024-02-26T04:59:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29fafbb740 1 -- 172.21.15.60:0/271009331 shutdown_connections 2024-02-26T04:59:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29fafbb740 1 --2- 172.21.15.60:0/271009331 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5558af863450 0x5558af865870 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29fafbb740 1 --2- 172.21.15.60:0/271009331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558af7fbe00 0x5558af805a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29fafbb740 1 --2- 172.21.15.60:0/271009331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558af7f6c90 0x5558af807870 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29fafbb740 1 --2- 172.21.15.60:0/271009331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5558af7f62a0 0x5558af807330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29d97fa640 0 client.15729 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29fafbb740 1 -- 172.21.15.60:0/271009331 >> 172.21.15.60:0/271009331 conn(0x5558af7f17c0 msgr2=0x5558af7f7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29fafbb740 1 -- 172.21.15.60:0/271009331 shutdown_connections 2024-02-26T04:59:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29fafbb740 1 -- 172.21.15.60:0/271009331 wait complete. 2024-02-26T04:59:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.968+0000 7f29fafbb740 20 client.15729 trim_cache size 0 max 16384 2024-02-26T04:59:01.978 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:59:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-02-26T04:59:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-02-26T04:59:01.978 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:59:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.969+0000 7f29f8108640 1 -- 172.21.15.60:0/370385820 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f29dc0a20c0 msgr2=0x7f29dc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.970+0000 7f29f8108640 1 --2- 172.21.15.60:0/370385820 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f29dc0a20c0 0x7f29dc0a4580 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f29e8005e30 tx=0x7f29e80326b0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.970+0000 7f29f8108640 1 -- 172.21.15.60:0/370385820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29f01045a0 msgr2=0x7f29f00ffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.970+0000 7f29f8108640 1 --2- 172.21.15.60:0/370385820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29f01045a0 0x7f29f00ffc20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f29e000e9f0 tx=0x7f29e000eeb0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.970+0000 7f29f8108640 1 -- 172.21.15.60:0/370385820 shutdown_connections 2024-02-26T04:59:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.970+0000 7f29f8108640 1 --2- 172.21.15.60:0/370385820 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f29dc0a20c0 0x7f29dc0a4580 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.970+0000 7f29f8108640 1 --2- 172.21.15.60:0/370385820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29f0106ed0 0x7f29f0100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.970+0000 7f29f8108640 1 --2- 172.21.15.60:0/370385820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29f01045a0 0x7f29f00ffc20 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.970+0000 7f29f8108640 1 --2- 172.21.15.60:0/370385820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29f0101c70 0x7f29f01015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.970+0000 7f29f8108640 1 -- 172.21.15.60:0/370385820 >> 172.21.15.60:0/370385820 conn(0x7f29f00fb620 msgr2=0x7f29f01025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.970+0000 7f29f8108640 1 -- 172.21.15.60:0/370385820 shutdown_connections 2024-02-26T04:59:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:01.970+0000 7f29f8108640 1 -- 172.21.15.60:0/370385820 wait complete. 2024-02-26T04:59:01.985 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T04:59:01.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:59:02.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.096+0000 7f169bc77640 1 -- 172.21.15.60:0/632143083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16941013c0 msgr2=0x7f1694107c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.096+0000 7f169bc77640 1 --2- 172.21.15.60:0/632143083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16941013c0 0x7f1694107c50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f168c009ca0 tx=0x7f168c02c0f0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.096+0000 7f169bc77640 1 -- 172.21.15.60:0/632143083 shutdown_connections 2024-02-26T04:59:02.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.096+0000 7f169bc77640 1 --2- 172.21.15.60:0/632143083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16941013c0 0x7f1694107c50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.096+0000 7f169bc77640 1 --2- 172.21.15.60:0/632143083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1694100a00 0x7f1694100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.096+0000 7f169bc77640 1 --2- 172.21.15.60:0/632143083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16940ff810 0x7f16940ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.096+0000 7f169bc77640 1 -- 172.21.15.60:0/632143083 >> 172.21.15.60:0/632143083 conn(0x7f16940fad80 msgr2=0x7f16940fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.096+0000 7f169bc77640 1 -- 172.21.15.60:0/632143083 shutdown_connections 2024-02-26T04:59:02.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.096+0000 7f169bc77640 1 -- 172.21.15.60:0/632143083 wait complete. 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.096+0000 7f169bc77640 1 Processor -- start 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.096+0000 7f169bc77640 1 -- start start 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f169bc77640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16940ff810 0x7f16941037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f169bc77640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1694100a00 0x7f1694101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f169bc77640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16941013c0 0x7f1694102320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f169bc77640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1694102a80 con 0x7f16940ff810 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f169bc77640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1694102bf0 con 0x7f1694100a00 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f169bc77640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16941a1240 con 0x7f16941013c0 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f1699a13640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16940ff810 0x7f16941037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f1699a13640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16940ff810 0x7f16941037a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33776/0 (socket says 172.21.15.60:33776) 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f1699a13640 1 -- 172.21.15.60:0/2465249621 learned_addr learned my addr 172.21.15.60:0/2465249621 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f1699212640 1 --2- 172.21.15.60:0/2465249621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1694100a00 0x7f1694101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f169a214640 1 --2- 172.21.15.60:0/2465249621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16941013c0 0x7f1694102320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f1699a13640 1 -- 172.21.15.60:0/2465249621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16941013c0 msgr2=0x7f1694102320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f1699a13640 1 --2- 172.21.15.60:0/2465249621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16941013c0 0x7f1694102320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f1699a13640 1 -- 172.21.15.60:0/2465249621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1694100a00 msgr2=0x7f1694101de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f1699a13640 1 --2- 172.21.15.60:0/2465249621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1694100a00 0x7f1694101de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f1699a13640 1 -- 172.21.15.60:0/2465249621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1684009490 con 0x7f16940ff810 2024-02-26T04:59:02.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f1699a13640 1 --2- 172.21.15.60:0/2465249621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16940ff810 0x7f16941037a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f169000e9f0 tx=0x7f169000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f168affd640 1 -- 172.21.15.60:0/2465249621 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16900096e0 con 0x7f16940ff810 2024-02-26T04:59:02.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f168affd640 1 -- 172.21.15.60:0/2465249621 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f169000cd80 con 0x7f16940ff810 2024-02-26T04:59:02.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f169bc77640 1 -- 172.21.15.60:0/2465249621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168c009900 con 0x7f16940ff810 2024-02-26T04:59:02.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f168affd640 1 -- 172.21.15.60:0/2465249621 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690010430 con 0x7f16940ff810 2024-02-26T04:59:02.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.097+0000 7f169bc77640 1 -- 172.21.15.60:0/2465249621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16941a1820 con 0x7f16940ff810 2024-02-26T04:59:02.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.098+0000 7f169bc77640 1 -- 172.21.15.60:0/2465249621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169410bac0 con 0x7f16940ff810 2024-02-26T04:59:02.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.099+0000 7f168affd640 1 -- 172.21.15.60:0/2465249621 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1690003680 con 0x7f16940ff810 2024-02-26T04:59:02.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.100+0000 7f168affd640 1 --2- 172.21.15.60:0/2465249621 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f16800a20c0 0x7f16800a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.100+0000 7f1699212640 1 --2- 172.21.15.60:0/2465249621 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f16800a20c0 0x7f16800a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.100+0000 7f168affd640 1 -- 172.21.15.60:0/2465249621 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f1690014070 con 0x7f16940ff810 2024-02-26T04:59:02.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.101+0000 7f1699212640 1 --2- 172.21.15.60:0/2465249621 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f16800a20c0 0x7f16800a4580 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1694103510 tx=0x7f1684005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.105+0000 7f168affd640 1 -- 172.21.15.60:0/2465249621 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1690096fc0 con 0x7f16940ff810 2024-02-26T04:59:02.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:01 smithi156 ceph-mon[35805]: pgmap v404: 129 pgs: 129 active+clean; 297 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 1.0 MiB/s wr, 867 op/s 2024-02-26T04:59:02.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:01 smithi060 ceph-mon[32382]: pgmap v404: 129 pgs: 129 active+clean; 297 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 1.0 MiB/s wr, 867 op/s 2024-02-26T04:59:02.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:01 smithi191 ceph-mon[36352]: pgmap v404: 129 pgs: 129 active+clean; 297 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 1.0 MiB/s wr, 867 op/s 2024-02-26T04:59:02.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.349+0000 7f169bc77640 1 -- 172.21.15.60:0/2465249621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f169410bca0 con 0x7f16940ff810 2024-02-26T04:59:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.350+0000 7f168affd640 1 -- 172.21.15.60:0/2465249621 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11347 (secure 0 0 0) 0x7f169009a680 con 0x7f16940ff810 2024-02-26T04:59:02.353 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:59:02.353 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:57:52.162616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24476,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T04:59:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-02-26T04:59:02.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.352+0000 7f169bc77640 1 -- 172.21.15.60:0/2465249621 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f16800a20c0 msgr2=0x7f16800a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.352+0000 7f169bc77640 1 --2- 172.21.15.60:0/2465249621 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f16800a20c0 0x7f16800a4580 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1694103510 tx=0x7f1684005f90 comp rx=0 tx=0).stop 2024-02-26T04:59:02.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.352+0000 7f169bc77640 1 -- 172.21.15.60:0/2465249621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16940ff810 msgr2=0x7f16941037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.352+0000 7f169bc77640 1 --2- 172.21.15.60:0/2465249621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16940ff810 0x7f16941037a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f169000e9f0 tx=0x7f169000eeb0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.352+0000 7f169bc77640 1 -- 172.21.15.60:0/2465249621 shutdown_connections 2024-02-26T04:59:02.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.352+0000 7f169bc77640 1 --2- 172.21.15.60:0/2465249621 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f16800a20c0 0x7f16800a4580 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.352+0000 7f169bc77640 1 --2- 172.21.15.60:0/2465249621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16941013c0 0x7f1694102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.352+0000 7f169bc77640 1 --2- 172.21.15.60:0/2465249621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1694100a00 0x7f1694101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.352+0000 7f169bc77640 1 --2- 172.21.15.60:0/2465249621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16940ff810 0x7f16941037a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.352+0000 7f169bc77640 1 -- 172.21.15.60:0/2465249621 >> 172.21.15.60:0/2465249621 conn(0x7f16940fad80 msgr2=0x7f16940fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.352+0000 7f169bc77640 1 -- 172.21.15.60:0/2465249621 shutdown_connections 2024-02-26T04:59:02.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.352+0000 7f169bc77640 1 -- 172.21.15.60:0/2465249621 wait complete. 2024-02-26T04:59:02.366 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T04:59:02.366 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T04:59:02.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.481+0000 7efee68a2640 1 -- 172.21.15.60:0/1242231810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee01011b0 msgr2=0x7efee01015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.481+0000 7efee68a2640 1 --2- 172.21.15.60:0/1242231810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee01011b0 0x7efee01015d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7efed0009900 tx=0x7efed002bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:02.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.481+0000 7efee68a2640 1 -- 172.21.15.60:0/1242231810 shutdown_connections 2024-02-26T04:59:02.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.481+0000 7efee68a2640 1 --2- 172.21.15.60:0/1242231810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efee0102d60 0x7efee01095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.481+0000 7efee68a2640 1 --2- 172.21.15.60:0/1242231810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efee01023a0 0x7efee0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.481+0000 7efee68a2640 1 --2- 172.21.15.60:0/1242231810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee01011b0 0x7efee01015d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.481+0000 7efee68a2640 1 -- 172.21.15.60:0/1242231810 >> 172.21.15.60:0/1242231810 conn(0x7efee00fc720 msgr2=0x7efee00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.481+0000 7efee68a2640 1 -- 172.21.15.60:0/1242231810 shutdown_connections 2024-02-26T04:59:02.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.481+0000 7efee68a2640 1 -- 172.21.15.60:0/1242231810 wait complete. 2024-02-26T04:59:02.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.481+0000 7efee68a2640 1 Processor -- start 2024-02-26T04:59:02.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.481+0000 7efee68a2640 1 -- start start 2024-02-26T04:59:02.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efee68a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee01011b0 0x7efee0197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efee68a2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efee01023a0 0x7efee0197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efee68a2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efee0102d60 0x7efee019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efee68a2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee019ed30 con 0x7efee01011b0 2024-02-26T04:59:02.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efee68a2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee019eea0 con 0x7efee01023a0 2024-02-26T04:59:02.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efee68a2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee019f060 con 0x7efee0102d60 2024-02-26T04:59:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efedffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee01011b0 0x7efee0197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efedffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee01011b0 0x7efee0197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33796/0 (socket says 172.21.15.60:33796) 2024-02-26T04:59:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efedffff640 1 -- 172.21.15.60:0/1078761284 learned_addr learned my addr 172.21.15.60:0/1078761284 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efee4e3f640 1 --2- 172.21.15.60:0/1078761284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efee0102d60 0x7efee019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efedf7fe640 1 --2- 172.21.15.60:0/1078761284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efee01023a0 0x7efee0197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efedffff640 1 -- 172.21.15.60:0/1078761284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efee0102d60 msgr2=0x7efee019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efedffff640 1 --2- 172.21.15.60:0/1078761284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efee0102d60 0x7efee019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efedffff640 1 -- 172.21.15.60:0/1078761284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efee01023a0 msgr2=0x7efee0197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efedffff640 1 --2- 172.21.15.60:0/1078761284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efee01023a0 0x7efee0197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efedffff640 1 -- 172.21.15.60:0/1078761284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed0009560 con 0x7efee01011b0 2024-02-26T04:59:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efedffff640 1 --2- 172.21.15.60:0/1078761284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee01011b0 0x7efee0197240 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efed0005ab0 tx=0x7efed0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.482+0000 7efedd7fa640 1 -- 172.21.15.60:0/1078761284 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed00043d0 con 0x7efee01011b0 2024-02-26T04:59:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.483+0000 7efee68a2640 1 -- 172.21.15.60:0/1078761284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee019f2e0 con 0x7efee01011b0 2024-02-26T04:59:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.483+0000 7efedd7fa640 1 -- 172.21.15.60:0/1078761284 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efed00052d0 con 0x7efee01011b0 2024-02-26T04:59:02.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.483+0000 7efedd7fa640 1 -- 172.21.15.60:0/1078761284 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0003d40 con 0x7efee01011b0 2024-02-26T04:59:02.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.483+0000 7efee68a2640 1 -- 172.21.15.60:0/1078761284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee019f7f0 con 0x7efee01011b0 2024-02-26T04:59:02.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efedd7fa640 1 -- 172.21.15.60:0/1078761284 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7efed0005440 con 0x7efee01011b0 2024-02-26T04:59:02.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efee9676740 1 Processor -- start 2024-02-26T04:59:02.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efee9676740 1 -- start start 2024-02-26T04:59:02.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efee9676740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f20650 0x55fa25f20a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efee9676740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f21040 0x55fa25f25b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efee9676740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f261b0 0x55fa25f28640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efee9676740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25ed8930 con 0x55fa25f21040 2024-02-26T04:59:02.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efee9676740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25e8c2e0 con 0x55fa25f261b0 2024-02-26T04:59:02.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efee9676740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25e89810 con 0x55fa25f20650 2024-02-26T04:59:02.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efedd7fa640 1 --2- 172.21.15.60:0/1078761284 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efec80a20c0 0x7efec80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efedffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f21040 0x55fa25f25b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efee4e3f640 1 --2- 172.21.15.60:0/1078761284 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efec80a20c0 0x7efec80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efedffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f21040 0x55fa25f25b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33804/0 (socket says 172.21.15.60:33804) 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efedffff640 1 -- 172.21.15.60:0/2891323688 learned_addr learned my addr 172.21.15.60:0/2891323688 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efedd7fa640 1 -- 172.21.15.60:0/1078761284 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7efed00f01b0 con 0x7efee01011b0 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efee4e3f640 1 --2- 172.21.15.60:0/2891323688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f20650 0x55fa25f20a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.485+0000 7efedf7fe640 1 --2- 172.21.15.60:0/2891323688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f261b0 0x55fa25f28640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.486+0000 7efedffff640 1 -- 172.21.15.60:0/2891323688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f20650 msgr2=0x55fa25f20a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.486+0000 7efedffff640 1 --2- 172.21.15.60:0/2891323688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f20650 0x55fa25f20a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.486+0000 7efedffff640 1 -- 172.21.15.60:0/2891323688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f261b0 msgr2=0x55fa25f28640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.486+0000 7efedffff640 1 --2- 172.21.15.60:0/2891323688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f261b0 0x55fa25f28640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.486+0000 7efedffff640 1 -- 172.21.15.60:0/2891323688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa25e47030 con 0x55fa25f21040 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.486+0000 7efee4e3f640 1 --2- 172.21.15.60:0/1078761284 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efec80a20c0 0x7efec80a4580 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efed8005fd0 tx=0x7efed8009ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.486+0000 7efedffff640 1 --2- 172.21.15.60:0/2891323688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f21040 0x55fa25f25b40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efed008fb30 tx=0x7efed008fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe35ed6e941a45fb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.486+0000 7efec67fc640 1 -- 172.21.15.60:0/2891323688 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed00bd1a0 con 0x55fa25f21040 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.486+0000 7efec67fc640 1 -- 172.21.15.60:0/2891323688 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efed00bcaa0 con 0x55fa25f21040 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.486+0000 7efec67fc640 1 -- 172.21.15.60:0/2891323688 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed00b9110 con 0x55fa25f21040 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 -- 172.21.15.60:0/2891323688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f21040 msgr2=0x55fa25f25b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 --2- 172.21.15.60:0/2891323688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f21040 0x55fa25f25b40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efed008fb30 tx=0x7efed008fb60 comp rx=0 tx=0).stop 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 -- 172.21.15.60:0/2891323688 shutdown_connections 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 --2- 172.21.15.60:0/2891323688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f261b0 0x55fa25f28640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 --2- 172.21.15.60:0/2891323688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f21040 0x55fa25f25b40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 --2- 172.21.15.60:0/2891323688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f20650 0x55fa25f20a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 -- 172.21.15.60:0/2891323688 >> 172.21.15.60:0/2891323688 conn(0x55fa25f177b0 msgr2=0x55fa25f17bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 -- 172.21.15.60:0/2891323688 shutdown_connections 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 -- 172.21.15.60:0/2891323688 wait complete. 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 -- start start 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 Processor -- start 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f20650 0x55fa25f2a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f21040 0x55fa25f2abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f261b0 0x55fa25f31c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25e84ff0 con 0x55fa25f21040 2024-02-26T04:59:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25b58430 con 0x55fa25f20650 2024-02-26T04:59:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25b14e00 con 0x55fa25f261b0 2024-02-26T04:59:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efee9676740 10 client.0 _ll_register_callbacks cb 0x55fa25e2cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f21040 0x55fa25f2abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f21040 0x55fa25f2abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33816/0 (socket says 172.21.15.60:33816) 2024-02-26T04:59:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.487+0000 7efedf7fe640 1 -- 172.21.15.60:0/1089401047 learned_addr learned my addr 172.21.15.60:0/1089401047 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efedffff640 1 --2- 172.21.15.60:0/1089401047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f20650 0x55fa25f2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efee4e3f640 1 --2- 172.21.15.60:0/1089401047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f261b0 0x55fa25f31c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efedf7fe640 1 -- 172.21.15.60:0/1089401047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f261b0 msgr2=0x55fa25f31c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efedf7fe640 1 --2- 172.21.15.60:0/1089401047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f261b0 0x55fa25f31c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efedf7fe640 1 -- 172.21.15.60:0/1089401047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f20650 msgr2=0x55fa25f2a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efedf7fe640 1 --2- 172.21.15.60:0/1089401047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f20650 0x55fa25f2a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efedf7fe640 1 -- 172.21.15.60:0/1089401047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee019f7f0 con 0x55fa25f21040 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efedf7fe640 1 --2- 172.21.15.60:0/1089401047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f21040 0x55fa25f2abc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efed4012700 tx=0x7efed4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e721727f6d461693 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efebf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efebf7fe640 1 -- 172.21.15.60:0/1089401047 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4017070 con 0x55fa25f21040 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efebf7fe640 1 -- 172.21.15.60:0/1089401047 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7efed4004050 con 0x55fa25f21040 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efee9676740 1 -- 172.21.15.60:0/1089401047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa25b847e0 con 0x55fa25f21040 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efebf7fe640 1 -- 172.21.15.60:0/1089401047 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efed4004210 con 0x55fa25f21040 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efee9676740 10 client.15747 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efee9676740 1 -- 172.21.15.60:0/1089401047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fa25e83670 con 0x55fa25f21040 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.488+0000 7efebf7fe640 1 -- 172.21.15.60:0/1089401047 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fa25b847e0 con 0x55fa25f21040 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.489+0000 7efebf7fe640 1 -- 172.21.15.60:0/1089401047 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efed4003950 con 0x55fa25f21040 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.489+0000 7efee9676740 10 client.15747 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.489+0000 7efee9676740 10 client.15747 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.489+0000 7efee9676740 20 client.15747 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.489+0000 7efee9676740 1 --2- 172.21.15.60:0/1089401047 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55fa25f36930 0x55fa25f38d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.489+0000 7efee9676740 4 client.15747 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.489+0000 7efee9676740 1 -- 172.21.15.60:0/1089401047 --> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa25bc1720 con 0x55fa25f36930 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.489+0000 7efedffff640 1 --2- 172.21.15.60:0/1089401047 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55fa25f36930 0x55fa25f38d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.489+0000 7efebf7fe640 10 client.15747 ms_handle_connect on v2:172.21.15.191:6838/640686517 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.489+0000 7efedffff640 1 --2- 172.21.15.60:0/1089401047 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55fa25f36930 0x55fa25f38d50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cbb20fb6ff99081 server_cookie=7638b1682f2d8338 in_seq=0 out_seq=0 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.490+0000 7efebf7fe640 1 -- 172.21.15.60:0/1089401047 <== mds.2 v2:172.21.15.191:6838/640686517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fa25bc1720 con 0x55fa25f36930 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.490+0000 7efebf7fe640 10 client.15747 handle_command_reply: tid=0 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.490+0000 7efee9676740 1 client.15747 shutdown 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.490+0000 7efee9676740 1 -- 172.21.15.60:0/1089401047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f21040 msgr2=0x55fa25f2abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.490+0000 7efee9676740 1 --2- 172.21.15.60:0/1089401047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f21040 0x55fa25f2abc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efed4012700 tx=0x7efed4012bc0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.490+0000 7efee9676740 1 -- 172.21.15.60:0/1089401047 shutdown_connections 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.490+0000 7efee9676740 1 --2- 172.21.15.60:0/1089401047 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55fa25f36930 0x55fa25f38d50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.490+0000 7efee9676740 1 --2- 172.21.15.60:0/1089401047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f261b0 0x55fa25f31c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.490+0000 7efee9676740 1 --2- 172.21.15.60:0/1089401047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f21040 0x55fa25f2abc0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.490+0000 7efee9676740 1 --2- 172.21.15.60:0/1089401047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f20650 0x55fa25f2a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.490+0000 7efebf7fe640 0 client.15747 ms_handle_reset on v2:172.21.15.191:6838/640686517 2024-02-26T04:59:02.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.490+0000 7efee9676740 1 -- 172.21.15.60:0/1089401047 >> 172.21.15.60:0/1089401047 conn(0x55fa25f177b0 msgr2=0x55fa25f16b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.490+0000 7efee9676740 1 -- 172.21.15.60:0/1089401047 shutdown_connections 2024-02-26T04:59:02.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.490+0000 7efee9676740 1 -- 172.21.15.60:0/1089401047 wait complete. 2024-02-26T04:59:02.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.491+0000 7efee9676740 20 client.15747 trim_cache size 0 max 16384 2024-02-26T04:59:02.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.504+0000 7efee9676740 1 Processor -- start 2024-02-26T04:59:02.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.504+0000 7efee9676740 1 -- start start 2024-02-26T04:59:02.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.504+0000 7efee9676740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f20650 0x55fa25f22590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.504+0000 7efee9676740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f21040 0x55fa25f22ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efee9676740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f261b0 0x55fa25f31bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efee9676740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25e0a100 con 0x55fa25f20650 2024-02-26T04:59:02.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efee9676740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25bc1720 con 0x55fa25f261b0 2024-02-26T04:59:02.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efee9676740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25b847e0 con 0x55fa25f21040 2024-02-26T04:59:02.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efedffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f20650 0x55fa25f22590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efedffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f20650 0x55fa25f22590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33822/0 (socket says 172.21.15.60:33822) 2024-02-26T04:59:02.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efedffff640 1 -- 172.21.15.60:0/3534356533 learned_addr learned my addr 172.21.15.60:0/3534356533 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efee4e3f640 1 --2- 172.21.15.60:0/3534356533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f261b0 0x55fa25f31bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efedf7fe640 1 --2- 172.21.15.60:0/3534356533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f21040 0x55fa25f22ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efedffff640 1 -- 172.21.15.60:0/3534356533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f21040 msgr2=0x55fa25f22ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efedffff640 1 --2- 172.21.15.60:0/3534356533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f21040 0x55fa25f22ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efedffff640 1 -- 172.21.15.60:0/3534356533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f261b0 msgr2=0x55fa25f31bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efedffff640 1 --2- 172.21.15.60:0/3534356533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f261b0 0x55fa25f31bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efedffff640 1 -- 172.21.15.60:0/3534356533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa25e1fbf0 con 0x55fa25f20650 2024-02-26T04:59:02.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efedffff640 1 --2- 172.21.15.60:0/3534356533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f20650 0x55fa25f22590 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efed002c840 tx=0x7efed0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d07b02ed3615a04 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efec5ffb640 1 -- 172.21.15.60:0/3534356533 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed00c0c70 con 0x55fa25f20650 2024-02-26T04:59:02.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efec5ffb640 1 -- 172.21.15.60:0/3534356533 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efed00bd1a0 con 0x55fa25f20650 2024-02-26T04:59:02.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.505+0000 7efec5ffb640 1 -- 172.21.15.60:0/3534356533 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed00c4db0 con 0x55fa25f20650 2024-02-26T04:59:02.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.506+0000 7efee9676740 1 -- 172.21.15.60:0/3534356533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f20650 msgr2=0x55fa25f22590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.506+0000 7efee9676740 1 --2- 172.21.15.60:0/3534356533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f20650 0x55fa25f22590 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efed002c840 tx=0x7efed0005fd0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.506+0000 7efee9676740 1 -- 172.21.15.60:0/3534356533 shutdown_connections 2024-02-26T04:59:02.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.506+0000 7efee9676740 1 --2- 172.21.15.60:0/3534356533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f261b0 0x55fa25f31bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.506+0000 7efee9676740 1 --2- 172.21.15.60:0/3534356533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f21040 0x55fa25f22ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.506+0000 7efee9676740 1 --2- 172.21.15.60:0/3534356533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f20650 0x55fa25f22590 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.506+0000 7efee9676740 1 -- 172.21.15.60:0/3534356533 >> 172.21.15.60:0/3534356533 conn(0x55fa25f177b0 msgr2=0x55fa25f17b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.506+0000 7efee9676740 1 -- 172.21.15.60:0/3534356533 shutdown_connections 2024-02-26T04:59:02.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.506+0000 7efee9676740 1 -- 172.21.15.60:0/3534356533 wait complete. 2024-02-26T04:59:02.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.507+0000 7efee9676740 1 -- start start 2024-02-26T04:59:02.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.507+0000 7efee9676740 1 Processor -- start 2024-02-26T04:59:02.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.507+0000 7efee9676740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f20650 0x55fa25f317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.507+0000 7efee9676740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f21040 0x55fa25f2fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.507+0000 7efee9676740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f261b0 0x55fa25f30340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.507+0000 7efee9676740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25f30880 con 0x55fa25f20650 2024-02-26T04:59:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.507+0000 7efee9676740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25f309f0 con 0x55fa25f21040 2024-02-26T04:59:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.507+0000 7efee9676740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25f30bb0 con 0x55fa25f261b0 2024-02-26T04:59:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.507+0000 7efee9676740 10 client.0 _ll_register_callbacks cb 0x7efed0004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.507+0000 7efedffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f20650 0x55fa25f317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.507+0000 7efedffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f20650 0x55fa25f317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33832/0 (socket says 172.21.15.60:33832) 2024-02-26T04:59:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.507+0000 7efedffff640 1 -- 172.21.15.60:0/1411586047 learned_addr learned my addr 172.21.15.60:0/1411586047 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.507+0000 7efee4e3f640 1 --2- 172.21.15.60:0/1411586047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f261b0 0x55fa25f30340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efedffff640 1 -- 172.21.15.60:0/1411586047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f261b0 msgr2=0x55fa25f30340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efedffff640 1 --2- 172.21.15.60:0/1411586047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f261b0 0x55fa25f30340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efedffff640 1 -- 172.21.15.60:0/1411586047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f21040 msgr2=0x55fa25f2fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efedffff640 1 --2- 172.21.15.60:0/1411586047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f21040 0x55fa25f2fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efedffff640 1 -- 172.21.15.60:0/1411586047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa25e47030 con 0x55fa25f20650 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efee4e3f640 1 --2- 172.21.15.60:0/1411586047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f261b0 0x55fa25f30340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efedffff640 1 --2- 172.21.15.60:0/1411586047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f20650 0x55fa25f317f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7efed00b9470 tx=0x7efed00bd310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9382549c54faed97 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efebf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efebf7fe640 1 -- 172.21.15.60:0/1411586047 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed00c0c70 con 0x55fa25f20650 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efebf7fe640 1 -- 172.21.15.60:0/1411586047 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7efed00c5240 con 0x55fa25f20650 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efee9676740 1 -- 172.21.15.60:0/1411586047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efed00bcaa0 con 0x55fa25f20650 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efebf7fe640 1 -- 172.21.15.60:0/1411586047 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efed00bc5e0 con 0x55fa25f20650 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efee9676740 10 client.15753 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efee9676740 1 -- 172.21.15.60:0/1411586047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fa25e45890 con 0x55fa25f20650 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efebf7fe640 1 -- 172.21.15.60:0/1411586047 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efed00bcaa0 con 0x55fa25f20650 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efebf7fe640 1 -- 172.21.15.60:0/1411586047 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efed003c070 con 0x55fa25f20650 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efee9676740 10 client.15753 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efee9676740 10 client.15753 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efee9676740 20 client.15753 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.508+0000 7efee9676740 1 --2- 172.21.15.60:0/1411586047 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55fa25f8e110 0x55fa25f90500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.509+0000 7efee9676740 4 client.15753 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.509+0000 7efee9676740 1 -- 172.21.15.60:0/1411586047 --> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fa25c10890 con 0x55fa25f8e110 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.509+0000 7efedf7fe640 1 --2- 172.21.15.60:0/1411586047 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55fa25f8e110 0x55fa25f90500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.509+0000 7efedf7fe640 1 --2- 172.21.15.60:0/1411586047 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55fa25f8e110 0x55fa25f90500 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b606b9380ec5357 server_cookie=da13716f7cae48e9 in_seq=0 out_seq=0 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.509+0000 7efebf7fe640 10 client.15753 ms_handle_connect on v2:172.21.15.191:6838/640686517 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.509+0000 7efebf7fe640 1 -- 172.21.15.60:0/1411586047 <== mds.2 v2:172.21.15.191:6838/640686517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fa25c10890 con 0x55fa25f8e110 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.509+0000 7efebf7fe640 10 client.15753 handle_command_reply: tid=0 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.509+0000 7efee9676740 1 client.15753 shutdown 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee9676740 1 -- 172.21.15.60:0/1411586047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f20650 msgr2=0x55fa25f317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee9676740 1 --2- 172.21.15.60:0/1411586047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f20650 0x55fa25f317f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7efed00b9470 tx=0x7efed00bd310 comp rx=0 tx=0).stop 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee9676740 1 -- 172.21.15.60:0/1411586047 shutdown_connections 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee9676740 1 --2- 172.21.15.60:0/1411586047 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x55fa25f8e110 0x55fa25f90500 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee9676740 1 --2- 172.21.15.60:0/1411586047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa25f261b0 0x55fa25f30340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee9676740 1 --2- 172.21.15.60:0/1411586047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fa25f21040 0x55fa25f2fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efebf7fe640 0 client.15753 ms_handle_reset on v2:172.21.15.191:6838/640686517 2024-02-26T04:59:02.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee9676740 1 --2- 172.21.15.60:0/1411586047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa25f20650 0x55fa25f317f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee9676740 1 -- 172.21.15.60:0/1411586047 >> 172.21.15.60:0/1411586047 conn(0x55fa25f177b0 msgr2=0x55fa25f2a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee9676740 1 -- 172.21.15.60:0/1411586047 shutdown_connections 2024-02-26T04:59:02.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee9676740 1 -- 172.21.15.60:0/1411586047 wait complete. 2024-02-26T04:59:02.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee9676740 20 client.15753 trim_cache size 0 max 16384 2024-02-26T04:59:02.518 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T04:59:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee68a2640 1 -- 172.21.15.60:0/1078761284 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efec80a20c0 msgr2=0x7efec80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee68a2640 1 --2- 172.21.15.60:0/1078761284 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efec80a20c0 0x7efec80a4580 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efed8005fd0 tx=0x7efed8009ee0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee68a2640 1 -- 172.21.15.60:0/1078761284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee01011b0 msgr2=0x7efee0197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee68a2640 1 --2- 172.21.15.60:0/1078761284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee01011b0 0x7efee0197240 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efed0005ab0 tx=0x7efed0005e50 comp rx=0 tx=0).stop 2024-02-26T04:59:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee68a2640 1 -- 172.21.15.60:0/1078761284 shutdown_connections 2024-02-26T04:59:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee68a2640 1 --2- 172.21.15.60:0/1078761284 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7efec80a20c0 0x7efec80a4580 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee68a2640 1 --2- 172.21.15.60:0/1078761284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efee0102d60 0x7efee019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee68a2640 1 --2- 172.21.15.60:0/1078761284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efee01023a0 0x7efee0197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee68a2640 1 --2- 172.21.15.60:0/1078761284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee01011b0 0x7efee0197240 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.510+0000 7efee68a2640 1 -- 172.21.15.60:0/1078761284 >> 172.21.15.60:0/1078761284 conn(0x7efee00fc720 msgr2=0x7efee0105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.511+0000 7efee68a2640 1 -- 172.21.15.60:0/1078761284 shutdown_connections 2024-02-26T04:59:02.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.511+0000 7efee68a2640 1 -- 172.21.15.60:0/1078761284 wait complete. 2024-02-26T04:59:02.524 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T04:59:02.525 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T04:59:02.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.635+0000 7f48cf51e640 1 -- 172.21.15.60:0/917506508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c8100090 msgr2=0x7f48c81004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.635+0000 7f48cf51e640 1 --2- 172.21.15.60:0/917506508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c8100090 0x7f48c81004b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009900 tx=0x7f48b802bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:02.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.635+0000 7f48cf51e640 1 -- 172.21.15.60:0/917506508 shutdown_connections 2024-02-26T04:59:02.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.635+0000 7f48cf51e640 1 --2- 172.21.15.60:0/917506508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48c8101c40 0x7f48c81084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.635+0000 7f48cf51e640 1 --2- 172.21.15.60:0/917506508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c8101280 0x7f48c8101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.635+0000 7f48cf51e640 1 --2- 172.21.15.60:0/917506508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c8100090 0x7f48c81004b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.635+0000 7f48cf51e640 1 -- 172.21.15.60:0/917506508 >> 172.21.15.60:0/917506508 conn(0x7f48c80fb620 msgr2=0x7f48c80fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.635+0000 7f48cf51e640 1 -- 172.21.15.60:0/917506508 shutdown_connections 2024-02-26T04:59:02.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.635+0000 7f48cf51e640 1 -- 172.21.15.60:0/917506508 wait complete. 2024-02-26T04:59:02.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.636+0000 7f48cf51e640 1 Processor -- start 2024-02-26T04:59:02.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.636+0000 7f48cf51e640 1 -- start start 2024-02-26T04:59:02.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.636+0000 7f48cf51e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c8100090 0x7f48c81972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.636+0000 7f48cf51e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48c8101280 0x7f48c8197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.636+0000 7f48cf51e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c8101c40 0x7f48c819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.636+0000 7f48cf51e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c819ed80 con 0x7f48c8100090 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.636+0000 7f48cf51e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c819eef0 con 0x7f48c8101c40 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.636+0000 7f48cf51e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c819f0b0 con 0x7f48c8101280 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.636+0000 7f48cd2ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c8100090 0x7f48c81972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.636+0000 7f48cd2ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c8100090 0x7f48c81972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33852/0 (socket says 172.21.15.60:33852) 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.636+0000 7f48cd2ba640 1 -- 172.21.15.60:0/2323940647 learned_addr learned my addr 172.21.15.60:0/2323940647 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.636+0000 7f48cdabb640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c8101c40 0x7f48c819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.636+0000 7f48ccab9640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48c8101280 0x7f48c8197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.637+0000 7f48cd2ba640 1 -- 172.21.15.60:0/2323940647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48c8101280 msgr2=0x7f48c8197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.637+0000 7f48cd2ba640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48c8101280 0x7f48c8197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.637+0000 7f48cd2ba640 1 -- 172.21.15.60:0/2323940647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c8101c40 msgr2=0x7f48c819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.637+0000 7f48cd2ba640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c8101c40 0x7f48c819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.637+0000 7f48cd2ba640 1 -- 172.21.15.60:0/2323940647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b8009560 con 0x7f48c8100090 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.637+0000 7f48cdabb640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c8101c40 0x7f48c819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:59:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.637+0000 7f48cd2ba640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c8100090 0x7f48c81972e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f48b8005ab0 tx=0x7f48b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.637+0000 7f48be7fc640 1 -- 172.21.15.60:0/2323940647 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8004aa0 con 0x7f48c8100090 2024-02-26T04:59:02.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.637+0000 7f48be7fc640 1 -- 172.21.15.60:0/2323940647 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48b80052d0 con 0x7f48c8100090 2024-02-26T04:59:02.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.637+0000 7f48cf51e640 1 -- 172.21.15.60:0/2323940647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c819f330 con 0x7f48c8100090 2024-02-26T04:59:02.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.637+0000 7f48be7fc640 1 -- 172.21.15.60:0/2323940647 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8003e80 con 0x7f48c8100090 2024-02-26T04:59:02.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.637+0000 7f48cf51e640 1 -- 172.21.15.60:0/2323940647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c819f890 con 0x7f48c8100090 2024-02-26T04:59:02.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.639+0000 7f48be7fc640 1 -- 172.21.15.60:0/2323940647 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f48b8005440 con 0x7f48c8100090 2024-02-26T04:59:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48d23fc740 1 Processor -- start 2024-02-26T04:59:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48d23fc740 1 -- start start 2024-02-26T04:59:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48be7fc640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48b40a2140 0x7f48b40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48d23fc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181bf650 0x5610181bfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48d23fc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181c0040 0x5610181c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48d23fc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181c51b0 0x5610181c7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48d23fc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561018177930 con 0x5610181c0040 2024-02-26T04:59:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48d23fc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101812b2e0 con 0x5610181c51b0 2024-02-26T04:59:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48d23fc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561018128810 con 0x5610181bf650 2024-02-26T04:59:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48cdabb640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48b40a2140 0x7f48b40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48ccab9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181c0040 0x5610181c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48be7fc640 1 -- 172.21.15.60:0/2323940647 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f48b80f0400 con 0x7f48c8100090 2024-02-26T04:59:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48ccab9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181c0040 0x5610181c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33854/0 (socket says 172.21.15.60:33854) 2024-02-26T04:59:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48cd2ba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181bf650 0x5610181bfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48ccab9640 1 -- 172.21.15.60:0/1441053443 learned_addr learned my addr 172.21.15.60:0/1441053443 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48cdabb640 1 --2- 172.21.15.60:0/1441053443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181c51b0 0x5610181c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48cd2ba640 1 -- 172.21.15.60:0/1441053443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181c51b0 msgr2=0x5610181c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48cd2ba640 1 --2- 172.21.15.60:0/1441053443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181c51b0 0x5610181c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48cd2ba640 1 -- 172.21.15.60:0/1441053443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181c0040 msgr2=0x5610181c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48cd2ba640 1 --2- 172.21.15.60:0/1441053443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181c0040 0x5610181c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48cd2ba640 1 -- 172.21.15.60:0/1441053443 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610180e6030 con 0x5610181bf650 2024-02-26T04:59:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.640+0000 7f48ccab9640 1 --2- 172.21.15.60:0/1441053443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181c0040 0x5610181c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:59:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.641+0000 7f48cdabb640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48b40a2140 0x7f48b40a4600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009fd0 tx=0x7f48c0009f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.641+0000 7f48cd2ba640 1 --2- 172.21.15.60:0/1441053443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181bf650 0x5610181bfa70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f48b80edbb0 tx=0x7f48b8003920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a4f11c470f509b7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.641+0000 7f48b37fe640 1 -- 172.21.15.60:0/1441053443 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b80bd490 con 0x5610181bf650 2024-02-26T04:59:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.641+0000 7f48b37fe640 1 -- 172.21.15.60:0/1441053443 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48b808e940 con 0x5610181bf650 2024-02-26T04:59:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.641+0000 7f48b37fe640 1 -- 172.21.15.60:0/1441053443 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b80c5530 con 0x5610181bf650 2024-02-26T04:59:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.641+0000 7f48d23fc740 1 -- 172.21.15.60:0/1441053443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181bf650 msgr2=0x5610181bfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.641+0000 7f48d23fc740 1 --2- 172.21.15.60:0/1441053443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181bf650 0x5610181bfa70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f48b80edbb0 tx=0x7f48b8003920 comp rx=0 tx=0).stop 2024-02-26T04:59:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 -- 172.21.15.60:0/1441053443 shutdown_connections 2024-02-26T04:59:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 --2- 172.21.15.60:0/1441053443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181c51b0 0x5610181c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 --2- 172.21.15.60:0/1441053443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181c0040 0x5610181c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 --2- 172.21.15.60:0/1441053443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181bf650 0x5610181bfa70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 -- 172.21.15.60:0/1441053443 >> 172.21.15.60:0/1441053443 conn(0x5610181b67b0 msgr2=0x5610181b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 -- 172.21.15.60:0/1441053443 shutdown_connections 2024-02-26T04:59:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 -- 172.21.15.60:0/1441053443 wait complete. 2024-02-26T04:59:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 -- start start 2024-02-26T04:59:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 Processor -- start 2024-02-26T04:59:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181bf650 0x5610181c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181c0040 0x5610181c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181c51b0 0x5610181d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561018123ff0 con 0x5610181c0040 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561017df7430 con 0x5610181c51b0 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561017db3e00 con 0x5610181bf650 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48d23fc740 10 client.0 _ll_register_callbacks cb 0x5610180cbc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48ccab9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181c0040 0x5610181c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48ccab9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181c0040 0x5610181c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33870/0 (socket says 172.21.15.60:33870) 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48ccab9640 1 -- 172.21.15.60:0/2419287824 learned_addr learned my addr 172.21.15.60:0/2419287824 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48cd2ba640 1 --2- 172.21.15.60:0/2419287824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181bf650 0x5610181c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.642+0000 7f48cdabb640 1 --2- 172.21.15.60:0/2419287824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181c51b0 0x5610181d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.643+0000 7f48ccab9640 1 -- 172.21.15.60:0/2419287824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181bf650 msgr2=0x5610181c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.643+0000 7f48ccab9640 1 --2- 172.21.15.60:0/2419287824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181bf650 0x5610181c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.643+0000 7f48ccab9640 1 -- 172.21.15.60:0/2419287824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181c51b0 msgr2=0x5610181d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.643+0000 7f48ccab9640 1 --2- 172.21.15.60:0/2419287824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181c51b0 0x5610181d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.643+0000 7f48ccab9640 1 -- 172.21.15.60:0/2419287824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c819f890 con 0x5610181c0040 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.643+0000 7f48ccab9640 1 --2- 172.21.15.60:0/2419287824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181c0040 0x5610181c9a90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f48c4012700 tx=0x7f48c4012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90de6210ac792a69 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.643+0000 7f48b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.643+0000 7f48b0ff9640 1 -- 172.21.15.60:0/2419287824 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4017070 con 0x5610181c0040 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.643+0000 7f48d23fc740 1 -- 172.21.15.60:0/2419287824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561017e237e0 con 0x5610181c0040 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.643+0000 7f48b0ff9640 1 -- 172.21.15.60:0/2419287824 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f48c4004050 con 0x5610181c0040 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.643+0000 7f48b0ff9640 1 -- 172.21.15.60:0/2419287824 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48c4004210 con 0x5610181c0040 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.643+0000 7f48d23fc740 10 client.15765 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.643+0000 7f48d23fc740 1 -- 172.21.15.60:0/2419287824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561018122670 con 0x5610181c0040 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.643+0000 7f48b0ff9640 1 -- 172.21.15.60:0/2419287824 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561017e237e0 con 0x5610181c0040 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.644+0000 7f48b0ff9640 1 -- 172.21.15.60:0/2419287824 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f48c400f5d0 con 0x5610181c0040 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.644+0000 7f48d23fc740 10 client.15765 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.644+0000 7f48d23fc740 10 client.15765 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.644+0000 7f48d23fc740 20 client.15765 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.644+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2419287824 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5610181d5810 0x5610181d7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.644+0000 7f48d23fc740 4 client.15765 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:59:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.644+0000 7f48d23fc740 1 -- 172.21.15.60:0/2419287824 --> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561017e60720 con 0x5610181d5810 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.644+0000 7f48cd2ba640 1 --2- 172.21.15.60:0/2419287824 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5610181d5810 0x5610181d7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.644+0000 7f48cd2ba640 1 --2- 172.21.15.60:0/2419287824 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5610181d5810 0x5610181d7c30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4be2e83d0c6fad12 server_cookie=58f2adb6e50c3dbc in_seq=0 out_seq=0 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.644+0000 7f48b0ff9640 10 client.15765 ms_handle_connect on v2:172.21.15.156:6836/707575792 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.645+0000 7f48b0ff9640 1 -- 172.21.15.60:0/2419287824 <== mds.1 v2:172.21.15.156:6836/707575792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561017e60720 con 0x5610181d5810 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.645+0000 7f48b0ff9640 10 client.15765 handle_command_reply: tid=0 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.645+0000 7f48d23fc740 1 client.15765 shutdown 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.645+0000 7f48d23fc740 1 -- 172.21.15.60:0/2419287824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181c0040 msgr2=0x5610181c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.645+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2419287824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181c0040 0x5610181c9a90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f48c4012700 tx=0x7f48c4012b00 comp rx=0 tx=0).stop 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.645+0000 7f48d23fc740 1 -- 172.21.15.60:0/2419287824 shutdown_connections 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.645+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2419287824 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5610181d5810 0x5610181d7c30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.645+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2419287824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181c51b0 0x5610181d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.645+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2419287824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181c0040 0x5610181c9a90 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.645+0000 7f48b0ff9640 0 client.15765 ms_handle_reset on v2:172.21.15.156:6836/707575792 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.645+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2419287824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181bf650 0x5610181c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.645+0000 7f48d23fc740 1 -- 172.21.15.60:0/2419287824 >> 172.21.15.60:0/2419287824 conn(0x5610181b67b0 msgr2=0x5610181b5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.646+0000 7f48d23fc740 1 -- 172.21.15.60:0/2419287824 shutdown_connections 2024-02-26T04:59:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.646+0000 7f48d23fc740 1 -- 172.21.15.60:0/2419287824 wait complete. 2024-02-26T04:59:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.646+0000 7f48d23fc740 20 client.15765 trim_cache size 0 max 16384 2024-02-26T04:59:02.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48d23fc740 1 Processor -- start 2024-02-26T04:59:02.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48d23fc740 1 -- start start 2024-02-26T04:59:02.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48d23fc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181b67b0 0x5610181c1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48d23fc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181bf650 0x5610181c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48d23fc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181c0040 0x5610181d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48d23fc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610180a9100 con 0x5610181b67b0 2024-02-26T04:59:02.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48d23fc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610181c0ce0 con 0x5610181bf650 2024-02-26T04:59:02.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48d23fc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610181b1e40 con 0x5610181c0040 2024-02-26T04:59:02.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48cd2ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181b67b0 0x5610181c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48ccab9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181bf650 0x5610181c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48cd2ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181b67b0 0x5610181c1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33880/0 (socket says 172.21.15.60:33880) 2024-02-26T04:59:02.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48cd2ba640 1 -- 172.21.15.60:0/2534004115 learned_addr learned my addr 172.21.15.60:0/2534004115 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48cdabb640 1 --2- 172.21.15.60:0/2534004115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181c0040 0x5610181d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48ccab9640 1 -- 172.21.15.60:0/2534004115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181c0040 msgr2=0x5610181d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48ccab9640 1 --2- 172.21.15.60:0/2534004115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181c0040 0x5610181d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48ccab9640 1 -- 172.21.15.60:0/2534004115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181b67b0 msgr2=0x5610181c1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48ccab9640 1 --2- 172.21.15.60:0/2534004115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181b67b0 0x5610181c1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48ccab9640 1 -- 172.21.15.60:0/2534004115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610180bebf0 con 0x5610181bf650 2024-02-26T04:59:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48cd2ba640 1 --2- 172.21.15.60:0/2534004115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181b67b0 0x5610181c1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:59:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.658+0000 7f48ccab9640 1 --2- 172.21.15.60:0/2534004115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181bf650 0x5610181c1a40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f48c400fa80 tx=0x7f48c4009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9720f1758b4e33d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.659+0000 7f48b2ffd640 1 -- 172.21.15.60:0/2534004115 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4017040 con 0x5610181bf650 2024-02-26T04:59:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.659+0000 7f48b2ffd640 1 -- 172.21.15.60:0/2534004115 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48c40055a0 con 0x5610181bf650 2024-02-26T04:59:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.659+0000 7f48b2ffd640 1 -- 172.21.15.60:0/2534004115 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4003ab0 con 0x5610181bf650 2024-02-26T04:59:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.659+0000 7f48d23fc740 1 -- 172.21.15.60:0/2534004115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181bf650 msgr2=0x5610181c1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.659+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2534004115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181bf650 0x5610181c1a40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f48c400fa80 tx=0x7f48c4009680 comp rx=0 tx=0).stop 2024-02-26T04:59:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.659+0000 7f48d23fc740 1 -- 172.21.15.60:0/2534004115 shutdown_connections 2024-02-26T04:59:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.659+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2534004115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181c0040 0x5610181d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.659+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2534004115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181bf650 0x5610181c1a40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.659+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2534004115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181b67b0 0x5610181c1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.659+0000 7f48d23fc740 1 -- 172.21.15.60:0/2534004115 >> 172.21.15.60:0/2534004115 conn(0x5610181c0e20 msgr2=0x5610181c9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.659+0000 7f48d23fc740 1 -- 172.21.15.60:0/2534004115 shutdown_connections 2024-02-26T04:59:02.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.659+0000 7f48d23fc740 1 -- 172.21.15.60:0/2534004115 wait complete. 2024-02-26T04:59:02.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.659+0000 7f48d23fc740 1 -- start start 2024-02-26T04:59:02.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48d23fc740 1 Processor -- start 2024-02-26T04:59:02.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48d23fc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181b67b0 0x5610181cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48d23fc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181bf650 0x5610181cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48d23fc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181c0040 0x5610181cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48d23fc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610181d0f30 con 0x5610181bf650 2024-02-26T04:59:02.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48d23fc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101822bd50 con 0x5610181b67b0 2024-02-26T04:59:02.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48d23fc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101822bf10 con 0x5610181c0040 2024-02-26T04:59:02.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48d23fc740 10 client.0 _ll_register_callbacks cb 0x7f48c4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:02.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48ccab9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181bf650 0x5610181cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48cd2ba640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181b67b0 0x5610181cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48ccab9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181bf650 0x5610181cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33890/0 (socket says 172.21.15.60:33890) 2024-02-26T04:59:02.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48ccab9640 1 -- 172.21.15.60:0/2677706785 learned_addr learned my addr 172.21.15.60:0/2677706785 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48cdabb640 1 --2- 172.21.15.60:0/2677706785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181c0040 0x5610181cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48ccab9640 1 -- 172.21.15.60:0/2677706785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181c0040 msgr2=0x5610181cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48ccab9640 1 --2- 172.21.15.60:0/2677706785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181c0040 0x5610181cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48ccab9640 1 -- 172.21.15.60:0/2677706785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181b67b0 msgr2=0x5610181cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48ccab9640 1 --2- 172.21.15.60:0/2677706785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181b67b0 0x5610181cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48ccab9640 1 -- 172.21.15.60:0/2677706785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610180e6030 con 0x5610181bf650 2024-02-26T04:59:02.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.660+0000 7f48cdabb640 1 --2- 172.21.15.60:0/2677706785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181c0040 0x5610181cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:59:02.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48ccab9640 1 --2- 172.21.15.60:0/2677706785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181bf650 0x5610181cf820 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000c00 tx=0x7f48c400fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc286ed4546f9be server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:02.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48b0ff9640 1 -- 172.21.15.60:0/2677706785 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4017040 con 0x5610181bf650 2024-02-26T04:59:02.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48d23fc740 1 -- 172.21.15.60:0/2677706785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48b808e940 con 0x5610181bf650 2024-02-26T04:59:02.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48b0ff9640 1 -- 172.21.15.60:0/2677706785 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f48c4003830 con 0x5610181bf650 2024-02-26T04:59:02.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48b0ff9640 1 -- 172.21.15.60:0/2677706785 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48c40039f0 con 0x5610181bf650 2024-02-26T04:59:02.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48d23fc740 10 client.15771 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48d23fc740 1 -- 172.21.15.60:0/2677706785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561018122670 con 0x5610181bf650 2024-02-26T04:59:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48b0ff9640 1 -- 172.21.15.60:0/2677706785 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48b808e940 con 0x5610181bf650 2024-02-26T04:59:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48b0ff9640 1 -- 172.21.15.60:0/2677706785 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f48c402f030 con 0x5610181bf650 2024-02-26T04:59:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48d23fc740 10 client.15771 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48d23fc740 10 client.15771 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-26T04:59:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48d23fc740 20 client.15771 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2677706785 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5610181c51b0 0x56101822f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48d23fc740 4 client.15771 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T04:59:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48d23fc740 1 -- 172.21.15.60:0/2677706785 --> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561017eaf890 con 0x5610181c51b0 2024-02-26T04:59:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.661+0000 7f48cd2ba640 1 --2- 172.21.15.60:0/2677706785 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5610181c51b0 0x56101822f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48cd2ba640 1 --2- 172.21.15.60:0/2677706785 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5610181c51b0 0x56101822f810 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=285d86a306ef00d2 server_cookie=4e2332d299318ab in_seq=0 out_seq=0 2024-02-26T04:59:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48b0ff9640 10 client.15771 ms_handle_connect on v2:172.21.15.156:6836/707575792 2024-02-26T04:59:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48b0ff9640 1 -- 172.21.15.60:0/2677706785 <== mds.1 v2:172.21.15.156:6836/707575792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561017eaf890 con 0x5610181c51b0 2024-02-26T04:59:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48b0ff9640 10 client.15771 handle_command_reply: tid=0 2024-02-26T04:59:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48d23fc740 1 client.15771 shutdown 2024-02-26T04:59:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48d23fc740 1 -- 172.21.15.60:0/2677706785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181bf650 msgr2=0x5610181cf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2677706785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181bf650 0x5610181cf820 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000c00 tx=0x7f48c400fe50 comp rx=0 tx=0).stop 2024-02-26T04:59:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48d23fc740 1 -- 172.21.15.60:0/2677706785 shutdown_connections 2024-02-26T04:59:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2677706785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610181c0040 0x5610181cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2677706785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5610181bf650 0x5610181cf820 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2677706785 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x5610181c51b0 0x56101822f810 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48d23fc740 1 --2- 172.21.15.60:0/2677706785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610181b67b0 0x5610181cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48b0ff9640 0 client.15771 ms_handle_reset on v2:172.21.15.156:6836/707575792 2024-02-26T04:59:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48d23fc740 1 -- 172.21.15.60:0/2677706785 >> 172.21.15.60:0/2677706785 conn(0x5610181c0e20 msgr2=0x5610181c9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48d23fc740 1 -- 172.21.15.60:0/2677706785 shutdown_connections 2024-02-26T04:59:02.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48d23fc740 1 -- 172.21.15.60:0/2677706785 wait complete. 2024-02-26T04:59:02.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.662+0000 7f48d23fc740 20 client.15771 trim_cache size 0 max 16384 2024-02-26T04:59:02.673 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T04:59:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.663+0000 7f48cf51e640 1 -- 172.21.15.60:0/2323940647 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48b40a2140 msgr2=0x7f48b40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.663+0000 7f48cf51e640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48b40a2140 0x7f48b40a4600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009fd0 tx=0x7f48c0009f40 comp rx=0 tx=0).stop 2024-02-26T04:59:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.663+0000 7f48cf51e640 1 -- 172.21.15.60:0/2323940647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c8100090 msgr2=0x7f48c81972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.663+0000 7f48cf51e640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c8100090 0x7f48c81972e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f48b8005ab0 tx=0x7f48b8004270 comp rx=0 tx=0).stop 2024-02-26T04:59:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.663+0000 7f48cf51e640 1 -- 172.21.15.60:0/2323940647 shutdown_connections 2024-02-26T04:59:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.663+0000 7f48cf51e640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f48b40a2140 0x7f48b40a4600 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.663+0000 7f48cf51e640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48c8101c40 0x7f48c819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.663+0000 7f48cf51e640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48c8101280 0x7f48c8197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.663+0000 7f48cf51e640 1 --2- 172.21.15.60:0/2323940647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c8100090 0x7f48c81972e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.663+0000 7f48cf51e640 1 -- 172.21.15.60:0/2323940647 >> 172.21.15.60:0/2323940647 conn(0x7f48c80fb620 msgr2=0x7f48c8069750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.663+0000 7f48cf51e640 1 -- 172.21.15.60:0/2323940647 shutdown_connections 2024-02-26T04:59:02.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.663+0000 7f48cf51e640 1 -- 172.21.15.60:0/2323940647 wait complete. 2024-02-26T04:59:02.676 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T04:59:02.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T04:59:02.731 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-02-26T04:59:02.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-02-26T04:59:02.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-02-26T04:59:02.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-02-26T04:59:02.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-02-26T04:59:02.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-02-26T04:59:02.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-02-26T04:59:02.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-02-26T04:59:02.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-02-26T04:59:02.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-02-26T04:59:02.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-02-26T04:59:02.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-02-26T04:59:02.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-02-26T04:59:02.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-02-26T04:59:02.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-02-26T04:59:02.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-02-26T04:59:02.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-02-26T04:59:02.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-02-26T04:59:02.737 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-02-26T04:59:02.737 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-02-26T04:59:02.737 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-02-26T04:59:02.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.798+0000 7fa5be0a2640 1 -- 172.21.15.60:0/1444231975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b8101190 msgr2=0x7fa5b81015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.798+0000 7fa5be0a2640 1 --2- 172.21.15.60:0/1444231975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b8101190 0x7fa5b81015b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009900 tx=0x7fa5ac02bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.798+0000 7fa5be0a2640 1 -- 172.21.15.60:0/1444231975 shutdown_connections 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.798+0000 7fa5be0a2640 1 --2- 172.21.15.60:0/1444231975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b8102d40 0x7fa5b81095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.798+0000 7fa5be0a2640 1 --2- 172.21.15.60:0/1444231975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5b8102380 0x7fa5b8102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.798+0000 7fa5be0a2640 1 --2- 172.21.15.60:0/1444231975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b8101190 0x7fa5b81015b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.798+0000 7fa5be0a2640 1 -- 172.21.15.60:0/1444231975 >> 172.21.15.60:0/1444231975 conn(0x7fa5b80fc6e0 msgr2=0x7fa5b80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.798+0000 7fa5be0a2640 1 -- 172.21.15.60:0/1444231975 shutdown_connections 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.798+0000 7fa5be0a2640 1 -- 172.21.15.60:0/1444231975 wait complete. 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.798+0000 7fa5be0a2640 1 Processor -- start 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5be0a2640 1 -- start start 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5be0a2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b8101190 0x7fa5b8074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5be0a2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5b8102380 0x7fa5b8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5be0a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b8102d40 0x7fa5b8073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5be0a2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b8074210 con 0x7fa5b8102d40 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5be0a2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b8074380 con 0x7fa5b8102380 2024-02-26T04:59:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5be0a2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b81a12c0 con 0x7fa5b8101190 2024-02-26T04:59:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b8102d40 0x7fa5b8073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5b77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b8101190 0x7fa5b8074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5b6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5b8102380 0x7fa5b8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b8102d40 0x7fa5b8073ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33906/0 (socket says 172.21.15.60:33906) 2024-02-26T04:59:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5b6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5b8102380 0x7fa5b8073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.60:45012/0 (socket says 172.21.15.60:45012) 2024-02-26T04:59:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5b77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b8101190 0x7fa5b8074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.60:47716/0 (socket says 172.21.15.60:47716) 2024-02-26T04:59:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5b7fff640 1 -- 172.21.15.60:0/1802243131 learned_addr learned my addr 172.21.15.60:0/1802243131 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5b6ffd640 1 -- 172.21.15.60:0/1802243131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b8101190 msgr2=0x7fa5b8074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5b6ffd640 1 --2- 172.21.15.60:0/1802243131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b8101190 0x7fa5b8074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5b6ffd640 1 -- 172.21.15.60:0/1802243131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b8102d40 msgr2=0x7fa5b8073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5b6ffd640 1 --2- 172.21.15.60:0/1802243131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b8102d40 0x7fa5b8073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5b6ffd640 1 -- 172.21.15.60:0/1802243131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac009560 con 0x7fa5b8102380 2024-02-26T04:59:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.799+0000 7fa5b7fff640 1 --2- 172.21.15.60:0/1802243131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b8102d40 0x7fa5b8073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:59:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.800+0000 7fa5b6ffd640 1 --2- 172.21.15.60:0/1802243131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5b8102380 0x7fa5b8073570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800ea20 tx=0x7fa5a800eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.800+0000 7fa5b4ff9640 1 -- 172.21.15.60:0/1802243131 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800ccc0 con 0x7fa5b8102380 2024-02-26T04:59:02.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.800+0000 7fa5be0a2640 1 -- 172.21.15.60:0/1802243131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b81a1570 con 0x7fa5b8102380 2024-02-26T04:59:02.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.800+0000 7fa5b4ff9640 1 -- 172.21.15.60:0/1802243131 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5a800ce20 con 0x7fa5b8102380 2024-02-26T04:59:02.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.800+0000 7fa5b4ff9640 1 -- 172.21.15.60:0/1802243131 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8018840 con 0x7fa5b8102380 2024-02-26T04:59:02.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.800+0000 7fa5be0a2640 1 -- 172.21.15.60:0/1802243131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b81a1ad0 con 0x7fa5b8102380 2024-02-26T04:59:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.802+0000 7fa5b4ff9640 1 -- 172.21.15.60:0/1802243131 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fa5a80189a0 con 0x7fa5b8102380 2024-02-26T04:59:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5b4ff9640 1 --2- 172.21.15.60:0/1802243131 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa5a00a20c0 0x7fa5a00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5c0e76740 1 Processor -- start 2024-02-26T04:59:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5c0e76740 1 -- start start 2024-02-26T04:59:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5b77fe640 1 --2- 172.21.15.60:0/1802243131 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa5a00a20c0 0x7fa5a00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5b4ff9640 1 -- 172.21.15.60:0/1802243131 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fa5a8014070 con 0x7fa5b8102380 2024-02-26T04:59:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5c0e76740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920240 0x557fc8920660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5c0e76740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920c30 0x557fc8925730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5c0e76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8925da0 0x557fc8928230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5c0e76740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc88dc930 con 0x557fc8920c30 2024-02-26T04:59:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5c0e76740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc88902e0 con 0x557fc8920240 2024-02-26T04:59:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5c0e76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc888d810 con 0x557fc8925da0 2024-02-26T04:59:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5b6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920c30 0x557fc8925730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5b6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920c30 0x557fc8925730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33920/0 (socket says 172.21.15.60:33920) 2024-02-26T04:59:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5b6ffd640 1 -- 172.21.15.60:0/1094387998 learned_addr learned my addr 172.21.15.60:0/1094387998 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5b77fe640 1 --2- 172.21.15.60:0/1094387998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920240 0x557fc8920660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5b77fe640 1 --2- 172.21.15.60:0/1802243131 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa5a00a20c0 0x7fa5a00a4580 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac005ab0 tx=0x7fa5ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5b6ffd640 1 -- 172.21.15.60:0/1094387998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8925da0 msgr2=0x557fc8928230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5b6ffd640 1 --2- 172.21.15.60:0/1094387998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8925da0 0x557fc8928230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5b6ffd640 1 -- 172.21.15.60:0/1094387998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920240 msgr2=0x557fc8920660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5b6ffd640 1 --2- 172.21.15.60:0/1094387998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920240 0x557fc8920660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.803+0000 7fa5b6ffd640 1 -- 172.21.15.60:0/1094387998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fc884b030 con 0x557fc8920c30 2024-02-26T04:59:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.804+0000 7fa5b6ffd640 1 --2- 172.21.15.60:0/1094387998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920c30 0x557fc8925730 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8097110 tx=0x7fa5a809a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2452d481e2b05a7d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.804+0000 7fa59dffb640 1 -- 172.21.15.60:0/1094387998 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80ce1b0 con 0x557fc8920c30 2024-02-26T04:59:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.804+0000 7fa59dffb640 1 -- 172.21.15.60:0/1094387998 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5a80cbe90 con 0x557fc8920c30 2024-02-26T04:59:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.804+0000 7fa59dffb640 1 -- 172.21.15.60:0/1094387998 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a809a100 con 0x557fc8920c30 2024-02-26T04:59:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.805+0000 7fa5c0e76740 1 -- 172.21.15.60:0/1094387998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920c30 msgr2=0x557fc8925730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.805+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/1094387998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920c30 0x557fc8925730 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8097110 tx=0x7fa5a809a8b0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.805+0000 7fa5c0e76740 1 -- 172.21.15.60:0/1094387998 shutdown_connections 2024-02-26T04:59:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.805+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/1094387998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8925da0 0x557fc8928230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.805+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/1094387998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920c30 0x557fc8925730 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.805+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/1094387998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920240 0x557fc8920660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.805+0000 7fa5c0e76740 1 -- 172.21.15.60:0/1094387998 >> 172.21.15.60:0/1094387998 conn(0x557fc891b7b0 msgr2=0x557fc891bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.805+0000 7fa5c0e76740 1 -- 172.21.15.60:0/1094387998 shutdown_connections 2024-02-26T04:59:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.805+0000 7fa5c0e76740 1 -- 172.21.15.60:0/1094387998 wait complete. 2024-02-26T04:59:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.805+0000 7fa5c0e76740 1 -- start start 2024-02-26T04:59:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.805+0000 7fa5c0e76740 1 Processor -- start 2024-02-26T04:59:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5c0e76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8920240 0x557fc892a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5c0e76740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920c30 0x557fc892a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5c0e76740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8925da0 0x557fc8931a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5c0e76740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc8888ff0 con 0x557fc8920c30 2024-02-26T04:59:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5c0e76740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc855c430 con 0x557fc8925da0 2024-02-26T04:59:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5c0e76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc8518e00 con 0x557fc8920240 2024-02-26T04:59:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5c0e76740 10 client.0 _ll_register_callbacks cb 0x557fc8830c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5b6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920c30 0x557fc892a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5b6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920c30 0x557fc892a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33926/0 (socket says 172.21.15.60:33926) 2024-02-26T04:59:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5b6ffd640 1 -- 172.21.15.60:0/2318341869 learned_addr learned my addr 172.21.15.60:0/2318341869 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5b7fff640 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8925da0 0x557fc8931a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5b77fe640 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8920240 0x557fc892a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5b6ffd640 1 -- 172.21.15.60:0/2318341869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8920240 msgr2=0x557fc892a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5b6ffd640 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8920240 0x557fc892a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5b6ffd640 1 -- 172.21.15.60:0/2318341869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8925da0 msgr2=0x557fc8931a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5b6ffd640 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8925da0 0x557fc8931a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5b6ffd640 1 -- 172.21.15.60:0/2318341869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b81a1ad0 con 0x557fc8920c30 2024-02-26T04:59:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5b77fe640 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8920240 0x557fc892a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5b7fff640 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8925da0 0x557fc8931a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa5b6ffd640 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920c30 0x557fc892a9c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80a2e70 tx=0x7fa5a80a33a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b03764d34a7015 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.806+0000 7fa596ffd640 1 -- 172.21.15.60:0/2318341869 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a809ed50 con 0x557fc8920c30 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.807+0000 7fa596ffd640 1 -- 172.21.15.60:0/2318341869 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fa5a809a310 con 0x557fc8920c30 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.807+0000 7fa5c0e76740 1 -- 172.21.15.60:0/2318341869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fc85887e0 con 0x557fc8920c30 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.807+0000 7fa596ffd640 1 -- 172.21.15.60:0/2318341869 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5a8079b20 con 0x557fc8920c30 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.807+0000 7fa5c0e76740 10 client.15783 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.807+0000 7fa5c0e76740 1 -- 172.21.15.60:0/2318341869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557fc8887670 con 0x557fc8920c30 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.807+0000 7fa596ffd640 1 -- 172.21.15.60:0/2318341869 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557fc85887e0 con 0x557fc8920c30 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.807+0000 7fa596ffd640 1 -- 172.21.15.60:0/2318341869 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa5a80852c0 con 0x557fc8920c30 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.807+0000 7fa5c0e76740 10 client.15783 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.807+0000 7fa5c0e76740 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.807+0000 7fa5c0e76740 20 client.15783 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.807+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x557fc89361f0 0x557fc89385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.807+0000 7fa5c0e76740 4 client.15783 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:59:02.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.807+0000 7fa5c0e76740 1 -- 172.21.15.60:0/2318341869 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fc85c5720 con 0x557fc89361f0 2024-02-26T04:59:02.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.807+0000 7fa5b77fe640 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x557fc89361f0 0x557fc89385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.808+0000 7fa596ffd640 10 client.15783 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:02.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.808+0000 7fa5b77fe640 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x557fc89361f0 0x557fc89385e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44daec346e2871e7 server_cookie=3ef56832c742fb9f in_seq=0 out_seq=0 2024-02-26T04:59:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa596ffd640 1 -- 172.21.15.60:0/2318341869 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557fc85c5720 con 0x557fc89361f0 2024-02-26T04:59:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa596ffd640 10 client.15783 handle_command_reply: tid=0 2024-02-26T04:59:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa5c0e76740 1 client.15783 shutdown 2024-02-26T04:59:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa5c0e76740 1 -- 172.21.15.60:0/2318341869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920c30 msgr2=0x557fc892a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920c30 0x557fc892a9c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80a2e70 tx=0x7fa5a80a33a0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa5c0e76740 1 -- 172.21.15.60:0/2318341869 shutdown_connections 2024-02-26T04:59:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x557fc89361f0 0x557fc89385e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8925da0 0x557fc8931a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920c30 0x557fc892a9c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa596ffd640 0 client.15783 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/2318341869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8920240 0x557fc892a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa5c0e76740 1 -- 172.21.15.60:0/2318341869 >> 172.21.15.60:0/2318341869 conn(0x557fc891b7b0 msgr2=0x557fc891ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa5c0e76740 1 -- 172.21.15.60:0/2318341869 shutdown_connections 2024-02-26T04:59:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa5c0e76740 1 -- 172.21.15.60:0/2318341869 wait complete. 2024-02-26T04:59:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.810+0000 7fa5c0e76740 20 client.15783 trim_cache size 0 max 16384 2024-02-26T04:59:02.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.823+0000 7fa5c0e76740 1 Processor -- start 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.823+0000 7fa5c0e76740 1 -- start start 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.823+0000 7fa5c0e76740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920240 0x557fc8928f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.823+0000 7fa5c0e76740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920c30 0x557fc89319c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.823+0000 7fa5c0e76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8925da0 0x557fc8929450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.823+0000 7fa5c0e76740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc880e100 con 0x557fc8920240 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.823+0000 7fa5c0e76740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc89283c0 con 0x557fc8920c30 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.823+0000 7fa5c0e76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc891bdc0 con 0x557fc8925da0 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.823+0000 7fa5b77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920240 0x557fc8928f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.823+0000 7fa5b77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920240 0x557fc8928f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33928/0 (socket says 172.21.15.60:33928) 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.823+0000 7fa5b77fe640 1 -- 172.21.15.60:0/752367862 learned_addr learned my addr 172.21.15.60:0/752367862 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.823+0000 7fa5b6ffd640 1 --2- 172.21.15.60:0/752367862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920c30 0x557fc89319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.824+0000 7fa5b77fe640 1 -- 172.21.15.60:0/752367862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8925da0 msgr2=0x557fc8929450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.824+0000 7fa5b77fe640 1 --2- 172.21.15.60:0/752367862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8925da0 0x557fc8929450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.824+0000 7fa5b77fe640 1 -- 172.21.15.60:0/752367862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920c30 msgr2=0x557fc89319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.824+0000 7fa5b77fe640 1 --2- 172.21.15.60:0/752367862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920c30 0x557fc89319c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.824+0000 7fa5b77fe640 1 -- 172.21.15.60:0/752367862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fc8823bf0 con 0x557fc8920240 2024-02-26T04:59:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.824+0000 7fa5b77fe640 1 --2- 172.21.15.60:0/752367862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920240 0x557fc8928f10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0048a0 tx=0x7fa5ac03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689873343f07619 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.824+0000 7fa59d7fa640 1 -- 172.21.15.60:0/752367862 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac004040 con 0x557fc8920240 2024-02-26T04:59:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.824+0000 7fa59d7fa640 1 -- 172.21.15.60:0/752367862 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5ac051e90 con 0x557fc8920240 2024-02-26T04:59:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.824+0000 7fa59d7fa640 1 -- 172.21.15.60:0/752367862 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac003850 con 0x557fc8920240 2024-02-26T04:59:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.825+0000 7fa5c0e76740 1 -- 172.21.15.60:0/752367862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920240 msgr2=0x557fc8928f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.825+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/752367862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920240 0x557fc8928f10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0048a0 tx=0x7fa5ac03f000 comp rx=0 tx=0).stop 2024-02-26T04:59:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.825+0000 7fa5c0e76740 1 -- 172.21.15.60:0/752367862 shutdown_connections 2024-02-26T04:59:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.825+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/752367862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8925da0 0x557fc8929450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.825+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/752367862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920c30 0x557fc89319c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.825+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/752367862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920240 0x557fc8928f10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.825+0000 7fa5c0e76740 1 -- 172.21.15.60:0/752367862 >> 172.21.15.60:0/752367862 conn(0x557fc891b7b0 msgr2=0x557fc89221c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.825+0000 7fa5c0e76740 1 -- 172.21.15.60:0/752367862 shutdown_connections 2024-02-26T04:59:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.825+0000 7fa5c0e76740 1 -- 172.21.15.60:0/752367862 wait complete. 2024-02-26T04:59:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.826+0000 7fa5c0e76740 1 -- start start 2024-02-26T04:59:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.826+0000 7fa5c0e76740 1 Processor -- start 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.826+0000 7fa5c0e76740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920240 0x557fc8931950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.826+0000 7fa5c0e76740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920c30 0x557fc892ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.826+0000 7fa5c0e76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8925da0 0x557fc8930540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.826+0000 7fa5c0e76740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc8916f40 con 0x557fc8920240 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.826+0000 7fa5c0e76740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc8930cc0 con 0x557fc8920c30 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.826+0000 7fa5c0e76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc898c670 con 0x557fc8925da0 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.826+0000 7fa5c0e76740 10 client.0 _ll_register_callbacks cb 0x7fa5a8010dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.826+0000 7fa5b77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920240 0x557fc8931950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.826+0000 7fa5b77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920240 0x557fc8931950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33942/0 (socket says 172.21.15.60:33942) 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.826+0000 7fa5b77fe640 1 -- 172.21.15.60:0/4193668590 learned_addr learned my addr 172.21.15.60:0/4193668590 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.826+0000 7fa5b6ffd640 1 --2- 172.21.15.60:0/4193668590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920c30 0x557fc892ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.827+0000 7fa5b6ffd640 1 -- 172.21.15.60:0/4193668590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8925da0 msgr2=0x557fc8930540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.827+0000 7fa5b6ffd640 1 --2- 172.21.15.60:0/4193668590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8925da0 0x557fc8930540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.827+0000 7fa5b6ffd640 1 -- 172.21.15.60:0/4193668590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920240 msgr2=0x557fc8931950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.827+0000 7fa5b6ffd640 1 --2- 172.21.15.60:0/4193668590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920240 0x557fc8931950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.827+0000 7fa5b6ffd640 1 -- 172.21.15.60:0/4193668590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b8109ed0 con 0x557fc8920c30 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.827+0000 7fa5b77fe640 1 --2- 172.21.15.60:0/4193668590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920240 0x557fc8931950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.827+0000 7fa5b6ffd640 1 --2- 172.21.15.60:0/4193668590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920c30 0x557fc892ff60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8010e70 tx=0x7fa5a80a00d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f3ab9a113f7aedf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.827+0000 7fa596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.827+0000 7fa596ffd640 1 -- 172.21.15.60:0/4193668590 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8085740 con 0x557fc8920c30 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.827+0000 7fa596ffd640 1 -- 172.21.15.60:0/4193668590 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fa5a80ce1b0 con 0x557fc8920c30 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.827+0000 7fa5c0e76740 1 -- 172.21.15.60:0/4193668590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5a80cbe90 con 0x557fc8920c30 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.827+0000 7fa596ffd640 1 -- 172.21.15.60:0/4193668590 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5a80a3410 con 0x557fc8920c30 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.827+0000 7fa5c0e76740 10 client.25388 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.827+0000 7fa5c0e76740 1 -- 172.21.15.60:0/4193668590 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557fc8849890 con 0x557fc8920c30 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.828+0000 7fa596ffd640 1 -- 172.21.15.60:0/4193668590 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5a809aa80 con 0x557fc8920c30 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.828+0000 7fa596ffd640 1 -- 172.21.15.60:0/4193668590 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa5a8099de0 con 0x557fc8920c30 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.828+0000 7fa5c0e76740 10 client.25388 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.828+0000 7fa5c0e76740 10 client.25388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.828+0000 7fa5c0e76740 20 client.25388 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.828+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/4193668590 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x557fc898d9e0 0x557fc898fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.828+0000 7fa5c0e76740 4 client.25388 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.828+0000 7fa5c0e76740 1 -- 172.21.15.60:0/4193668590 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fa5a80cbe90 con 0x557fc898d9e0 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.828+0000 7fa5b77fe640 1 --2- 172.21.15.60:0/4193668590 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x557fc898d9e0 0x557fc898fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.828+0000 7fa5b77fe640 1 --2- 172.21.15.60:0/4193668590 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x557fc898d9e0 0x557fc898fe00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94b68df9676e1ab5 server_cookie=2cc6aa199a78e919 in_seq=0 out_seq=0 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.828+0000 7fa596ffd640 10 client.25388 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa596ffd640 1 -- 172.21.15.60:0/4193668590 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fa5a80cbe90 con 0x557fc898d9e0 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa596ffd640 10 client.25388 handle_command_reply: tid=0 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa5c0e76740 1 client.25388 shutdown 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa5c0e76740 1 -- 172.21.15.60:0/4193668590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920c30 msgr2=0x557fc892ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/4193668590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920c30 0x557fc892ff60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8010e70 tx=0x7fa5a80a00d0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa5c0e76740 1 -- 172.21.15.60:0/4193668590 shutdown_connections 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/4193668590 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x557fc898d9e0 0x557fc898fe00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/4193668590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557fc8925da0 0x557fc8930540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/4193668590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fc8920c30 0x557fc892ff60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa596ffd640 0 client.25388 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa5c0e76740 1 --2- 172.21.15.60:0/4193668590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557fc8920240 0x557fc8931950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa5c0e76740 1 -- 172.21.15.60:0/4193668590 >> 172.21.15.60:0/4193668590 conn(0x557fc891b7b0 msgr2=0x557fc892a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa5c0e76740 1 -- 172.21.15.60:0/4193668590 shutdown_connections 2024-02-26T04:59:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa5c0e76740 1 -- 172.21.15.60:0/4193668590 wait complete. 2024-02-26T04:59:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.829+0000 7fa5c0e76740 20 client.25388 trim_cache size 0 max 16384 2024-02-26T04:59:02.836 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T04:59:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.830+0000 7fa5be0a2640 1 -- 172.21.15.60:0/1802243131 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa5a00a20c0 msgr2=0x7fa5a00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.830+0000 7fa5be0a2640 1 --2- 172.21.15.60:0/1802243131 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa5a00a20c0 0x7fa5a00a4580 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac005ab0 tx=0x7fa5ac005fd0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.830+0000 7fa5be0a2640 1 -- 172.21.15.60:0/1802243131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5b8102380 msgr2=0x7fa5b8073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.830+0000 7fa5be0a2640 1 --2- 172.21.15.60:0/1802243131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5b8102380 0x7fa5b8073570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800ea20 tx=0x7fa5a800eee0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.830+0000 7fa5be0a2640 1 -- 172.21.15.60:0/1802243131 shutdown_connections 2024-02-26T04:59:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.830+0000 7fa5be0a2640 1 --2- 172.21.15.60:0/1802243131 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fa5a00a20c0 0x7fa5a00a4580 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.830+0000 7fa5be0a2640 1 --2- 172.21.15.60:0/1802243131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b8102d40 0x7fa5b8073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.830+0000 7fa5be0a2640 1 --2- 172.21.15.60:0/1802243131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5b8102380 0x7fa5b8073570 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.830+0000 7fa5be0a2640 1 --2- 172.21.15.60:0/1802243131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b8101190 0x7fa5b8074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.830+0000 7fa5be0a2640 1 -- 172.21.15.60:0/1802243131 >> 172.21.15.60:0/1802243131 conn(0x7fa5b80fc6e0 msgr2=0x7fa5b8105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.830+0000 7fa5be0a2640 1 -- 172.21.15.60:0/1802243131 shutdown_connections 2024-02-26T04:59:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:02.830+0000 7fa5be0a2640 1 -- 172.21.15.60:0/1802243131 wait complete. 2024-02-26T04:59:02.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:02 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2465249621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:59:03.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:02 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2465249621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:59:03.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:02 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2465249621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:59:03.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-02-26T04:59:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-02-26T04:59:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-02-26T04:59:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-02-26T04:59:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-02-26T04:59:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-02-26T04:59:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-02-26T04:59:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-02-26T04:59:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-02-26T04:59:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-02-26T04:59:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-02-26T04:59:03.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-02-26T04:59:03.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-02-26T04:59:03.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-02-26T04:59:03.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-02-26T04:59:03.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-02-26T04:59:03.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-02-26T04:59:03.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-02-26T04:59:03.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-02-26T04:59:03.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-02-26T04:59:03.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-02-26T04:59:03.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-02-26T04:59:03.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-02-26T04:59:03.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-02-26T04:59:03.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-02-26T04:59:03.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-02-26T04:59:03.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-02-26T04:59:03.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-02-26T04:59:03.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-02-26T04:59:03.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-02-26T04:59:03.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-02-26T04:59:03.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-02-26T04:59:03.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-02-26T04:59:03.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-02-26T04:59:03.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-02-26T04:59:03.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-02-26T04:59:03.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-02-26T04:59:03.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-02-26T04:59:03.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-02-26T04:59:03.475 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-02-26T04:59:03.475 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-02-26T04:59:03.475 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-02-26T04:59:03.475 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-02-26T04:59:03.475 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-02-26T04:59:03.475 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-02-26T04:59:03.475 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-02-26T04:59:03.475 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-02-26T04:59:03.475 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-02-26T04:59:03.475 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-02-26T04:59:03.475 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-02-26T04:59:03.475 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-02-26T04:59:03.475 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-02-26T04:59:03.475 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-02-26T04:59:03.476 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-02-26T04:59:03.476 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-02-26T04:59:03.476 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-02-26T04:59:03.476 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-02-26T04:59:03.476 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-02-26T04:59:03.476 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-02-26T04:59:03.476 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-02-26T04:59:03.476 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-02-26T04:59:03.476 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-02-26T04:59:03.476 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-02-26T04:59:03.476 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-02-26T04:59:03.476 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-02-26T04:59:03.476 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-02-26T04:59:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-02-26T04:59:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-02-26T04:59:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-02-26T04:59:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-02-26T04:59:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-02-26T04:59:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-02-26T04:59:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-02-26T04:59:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-02-26T04:59:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-02-26T04:59:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-02-26T04:59:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-02-26T04:59:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-02-26T04:59:03.477 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-02-26T04:59:03.478 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-02-26T04:59:03.846 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-26T04:59:03.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T04:59:03.847 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-26T04:59:03.906 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-02-26T04:59:03.907 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-02-26T04:59:03.907 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-02-26T04:59:03.907 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-02-26T04:59:03.907 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-02-26T04:59:03.907 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-02-26T04:59:03.907 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-02-26T04:59:03.907 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-02-26T04:59:03.907 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-02-26T04:59:03.907 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-02-26T04:59:03.907 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-02-26T04:59:03.907 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-02-26T04:59:03.907 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-02-26T04:59:03.907 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-02-26T04:59:03.907 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-02-26T04:59:03.908 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-02-26T04:59:03.908 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-02-26T04:59:03.908 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-02-26T04:59:03.908 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-02-26T04:59:03.908 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-02-26T04:59:03.908 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-02-26T04:59:03.908 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-02-26T04:59:03.908 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-02-26T04:59:03.908 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-02-26T04:59:03.908 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-02-26T04:59:03.908 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-02-26T04:59:03.908 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-02-26T04:59:03.908 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-02-26T04:59:03.908 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-02-26T04:59:03.909 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-02-26T04:59:03.910 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-02-26T04:59:03.911 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-02-26T04:59:03.912 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-02-26T04:59:03.912 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-02-26T04:59:03.912 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-02-26T04:59:03.912 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-02-26T04:59:03.912 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-02-26T04:59:03.912 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-02-26T04:59:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.957+0000 7f0c1e174640 1 -- 172.21.15.60:0/1141454680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c18101270 msgr2=0x7f0c181016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.957+0000 7f0c1e174640 1 --2- 172.21.15.60:0/1141454680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c18101270 0x7f0c181016f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009900 tx=0x7f0c0c02bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.957+0000 7f0c1e174640 1 -- 172.21.15.60:0/1141454680 shutdown_connections 2024-02-26T04:59:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.957+0000 7f0c1e174640 1 --2- 172.21.15.60:0/1141454680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c18101c30 0x7f0c181084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.957+0000 7f0c1e174640 1 --2- 172.21.15.60:0/1141454680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c18101270 0x7f0c181016f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.957+0000 7f0c1e174640 1 --2- 172.21.15.60:0/1141454680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c18100080 0x7f0c181004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.957+0000 7f0c1e174640 1 -- 172.21.15.60:0/1141454680 >> 172.21.15.60:0/1141454680 conn(0x7f0c180fb610 msgr2=0x7f0c180fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.957+0000 7f0c1e174640 1 -- 172.21.15.60:0/1141454680 shutdown_connections 2024-02-26T04:59:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.957+0000 7f0c1e174640 1 -- 172.21.15.60:0/1141454680 wait complete. 2024-02-26T04:59:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c1e174640 1 Processor -- start 2024-02-26T04:59:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c1e174640 1 -- start start 2024-02-26T04:59:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c1e174640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c18100080 0x7f0c18197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c1e174640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c18101270 0x7f0c181977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c1e174640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c18101c30 0x7f0c1819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c1e174640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1819ed10 con 0x7f0c18100080 2024-02-26T04:59:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c1e174640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1819ee80 con 0x7f0c18101c30 2024-02-26T04:59:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c1e174640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1819f040 con 0x7f0c18101270 2024-02-26T04:59:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c18100080 0x7f0c18197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c18100080 0x7f0c18197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33962/0 (socket says 172.21.15.60:33962) 2024-02-26T04:59:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c17fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c18101c30 0x7f0c1819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c177fe640 1 -- 172.21.15.60:0/3214403411 learned_addr learned my addr 172.21.15.60:0/3214403411 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c16ffd640 1 --2- 172.21.15.60:0/3214403411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c18101270 0x7f0c181977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c17fff640 1 -- 172.21.15.60:0/3214403411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c18101270 msgr2=0x7f0c181977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c17fff640 1 --2- 172.21.15.60:0/3214403411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c18101270 0x7f0c181977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c17fff640 1 -- 172.21.15.60:0/3214403411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c18100080 msgr2=0x7f0c18197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c17fff640 1 --2- 172.21.15.60:0/3214403411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c18100080 0x7f0c18197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c17fff640 1 -- 172.21.15.60:0/3214403411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c009560 con 0x7f0c18101c30 2024-02-26T04:59:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.958+0000 7f0c177fe640 1 --2- 172.21.15.60:0/3214403411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c18100080 0x7f0c18197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:59:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.959+0000 7f0c17fff640 1 --2- 172.21.15.60:0/3214403411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c18101c30 0x7f0c1819e7d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009fd0 tx=0x7f0c1000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.959+0000 7f0c14ff9640 1 -- 172.21.15.60:0/3214403411 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1000ce20 con 0x7f0c18101c30 2024-02-26T04:59:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.959+0000 7f0c1e174640 1 -- 172.21.15.60:0/3214403411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1819f320 con 0x7f0c18101c30 2024-02-26T04:59:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.959+0000 7f0c14ff9640 1 -- 172.21.15.60:0/3214403411 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c10005020 con 0x7f0c18101c30 2024-02-26T04:59:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.959+0000 7f0c14ff9640 1 -- 172.21.15.60:0/3214403411 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10010690 con 0x7f0c18101c30 2024-02-26T04:59:03.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.959+0000 7f0c1e174640 1 -- 172.21.15.60:0/3214403411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1819f850 con 0x7f0c18101c30 2024-02-26T04:59:03.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.961+0000 7f0c14ff9640 1 -- 172.21.15.60:0/3214403411 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0c10005500 con 0x7f0c18101c30 2024-02-26T04:59:03.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c14ff9640 1 --2- 172.21.15.60:0/3214403411 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0c000a20c0 0x7f0c000a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c21021740 1 Processor -- start 2024-02-26T04:59:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c21021740 1 -- start start 2024-02-26T04:59:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c14ff9640 1 -- 172.21.15.60:0/3214403411 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f0c10014070 con 0x7f0c18101c30 2024-02-26T04:59:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c21021740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2ef490 0x55abcd2ef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c21021740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f0660 0x55abcd2f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c21021740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f1020 0x55abcd2f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c21021740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abcd25ef60 con 0x55abcd2ef490 2024-02-26T04:59:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c21021740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abcd25c490 con 0x55abcd2f0660 2024-02-26T04:59:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c177fe640 1 --2- 172.21.15.60:0/3214403411 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0c000a20c0 0x7f0c000a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c21021740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abcd257c70 con 0x55abcd2f1020 2024-02-26T04:59:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c16ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f0660 0x55abcd2f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c16ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f0660 0x55abcd2f0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.60:45080/0 (socket says 172.21.15.60:45080) 2024-02-26T04:59:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c16ffd640 1 -- 172.21.15.60:0/3045175243 learned_addr learned my addr 172.21.15.60:0/3045175243 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c177fe640 1 --2- 172.21.15.60:0/3045175243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2ef490 0x55abcd2ef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.962+0000 7f0c17fff640 1 --2- 172.21.15.60:0/3045175243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f1020 0x55abcd2f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.963+0000 7f0c16ffd640 1 -- 172.21.15.60:0/3045175243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f1020 msgr2=0x55abcd2f78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.963+0000 7f0c16ffd640 1 --2- 172.21.15.60:0/3045175243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f1020 0x55abcd2f78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.963+0000 7f0c16ffd640 1 -- 172.21.15.60:0/3045175243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2ef490 msgr2=0x55abcd2ef8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.963+0000 7f0c16ffd640 1 --2- 172.21.15.60:0/3045175243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2ef490 0x55abcd2ef8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.963+0000 7f0c16ffd640 1 -- 172.21.15.60:0/3045175243 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abcd21a080 con 0x55abcd2f0660 2024-02-26T04:59:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.963+0000 7f0c17fff640 1 --2- 172.21.15.60:0/3045175243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f1020 0x55abcd2f78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:59:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.963+0000 7f0c177fe640 1 --2- 172.21.15.60:0/3214403411 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0c000a20c0 0x7f0c000a4580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0c080058c0 tx=0x7f0c08013cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.963+0000 7f0c16ffd640 1 --2- 172.21.15.60:0/3045175243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f0660 0x55abcd2f0ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009900 tx=0x7f0c0c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3328ddb3b343a829 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.963+0000 7f0bfdffb640 1 -- 172.21.15.60:0/3045175243 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c02c960 con 0x55abcd2f0660 2024-02-26T04:59:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.963+0000 7f0bfdffb640 1 -- 172.21.15.60:0/3045175243 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c0c02cac0 con 0x55abcd2f0660 2024-02-26T04:59:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.963+0000 7f0bfdffb640 1 -- 172.21.15.60:0/3045175243 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0050c0 con 0x55abcd2f0660 2024-02-26T04:59:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.964+0000 7f0c21021740 1 -- 172.21.15.60:0/3045175243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f0660 msgr2=0x55abcd2f0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.964+0000 7f0c21021740 1 --2- 172.21.15.60:0/3045175243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f0660 0x55abcd2f0ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009900 tx=0x7f0c0c005d50 comp rx=0 tx=0).stop 2024-02-26T04:59:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.964+0000 7f0c21021740 1 -- 172.21.15.60:0/3045175243 shutdown_connections 2024-02-26T04:59:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.964+0000 7f0c21021740 1 --2- 172.21.15.60:0/3045175243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f1020 0x55abcd2f78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.964+0000 7f0c21021740 1 --2- 172.21.15.60:0/3045175243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f0660 0x55abcd2f0ae0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.964+0000 7f0c21021740 1 --2- 172.21.15.60:0/3045175243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2ef490 0x55abcd2ef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.964+0000 7f0c21021740 1 -- 172.21.15.60:0/3045175243 >> 172.21.15.60:0/3045175243 conn(0x55abcd2ea670 msgr2=0x55abcd2eaa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.964+0000 7f0c21021740 1 -- 172.21.15.60:0/3045175243 shutdown_connections 2024-02-26T04:59:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.964+0000 7f0c21021740 1 -- 172.21.15.60:0/3045175243 wait complete. 2024-02-26T04:59:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.964+0000 7f0c21021740 1 -- start start 2024-02-26T04:59:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.964+0000 7f0c21021740 1 Processor -- start 2024-02-26T04:59:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c21021740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2ef490 0x55abcd2f95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c21021740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2f0660 0x55abcd2f9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c21021740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f1020 0x55abcd300b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c21021740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abcd1fcf10 con 0x55abcd2f0660 2024-02-26T04:59:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c21021740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abccf2b4b0 con 0x55abcd2f1020 2024-02-26T04:59:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c21021740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abcd1dd640 con 0x55abcd2ef490 2024-02-26T04:59:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c21021740 10 client.0 _ll_register_callbacks cb 0x55abcd24af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c16ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2f0660 0x55abcd2f9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c16ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2f0660 0x55abcd2f9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33990/0 (socket says 172.21.15.60:33990) 2024-02-26T04:59:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c16ffd640 1 -- 172.21.15.60:0/3398416546 learned_addr learned my addr 172.21.15.60:0/3398416546 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c17fff640 1 --2- 172.21.15.60:0/3398416546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f1020 0x55abcd300b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c177fe640 1 --2- 172.21.15.60:0/3398416546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2ef490 0x55abcd2f95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c16ffd640 1 -- 172.21.15.60:0/3398416546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2ef490 msgr2=0x55abcd2f95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c16ffd640 1 --2- 172.21.15.60:0/3398416546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2ef490 0x55abcd2f95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c16ffd640 1 -- 172.21.15.60:0/3398416546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f1020 msgr2=0x55abcd300b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c16ffd640 1 --2- 172.21.15.60:0/3398416546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f1020 0x55abcd300b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c16ffd640 1 -- 172.21.15.60:0/3398416546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c18108dc0 con 0x55abcd2f0660 2024-02-26T04:59:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c16ffd640 1 --2- 172.21.15.60:0/3398416546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2f0660 0x55abcd2f9b10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c004970 tx=0x7f0c0c0049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02797d94e294551 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0bf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0bf6ffd640 1 -- 172.21.15.60:0/3398416546 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c005270 con 0x55abcd2f0660 2024-02-26T04:59:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0c21021740 1 -- 172.21.15.60:0/3398416546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abcd231f10 con 0x55abcd2f0660 2024-02-26T04:59:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.965+0000 7f0bf6ffd640 1 -- 172.21.15.60:0/3398416546 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f0c0c04d070 con 0x55abcd2f0660 2024-02-26T04:59:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.966+0000 7f0bf6ffd640 1 -- 172.21.15.60:0/3398416546 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c0c0053d0 con 0x55abcd2f0660 2024-02-26T04:59:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.966+0000 7f0c21021740 10 client.15801 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.966+0000 7f0c21021740 1 -- 172.21.15.60:0/3398416546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55abcd2562f0 con 0x55abcd2f0660 2024-02-26T04:59:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.966+0000 7f0bf6ffd640 1 -- 172.21.15.60:0/3398416546 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55abcd231f10 con 0x55abcd2f0660 2024-02-26T04:59:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.966+0000 7f0bf6ffd640 1 -- 172.21.15.60:0/3398416546 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0c0c03c050 con 0x55abcd2f0660 2024-02-26T04:59:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.966+0000 7f0c21021740 10 client.15801 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.966+0000 7f0c21021740 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:59:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.966+0000 7f0c21021740 20 client.15801 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.966+0000 7f0c21021740 1 --2- 172.21.15.60:0/3398416546 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55abcd305430 0x55abcd307850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.966+0000 7f0c21021740 4 client.15801 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:59:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.966+0000 7f0c21021740 1 -- 172.21.15.60:0/3398416546 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abcd24e2e0 con 0x55abcd305430 2024-02-26T04:59:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.967+0000 7f0c177fe640 1 --2- 172.21.15.60:0/3398416546 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55abcd305430 0x55abcd307850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.967+0000 7f0c177fe640 1 --2- 172.21.15.60:0/3398416546 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55abcd305430 0x55abcd307850 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1db06d7bdd3ea30 server_cookie=1976be879fd30716 in_seq=0 out_seq=0 2024-02-26T04:59:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.967+0000 7f0bf6ffd640 10 client.15801 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0bf6ffd640 1 -- 172.21.15.60:0/3398416546 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55abcd24e2e0 con 0x55abcd305430 2024-02-26T04:59:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0bf6ffd640 10 client.15801 handle_command_reply: tid=0 2024-02-26T04:59:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0c21021740 1 client.15801 shutdown 2024-02-26T04:59:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0c21021740 1 -- 172.21.15.60:0/3398416546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2f0660 msgr2=0x55abcd2f9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0c21021740 1 --2- 172.21.15.60:0/3398416546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2f0660 0x55abcd2f9b10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c004970 tx=0x7f0c0c0049a0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0c21021740 1 -- 172.21.15.60:0/3398416546 shutdown_connections 2024-02-26T04:59:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0c21021740 1 --2- 172.21.15.60:0/3398416546 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55abcd305430 0x55abcd307850 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0c21021740 1 --2- 172.21.15.60:0/3398416546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f1020 0x55abcd300b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0c21021740 1 --2- 172.21.15.60:0/3398416546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2f0660 0x55abcd2f9b10 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0c21021740 1 --2- 172.21.15.60:0/3398416546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2ef490 0x55abcd2f95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0c21021740 1 -- 172.21.15.60:0/3398416546 >> 172.21.15.60:0/3398416546 conn(0x55abcd2ea670 msgr2=0x55abcd2e98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0bf6ffd640 0 client.15801 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0c21021740 1 -- 172.21.15.60:0/3398416546 shutdown_connections 2024-02-26T04:59:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0c21021740 1 -- 172.21.15.60:0/3398416546 wait complete. 2024-02-26T04:59:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.968+0000 7f0c21021740 20 client.15801 trim_cache size 0 max 16384 2024-02-26T04:59:03.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.984+0000 7f0c21021740 1 Processor -- start 2024-02-26T04:59:03.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.984+0000 7f0c21021740 1 -- start start 2024-02-26T04:59:03.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.984+0000 7f0c21021740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2ef490 0x55abcd2f4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.984+0000 7f0c21021740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f0660 0x55abcd2f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.984+0000 7f0c21021740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2f1020 0x55abcd2e9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.984+0000 7f0c21021740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abcd24e2e0 con 0x55abcd2f1020 2024-02-26T04:59:03.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.984+0000 7f0c21021740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abcd231f10 con 0x55abcd2ef490 2024-02-26T04:59:03.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.984+0000 7f0c21021740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abcd2f03f0 con 0x55abcd2f0660 2024-02-26T04:59:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.984+0000 7f0c17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2f1020 0x55abcd2e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.984+0000 7f0c17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2f1020 0x55abcd2e9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34000/0 (socket says 172.21.15.60:34000) 2024-02-26T04:59:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.984+0000 7f0c17fff640 1 -- 172.21.15.60:0/3612177693 learned_addr learned my addr 172.21.15.60:0/3612177693 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.984+0000 7f0c16ffd640 1 --2- 172.21.15.60:0/3612177693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f0660 0x55abcd2f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.984+0000 7f0c177fe640 1 --2- 172.21.15.60:0/3612177693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2ef490 0x55abcd2f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.985+0000 7f0c17fff640 1 -- 172.21.15.60:0/3612177693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f0660 msgr2=0x55abcd2f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.985+0000 7f0c17fff640 1 --2- 172.21.15.60:0/3612177693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f0660 0x55abcd2f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.985+0000 7f0c17fff640 1 -- 172.21.15.60:0/3612177693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2ef490 msgr2=0x55abcd2f4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.985+0000 7f0c17fff640 1 --2- 172.21.15.60:0/3612177693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2ef490 0x55abcd2f4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.985+0000 7f0c17fff640 1 -- 172.21.15.60:0/3612177693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abcd1f2bd0 con 0x55abcd2f1020 2024-02-26T04:59:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.985+0000 7f0c17fff640 1 --2- 172.21.15.60:0/3612177693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2f1020 0x55abcd2e9020 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0c10000f80 tx=0x7f0c10097460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102e958503fa99b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.985+0000 7f0bfd7fa640 1 -- 172.21.15.60:0/3612177693 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100a3830 con 0x55abcd2f1020 2024-02-26T04:59:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.985+0000 7f0bfd7fa640 1 -- 172.21.15.60:0/3612177693 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c100ae070 con 0x55abcd2f1020 2024-02-26T04:59:03.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.985+0000 7f0bfd7fa640 1 -- 172.21.15.60:0/3612177693 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100aaa30 con 0x55abcd2f1020 2024-02-26T04:59:03.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 -- 172.21.15.60:0/3612177693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2f1020 msgr2=0x55abcd2e9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:03.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 --2- 172.21.15.60:0/3612177693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2f1020 0x55abcd2e9020 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0c10000f80 tx=0x7f0c10097460 comp rx=0 tx=0).stop 2024-02-26T04:59:03.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 -- 172.21.15.60:0/3612177693 shutdown_connections 2024-02-26T04:59:03.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 --2- 172.21.15.60:0/3612177693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2f1020 0x55abcd2e9020 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 --2- 172.21.15.60:0/3612177693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f0660 0x55abcd2f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 --2- 172.21.15.60:0/3612177693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2ef490 0x55abcd2f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 -- 172.21.15.60:0/3612177693 >> 172.21.15.60:0/3612177693 conn(0x55abcd2ea670 msgr2=0x55abcd2f95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:03.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 -- 172.21.15.60:0/3612177693 shutdown_connections 2024-02-26T04:59:03.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 -- 172.21.15.60:0/3612177693 wait complete. 2024-02-26T04:59:03.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 -- start start 2024-02-26T04:59:03.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 Processor -- start 2024-02-26T04:59:03.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2ef490 0x55abcd358330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f0660 0x55abcd358870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f1020 0x55abcd35f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abcd35fdf0 con 0x55abcd2ef490 2024-02-26T04:59:03.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abcd35ff30 con 0x55abcd2f0660 2024-02-26T04:59:03.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.986+0000 7f0c21021740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abcd3600a0 con 0x55abcd2f1020 2024-02-26T04:59:03.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0c21021740 10 client.0 _ll_register_callbacks cb 0x7f0c100093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:03.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0c177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2ef490 0x55abcd358330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0c16ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f0660 0x55abcd358870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0c177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2ef490 0x55abcd358330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34006/0 (socket says 172.21.15.60:34006) 2024-02-26T04:59:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0c17fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f1020 0x55abcd35f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0c177fe640 1 -- 172.21.15.60:0/2744829530 learned_addr learned my addr 172.21.15.60:0/2744829530 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0c17fff640 1 -- 172.21.15.60:0/2744829530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f0660 msgr2=0x55abcd358870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0c17fff640 1 --2- 172.21.15.60:0/2744829530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f0660 0x55abcd358870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0c17fff640 1 -- 172.21.15.60:0/2744829530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2ef490 msgr2=0x55abcd358330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0c17fff640 1 --2- 172.21.15.60:0/2744829530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2ef490 0x55abcd358330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0c17fff640 1 -- 172.21.15.60:0/2744829530 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1819f320 con 0x55abcd2f1020 2024-02-26T04:59:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0c177fe640 1 --2- 172.21.15.60:0/2744829530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2ef490 0x55abcd358330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:59:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0c17fff640 1 --2- 172.21.15.60:0/2744829530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f1020 0x55abcd35f8b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0c100cbeb0 tx=0x7f0c1009f320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae648dd35da6f9b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0bf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-02-26T04:59:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0bf6ffd640 1 -- 172.21.15.60:0/2744829530 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100a9070 con 0x55abcd2f1020 2024-02-26T04:59:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0bf6ffd640 1 -- 172.21.15.60:0/2744829530 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f0c100aaa30 con 0x55abcd2f1020 2024-02-26T04:59:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0c21021740 1 -- 172.21.15.60:0/2744829530 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abcd27d8d0 con 0x55abcd2f1020 2024-02-26T04:59:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.987+0000 7f0bf6ffd640 1 -- 172.21.15.60:0/2744829530 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c100ce610 con 0x55abcd2f1020 2024-02-26T04:59:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.988+0000 7f0c21021740 10 client.25357 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.988+0000 7f0c21021740 1 -- 172.21.15.60:0/2744829530 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55abcd2188e0 con 0x55abcd2f1020 2024-02-26T04:59:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.988+0000 7f0bf6ffd640 1 -- 172.21.15.60:0/2744829530 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c10082080 con 0x55abcd2f1020 2024-02-26T04:59:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.988+0000 7f0bf6ffd640 1 -- 172.21.15.60:0/2744829530 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0c1001f070 con 0x55abcd2f1020 2024-02-26T04:59:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.988+0000 7f0c21021740 10 client.25357 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.988+0000 7f0c21021740 10 client.25357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:59:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.988+0000 7f0c21021740 20 client.25357 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.988+0000 7f0c21021740 1 --2- 172.21.15.60:0/2744829530 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55abcd361900 0x55abcd363d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.988+0000 7f0c21021740 4 client.25357 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-26T04:59:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.988+0000 7f0c21021740 1 -- 172.21.15.60:0/2744829530 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55abccf57860 con 0x55abcd361900 2024-02-26T04:59:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.988+0000 7f0c177fe640 1 --2- 172.21.15.60:0/2744829530 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55abcd361900 0x55abcd363d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.989+0000 7f0bf6ffd640 10 client.25357 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.989+0000 7f0c177fe640 1 --2- 172.21.15.60:0/2744829530 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55abcd361900 0x55abcd363d20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2ea3aba93c52e1f server_cookie=d47c50bb8942b325 in_seq=0 out_seq=0 2024-02-26T04:59:04.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.989+0000 7f0bf6ffd640 1 -- 172.21.15.60:0/2744829530 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55abccf57860 con 0x55abcd361900 2024-02-26T04:59:04.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.989+0000 7f0bf6ffd640 10 client.25357 handle_command_reply: tid=0 2024-02-26T04:59:04.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.989+0000 7f0c21021740 1 client.25357 shutdown 2024-02-26T04:59:04.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c21021740 1 -- 172.21.15.60:0/2744829530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f1020 msgr2=0x55abcd35f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:04.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c21021740 1 --2- 172.21.15.60:0/2744829530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f1020 0x55abcd35f8b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0c100cbeb0 tx=0x7f0c1009f320 comp rx=0 tx=0).stop 2024-02-26T04:59:04.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c21021740 1 -- 172.21.15.60:0/2744829530 shutdown_connections 2024-02-26T04:59:04.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c21021740 1 --2- 172.21.15.60:0/2744829530 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55abcd361900 0x55abcd363d20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c21021740 1 --2- 172.21.15.60:0/2744829530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55abcd2f1020 0x55abcd35f8b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c21021740 1 --2- 172.21.15.60:0/2744829530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55abcd2f0660 0x55abcd358870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c21021740 1 --2- 172.21.15.60:0/2744829530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55abcd2ef490 0x55abcd358330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0bf6ffd640 0 client.25357 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:04.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c21021740 1 -- 172.21.15.60:0/2744829530 >> 172.21.15.60:0/2744829530 conn(0x55abcd2ea670 msgr2=0x55abcd2f9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:04.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c21021740 1 -- 172.21.15.60:0/2744829530 shutdown_connections 2024-02-26T04:59:04.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c21021740 1 -- 172.21.15.60:0/2744829530 wait complete. 2024-02-26T04:59:04.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c21021740 20 client.25357 trim_cache size 0 max 16384 2024-02-26T04:59:04.001 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:59:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-02-26T04:59:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "e07df569-17c8-4f1b-81e3-4f9111544967", 2024-02-26T04:59:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-02-26T04:59:04.001 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:59:04.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c1e174640 1 -- 172.21.15.60:0/3214403411 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0c000a20c0 msgr2=0x7f0c000a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:04.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c1e174640 1 --2- 172.21.15.60:0/3214403411 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0c000a20c0 0x7f0c000a4580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0c080058c0 tx=0x7f0c08013cf0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c1e174640 1 -- 172.21.15.60:0/3214403411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c18101c30 msgr2=0x7f0c1819e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:04.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c1e174640 1 --2- 172.21.15.60:0/3214403411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c18101c30 0x7f0c1819e7d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009fd0 tx=0x7f0c1000ef10 comp rx=0 tx=0).stop 2024-02-26T04:59:04.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c1e174640 1 -- 172.21.15.60:0/3214403411 shutdown_connections 2024-02-26T04:59:04.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c1e174640 1 --2- 172.21.15.60:0/3214403411 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f0c000a20c0 0x7f0c000a4580 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c1e174640 1 --2- 172.21.15.60:0/3214403411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c18101c30 0x7f0c1819e7d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c1e174640 1 --2- 172.21.15.60:0/3214403411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c18101270 0x7f0c181977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c1e174640 1 --2- 172.21.15.60:0/3214403411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c18100080 0x7f0c18197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c1e174640 1 -- 172.21.15.60:0/3214403411 >> 172.21.15.60:0/3214403411 conn(0x7f0c180fb610 msgr2=0x7f0c180fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:04.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c1e174640 1 -- 172.21.15.60:0/3214403411 shutdown_connections 2024-02-26T04:59:04.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:03.990+0000 7f0c1e174640 1 -- 172.21.15.60:0/3214403411 wait complete. 2024-02-26T04:59:04.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T04:59:04.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T04:59:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.122+0000 7f5ebec36640 1 -- 172.21.15.60:0/2814056584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb8101270 msgr2=0x7f5eb81016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.122+0000 7f5ebec36640 1 --2- 172.21.15.60:0/2814056584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb8101270 0x7f5eb81016f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009900 tx=0x7f5ea002bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.122+0000 7f5ebec36640 1 -- 172.21.15.60:0/2814056584 shutdown_connections 2024-02-26T04:59:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.122+0000 7f5ebec36640 1 --2- 172.21.15.60:0/2814056584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eb8101c30 0x7f5eb81084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.122+0000 7f5ebec36640 1 --2- 172.21.15.60:0/2814056584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb8101270 0x7f5eb81016f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.122+0000 7f5ebec36640 1 --2- 172.21.15.60:0/2814056584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eb8100080 0x7f5eb81004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.122+0000 7f5ebec36640 1 -- 172.21.15.60:0/2814056584 >> 172.21.15.60:0/2814056584 conn(0x7f5eb80fb5f0 msgr2=0x7f5eb80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.122+0000 7f5ebec36640 1 -- 172.21.15.60:0/2814056584 shutdown_connections 2024-02-26T04:59:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.122+0000 7f5ebec36640 1 -- 172.21.15.60:0/2814056584 wait complete. 2024-02-26T04:59:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.123+0000 7f5ebec36640 1 Processor -- start 2024-02-26T04:59:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.123+0000 7f5ebec36640 1 -- start start 2024-02-26T04:59:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.123+0000 7f5ebec36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb8100080 0x7f5eb8197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.123+0000 7f5ebec36640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eb8101270 0x7f5eb8197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.123+0000 7f5ebec36640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eb8101c30 0x7f5eb819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.123+0000 7f5ebec36640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb819ecd0 con 0x7f5eb8100080 2024-02-26T04:59:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.123+0000 7f5ebec36640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb819ee40 con 0x7f5eb8101270 2024-02-26T04:59:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.123+0000 7f5ebec36640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb819f000 con 0x7f5eb8101c30 2024-02-26T04:59:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.123+0000 7f5ebc9d2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb8100080 0x7f5eb8197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.123+0000 7f5ebc9d2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb8100080 0x7f5eb8197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34024/0 (socket says 172.21.15.60:34024) 2024-02-26T04:59:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.123+0000 7f5ebc9d2640 1 -- 172.21.15.60:0/3208973559 learned_addr learned my addr 172.21.15.60:0/3208973559 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.123+0000 7f5ebd1d3640 1 --2- 172.21.15.60:0/3208973559 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eb8101c30 0x7f5eb819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.123+0000 7f5eb7fff640 1 --2- 172.21.15.60:0/3208973559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eb8101270 0x7f5eb8197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.124+0000 7f5ebc9d2640 1 -- 172.21.15.60:0/3208973559 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eb8101c30 msgr2=0x7f5eb819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.124+0000 7f5ebc9d2640 1 --2- 172.21.15.60:0/3208973559 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eb8101c30 0x7f5eb819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.124+0000 7f5ebc9d2640 1 -- 172.21.15.60:0/3208973559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eb8101270 msgr2=0x7f5eb8197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.124+0000 7f5ebc9d2640 1 --2- 172.21.15.60:0/3208973559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eb8101270 0x7f5eb8197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.124+0000 7f5ebc9d2640 1 -- 172.21.15.60:0/3208973559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea0009560 con 0x7f5eb8100080 2024-02-26T04:59:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.124+0000 7f5ebc9d2640 1 --2- 172.21.15.60:0/3208973559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb8100080 0x7f5eb8197230 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000d950 tx=0x7f5eb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.124+0000 7f5eb5ffb640 1 -- 172.21.15.60:0/3208973559 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0009c30 con 0x7f5eb8100080 2024-02-26T04:59:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.124+0000 7f5eb5ffb640 1 -- 172.21.15.60:0/3208973559 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5eb0009d90 con 0x7f5eb8100080 2024-02-26T04:59:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.124+0000 7f5eb5ffb640 1 -- 172.21.15.60:0/3208973559 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb000f850 con 0x7f5eb8100080 2024-02-26T04:59:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.124+0000 7f5ebec36640 1 -- 172.21.15.60:0/3208973559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb819f2e0 con 0x7f5eb8100080 2024-02-26T04:59:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.124+0000 7f5ebec36640 1 -- 172.21.15.60:0/3208973559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb819f810 con 0x7f5eb8100080 2024-02-26T04:59:04.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.127+0000 7f5eb5ffb640 1 -- 172.21.15.60:0/3208973559 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5eb000f9b0 con 0x7f5eb8100080 2024-02-26T04:59:04.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.127+0000 7f5ec1a15740 1 Processor -- start 2024-02-26T04:59:04.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.127+0000 7f5ec1a15740 1 -- start start 2024-02-26T04:59:04.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.127+0000 7f5ec1a15740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7ca2a0 0x564e3c7ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.127+0000 7f5ec1a15740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cac90 0x564e3c7cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.127+0000 7f5ec1a15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cfe00 0x564e3c7d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.127+0000 7f5ec1a15740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3c73a250 con 0x564e3c7cac90 2024-02-26T04:59:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.127+0000 7f5ec1a15740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3c737780 con 0x564e3c7ca2a0 2024-02-26T04:59:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.127+0000 7f5ec1a15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3c732f60 con 0x564e3c7cfe00 2024-02-26T04:59:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.127+0000 7f5eb5ffb640 1 --2- 172.21.15.60:0/3208973559 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ea40a2140 0x7f5ea40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5ebc9d2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cac90 0x564e3c7cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5ebc9d2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cac90 0x564e3c7cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34038/0 (socket says 172.21.15.60:34038) 2024-02-26T04:59:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5ebc9d2640 1 -- 172.21.15.60:0/2761378603 learned_addr learned my addr 172.21.15.60:0/2761378603 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5eb7fff640 1 --2- 172.21.15.60:0/2761378603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cfe00 0x564e3c7d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5ebd1d3640 1 --2- 172.21.15.60:0/3208973559 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ea40a2140 0x7f5ea40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5ebd1d3640 1 --2- 172.21.15.60:0/2761378603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7ca2a0 0x564e3c7ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5eb5ffb640 1 -- 172.21.15.60:0/3208973559 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f5eb00cdc90 con 0x7f5eb8100080 2024-02-26T04:59:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5ebc9d2640 1 -- 172.21.15.60:0/2761378603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cfe00 msgr2=0x564e3c7d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5ebc9d2640 1 --2- 172.21.15.60:0/2761378603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cfe00 0x564e3c7d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5ebc9d2640 1 -- 172.21.15.60:0/2761378603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7ca2a0 msgr2=0x564e3c7ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5ebc9d2640 1 --2- 172.21.15.60:0/2761378603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7ca2a0 0x564e3c7ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5ebc9d2640 1 -- 172.21.15.60:0/2761378603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e3c6f1540 con 0x564e3c7cac90 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5ebc9d2640 1 --2- 172.21.15.60:0/2761378603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cac90 0x564e3c7cf790 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0096f50 tx=0x7f5eb0010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a069b19ff474a856 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5ebd1d3640 1 --2- 172.21.15.60:0/3208973559 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ea40a2140 0x7f5ea40a4600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80098b0 tx=0x7f5ea8009d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5eaeffd640 1 -- 172.21.15.60:0/2761378603 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb009ac80 con 0x564e3c7cac90 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5eaeffd640 1 -- 172.21.15.60:0/2761378603 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5eb00b3070 con 0x564e3c7cac90 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.128+0000 7f5eaeffd640 1 -- 172.21.15.60:0/2761378603 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0098910 con 0x564e3c7cac90 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.129+0000 7f5ec1a15740 1 -- 172.21.15.60:0/2761378603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cac90 msgr2=0x564e3c7cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.129+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/2761378603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cac90 0x564e3c7cf790 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0096f50 tx=0x7f5eb0010f30 comp rx=0 tx=0).stop 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.129+0000 7f5ec1a15740 1 -- 172.21.15.60:0/2761378603 shutdown_connections 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.129+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/2761378603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cfe00 0x564e3c7d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.129+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/2761378603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cac90 0x564e3c7cf790 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.129+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/2761378603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7ca2a0 0x564e3c7ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.129+0000 7f5ec1a15740 1 -- 172.21.15.60:0/2761378603 >> 172.21.15.60:0/2761378603 conn(0x564e3c7c57c0 msgr2=0x564e3c7c5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.129+0000 7f5ec1a15740 1 -- 172.21.15.60:0/2761378603 shutdown_connections 2024-02-26T04:59:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.129+0000 7f5ec1a15740 1 -- 172.21.15.60:0/2761378603 wait complete. 2024-02-26T04:59:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.129+0000 7f5ec1a15740 1 -- start start 2024-02-26T04:59:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.129+0000 7f5ec1a15740 1 Processor -- start 2024-02-26T04:59:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ec1a15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7ca2a0 0x564e3c7d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ec1a15740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7cac90 0x564e3c7d4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ec1a15740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cfe00 0x564e3c7db770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ec1a15740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3c406430 con 0x564e3c7cfe00 2024-02-26T04:59:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ec1a15740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3c3c2e00 con 0x564e3c7cac90 2024-02-26T04:59:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ec1a15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3c6b7fa0 con 0x564e3c7ca2a0 2024-02-26T04:59:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ec1a15740 10 client.0 _ll_register_callbacks cb 0x564e3c70dfa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ebd1d3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cfe00 0x564e3c7db770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ebd1d3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cfe00 0x564e3c7db770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34042/0 (socket says 172.21.15.60:34042) 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ebd1d3640 1 -- 172.21.15.60:0/4246526884 learned_addr learned my addr 172.21.15.60:0/4246526884 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5eb7fff640 1 --2- 172.21.15.60:0/4246526884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7cac90 0x564e3c7d4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ebd1d3640 1 -- 172.21.15.60:0/4246526884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7ca2a0 msgr2=0x564e3c7d41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ebd1d3640 1 --2- 172.21.15.60:0/4246526884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7ca2a0 0x564e3c7d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ebd1d3640 1 -- 172.21.15.60:0/4246526884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7cac90 msgr2=0x564e3c7d4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ebd1d3640 1 --2- 172.21.15.60:0/4246526884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7cac90 0x564e3c7d4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ebd1d3640 1 -- 172.21.15.60:0/4246526884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb819f810 con 0x564e3c7cfe00 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5ebd1d3640 1 --2- 172.21.15.60:0/4246526884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cfe00 0x564e3c7db770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8010d40 tx=0x7f5ea8009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3363adf2dcd45941 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.130+0000 7f5e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5e97fff640 1 -- 172.21.15.60:0/4246526884 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8004530 con 0x564e3c7cfe00 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5ec1a15740 1 -- 172.21.15.60:0/4246526884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e3c4327e0 con 0x564e3c7cfe00 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5e97fff640 1 -- 172.21.15.60:0/4246526884 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f5ea8004690 con 0x564e3c7cfe00 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5e97fff640 1 -- 172.21.15.60:0/4246526884 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea8004850 con 0x564e3c7cfe00 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5ec1a15740 10 client.15819 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:04.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5ec1a15740 1 -- 172.21.15.60:0/4246526884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564e3c7315e0 con 0x564e3c7cfe00 2024-02-26T04:59:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5e97fff640 1 -- 172.21.15.60:0/4246526884 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564e3c4327e0 con 0x564e3c7cfe00 2024-02-26T04:59:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5e97fff640 1 -- 172.21.15.60:0/4246526884 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5ea8026030 con 0x564e3c7cfe00 2024-02-26T04:59:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5ec1a15740 10 client.15819 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5ec1a15740 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:59:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5ec1a15740 20 client.15819 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/4246526884 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564e3c7e03f0 0x564e3c7e2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5ec1a15740 4 client.15819 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:59:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5ec1a15740 1 -- 172.21.15.60:0/4246526884 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e3c46f720 con 0x564e3c7e03f0 2024-02-26T04:59:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.131+0000 7f5ebc9d2640 1 --2- 172.21.15.60:0/4246526884 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564e3c7e03f0 0x564e3c7e2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.132+0000 7f5e97fff640 10 client.15819 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.132+0000 7f5ebc9d2640 1 --2- 172.21.15.60:0/4246526884 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564e3c7e03f0 0x564e3c7e2830 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=207628cdd90a5d8 server_cookie=6dcb6e65564d92b5 in_seq=0 out_seq=0 2024-02-26T04:59:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.132+0000 7f5e97fff640 1 -- 172.21.15.60:0/4246526884 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e3c46f720 con 0x564e3c7e03f0 2024-02-26T04:59:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.132+0000 7f5e97fff640 10 client.15819 handle_command_reply: tid=0 2024-02-26T04:59:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.133+0000 7f5ec1a15740 1 client.15819 shutdown 2024-02-26T04:59:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.133+0000 7f5ec1a15740 1 -- 172.21.15.60:0/4246526884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cfe00 msgr2=0x564e3c7db770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.133+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/4246526884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cfe00 0x564e3c7db770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8010d40 tx=0x7f5ea8009600 comp rx=0 tx=0).stop 2024-02-26T04:59:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.133+0000 7f5ec1a15740 1 -- 172.21.15.60:0/4246526884 shutdown_connections 2024-02-26T04:59:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.133+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/4246526884 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564e3c7e03f0 0x564e3c7e2830 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.133+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/4246526884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cfe00 0x564e3c7db770 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.133+0000 7f5e97fff640 0 client.15819 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.133+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/4246526884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7cac90 0x564e3c7d4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.133+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/4246526884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7ca2a0 0x564e3c7d41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.133+0000 7f5ec1a15740 1 -- 172.21.15.60:0/4246526884 >> 172.21.15.60:0/4246526884 conn(0x564e3c7c57c0 msgr2=0x564e3c7c4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.133+0000 7f5ec1a15740 1 -- 172.21.15.60:0/4246526884 shutdown_connections 2024-02-26T04:59:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.133+0000 7f5ec1a15740 1 -- 172.21.15.60:0/4246526884 wait complete. 2024-02-26T04:59:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.133+0000 7f5ec1a15740 20 client.15819 trim_cache size 0 max 16384 2024-02-26T04:59:04.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.149+0000 7f5ec1a15740 1 Processor -- start 2024-02-26T04:59:04.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.149+0000 7f5ec1a15740 1 -- start start 2024-02-26T04:59:04.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.149+0000 7f5ec1a15740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7ca2a0 0x564e3c7d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.149+0000 7f5ec1a15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cac90 0x564e3c7d4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.149+0000 7f5ec1a15740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7cfe00 0x564e3c7c4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.149+0000 7f5ec1a15740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3c4327e0 con 0x564e3c7ca2a0 2024-02-26T04:59:04.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.149+0000 7f5ec1a15740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3c7d2750 con 0x564e3c7cfe00 2024-02-26T04:59:04.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.149+0000 7f5ec1a15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3c7cb0c0 con 0x564e3c7cac90 2024-02-26T04:59:04.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.149+0000 7f5ebc9d2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7ca2a0 0x564e3c7d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.149+0000 7f5ebc9d2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7ca2a0 0x564e3c7d41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34050/0 (socket says 172.21.15.60:34050) 2024-02-26T04:59:04.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.149+0000 7f5ebc9d2640 1 -- 172.21.15.60:0/654993980 learned_addr learned my addr 172.21.15.60:0/654993980 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.149+0000 7f5ebd1d3640 1 --2- 172.21.15.60:0/654993980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7cfe00 0x564e3c7c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.149+0000 7f5eb7fff640 1 --2- 172.21.15.60:0/654993980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cac90 0x564e3c7d4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.150+0000 7f5ebc9d2640 1 -- 172.21.15.60:0/654993980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cac90 msgr2=0x564e3c7d4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.150+0000 7f5ebc9d2640 1 --2- 172.21.15.60:0/654993980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cac90 0x564e3c7d4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.150+0000 7f5ebc9d2640 1 -- 172.21.15.60:0/654993980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7cfe00 msgr2=0x564e3c7c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.150+0000 7f5ebc9d2640 1 --2- 172.21.15.60:0/654993980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7cfe00 0x564e3c7c4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.150+0000 7f5ebc9d2640 1 -- 172.21.15.60:0/654993980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e3c6cda90 con 0x564e3c7ca2a0 2024-02-26T04:59:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.150+0000 7f5ebd1d3640 1 --2- 172.21.15.60:0/654993980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7cfe00 0x564e3c7c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T04:59:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.150+0000 7f5ebc9d2640 1 --2- 172.21.15.60:0/654993980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7ca2a0 0x564e3c7d41f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0096260 tx=0x7f5eb0096950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b70c98225ba7236 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.150+0000 7f5eae7fc640 1 -- 172.21.15.60:0/654993980 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb009a600 con 0x564e3c7ca2a0 2024-02-26T04:59:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.150+0000 7f5eae7fc640 1 -- 172.21.15.60:0/654993980 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5eb00a2d90 con 0x564e3c7ca2a0 2024-02-26T04:59:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.150+0000 7f5eae7fc640 1 -- 172.21.15.60:0/654993980 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb00a2890 con 0x564e3c7ca2a0 2024-02-26T04:59:04.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.151+0000 7f5ec1a15740 1 -- 172.21.15.60:0/654993980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7ca2a0 msgr2=0x564e3c7d41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:04.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.151+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/654993980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7ca2a0 0x564e3c7d41f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0096260 tx=0x7f5eb0096950 comp rx=0 tx=0).stop 2024-02-26T04:59:04.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.151+0000 7f5ec1a15740 1 -- 172.21.15.60:0/654993980 shutdown_connections 2024-02-26T04:59:04.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.151+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/654993980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7cfe00 0x564e3c7c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.151+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/654993980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cac90 0x564e3c7d4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.151+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/654993980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7ca2a0 0x564e3c7d41f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.151+0000 7f5ec1a15740 1 -- 172.21.15.60:0/654993980 >> 172.21.15.60:0/654993980 conn(0x564e3c7c57c0 msgr2=0x564e3c7cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.151+0000 7f5ec1a15740 1 -- 172.21.15.60:0/654993980 shutdown_connections 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.151+0000 7f5ec1a15740 1 -- 172.21.15.60:0/654993980 wait complete. 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.151+0000 7f5ec1a15740 1 -- start start 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.151+0000 7f5ec1a15740 1 Processor -- start 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ec1a15740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7ca2a0 0x564e3c833450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ec1a15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cac90 0x564e3c833990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ec1a15740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cfe00 0x564e3c83a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ec1a15740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3c83af10 con 0x564e3c7cfe00 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ec1a15740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3c83b050 con 0x564e3c7ca2a0 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ec1a15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e3c83b1c0 con 0x564e3c7cac90 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ec1a15740 10 client.0 _ll_register_callbacks cb 0x7f5eb0096100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:04.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ebd1d3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cfe00 0x564e3c83a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ebd1d3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cfe00 0x564e3c83a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34056/0 (socket says 172.21.15.60:34056) 2024-02-26T04:59:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ebd1d3640 1 -- 172.21.15.60:0/1584099753 learned_addr learned my addr 172.21.15.60:0/1584099753 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ebc9d2640 1 --2- 172.21.15.60:0/1584099753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7ca2a0 0x564e3c833450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5eb7fff640 1 --2- 172.21.15.60:0/1584099753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cac90 0x564e3c833990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ebd1d3640 1 -- 172.21.15.60:0/1584099753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cac90 msgr2=0x564e3c833990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ebd1d3640 1 --2- 172.21.15.60:0/1584099753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cac90 0x564e3c833990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ebd1d3640 1 -- 172.21.15.60:0/1584099753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7ca2a0 msgr2=0x564e3c833450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ebd1d3640 1 --2- 172.21.15.60:0/1584099753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7ca2a0 0x564e3c833450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ebd1d3640 1 -- 172.21.15.60:0/1584099753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e3c7315e0 con 0x564e3c7cfe00 2024-02-26T04:59:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ebd1d3640 1 --2- 172.21.15.60:0/1584099753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cfe00 0x564e3c83a9d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8010d10 tx=0x7f5ea8012640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2563d9aee8e192 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5e97fff640 1 -- 172.21.15.60:0/1584099753 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8004400 con 0x564e3c7cfe00 2024-02-26T04:59:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5e97fff640 1 -- 172.21.15.60:0/1584099753 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f5ea8004560 con 0x564e3c7cfe00 2024-02-26T04:59:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ec1a15740 1 -- 172.21.15.60:0/1584099753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e3c4be890 con 0x564e3c7cfe00 2024-02-26T04:59:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5e97fff640 1 -- 172.21.15.60:0/1584099753 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea8004720 con 0x564e3c7cfe00 2024-02-26T04:59:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.152+0000 7f5ec1a15740 10 client.15825 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.153+0000 7f5ec1a15740 1 -- 172.21.15.60:0/1584099753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564e3c6f1540 con 0x564e3c7cfe00 2024-02-26T04:59:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.153+0000 7f5e97fff640 1 -- 172.21.15.60:0/1584099753 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564e3c4be890 con 0x564e3c7cfe00 2024-02-26T04:59:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.153+0000 7f5e97fff640 1 -- 172.21.15.60:0/1584099753 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5ea8026040 con 0x564e3c7cfe00 2024-02-26T04:59:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.153+0000 7f5ec1a15740 10 client.15825 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.153+0000 7f5ec1a15740 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:59:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.153+0000 7f5ec1a15740 20 client.15825 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.153+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/1584099753 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564e3c83cbd0 0x564e3c83eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.153+0000 7f5ec1a15740 4 client.15825 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T04:59:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.153+0000 7f5ec1a15740 1 -- 172.21.15.60:0/1584099753 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e3c4214e0 con 0x564e3c83cbd0 2024-02-26T04:59:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.153+0000 7f5ebc9d2640 1 --2- 172.21.15.60:0/1584099753 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564e3c83cbd0 0x564e3c83eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.153+0000 7f5ebc9d2640 1 --2- 172.21.15.60:0/1584099753 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564e3c83cbd0 0x564e3c83eff0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ee68bbc28269dcf server_cookie=7eec960e412e9621 in_seq=0 out_seq=0 2024-02-26T04:59:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.153+0000 7f5e97fff640 10 client.15825 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5e97fff640 1 -- 172.21.15.60:0/1584099753 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564e3c4214e0 con 0x564e3c83cbd0 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5e97fff640 10 client.15825 handle_command_reply: tid=0 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5ec1a15740 1 client.15825 shutdown 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5ec1a15740 1 -- 172.21.15.60:0/1584099753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cfe00 msgr2=0x564e3c83a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/1584099753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cfe00 0x564e3c83a9d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8010d10 tx=0x7f5ea8012640 comp rx=0 tx=0).stop 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5ec1a15740 1 -- 172.21.15.60:0/1584099753 shutdown_connections 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/1584099753 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x564e3c83cbd0 0x564e3c83eff0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/1584099753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e3c7cfe00 0x564e3c83a9d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5e97fff640 0 client.15825 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/1584099753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564e3c7cac90 0x564e3c833990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5ec1a15740 1 --2- 172.21.15.60:0/1584099753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e3c7ca2a0 0x564e3c833450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5ec1a15740 1 -- 172.21.15.60:0/1584099753 >> 172.21.15.60:0/1584099753 conn(0x564e3c7c57c0 msgr2=0x564e3c7d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5ec1a15740 1 -- 172.21.15.60:0/1584099753 shutdown_connections 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5ec1a15740 1 -- 172.21.15.60:0/1584099753 wait complete. 2024-02-26T04:59:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.154+0000 7f5ec1a15740 20 client.15825 trim_cache size 0 max 16384 2024-02-26T04:59:04.163 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:59:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-26T04:59:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-02-26T04:59:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: "e07df569-17c8-4f1b-81e3-4f9111544967": { 2024-02-26T04:59:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-02-26T04:59:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "e07df569-17c8-4f1b-81e3-4f9111544967", 2024-02-26T04:59:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,force" 2024-02-26T04:59:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:04.163 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:59:04.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.155+0000 7f5ebec36640 1 -- 172.21.15.60:0/3208973559 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ea40a2140 msgr2=0x7f5ea40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:04.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.155+0000 7f5ebec36640 1 --2- 172.21.15.60:0/3208973559 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ea40a2140 0x7f5ea40a4600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80098b0 tx=0x7f5ea8009d90 comp rx=0 tx=0).stop 2024-02-26T04:59:04.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.155+0000 7f5ebec36640 1 -- 172.21.15.60:0/3208973559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb8100080 msgr2=0x7f5eb8197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:04.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.155+0000 7f5ebec36640 1 --2- 172.21.15.60:0/3208973559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb8100080 0x7f5eb8197230 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000d950 tx=0x7f5eb000de10 comp rx=0 tx=0).stop 2024-02-26T04:59:04.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.155+0000 7f5ebec36640 1 -- 172.21.15.60:0/3208973559 shutdown_connections 2024-02-26T04:59:04.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.155+0000 7f5ebec36640 1 --2- 172.21.15.60:0/3208973559 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ea40a2140 0x7f5ea40a4600 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.155+0000 7f5ebec36640 1 --2- 172.21.15.60:0/3208973559 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eb8101c30 0x7f5eb819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.155+0000 7f5ebec36640 1 --2- 172.21.15.60:0/3208973559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5eb8101270 0x7f5eb8197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.155+0000 7f5ebec36640 1 --2- 172.21.15.60:0/3208973559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb8100080 0x7f5eb8197230 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:04.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.155+0000 7f5ebec36640 1 -- 172.21.15.60:0/3208973559 >> 172.21.15.60:0/3208973559 conn(0x7f5eb80fb5f0 msgr2=0x7f5eb80fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:04.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.155+0000 7f5ebec36640 1 -- 172.21.15.60:0/3208973559 shutdown_connections 2024-02-26T04:59:04.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:04.155+0000 7f5ebec36640 1 -- 172.21.15.60:0/3208973559 wait complete. 2024-02-26T04:59:04.167 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:03 smithi060 ceph-mon[32382]: pgmap v405: 129 pgs: 129 active+clean; 329 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 1.1 MiB/s wr, 915 op/s 2024-02-26T04:59:04.168 INFO:tasks.cephfs.filesystem:scrub status for tag:e07df569-17c8-4f1b-81e3-4f9111544967 - {'path': '/', 'tag': 'e07df569-17c8-4f1b-81e3-4f9111544967', 'options': 'recursive,force'} 2024-02-26T04:59:04.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:03 smithi156 ceph-mon[35805]: pgmap v405: 129 pgs: 129 active+clean; 329 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 1.1 MiB/s wr, 915 op/s 2024-02-26T04:59:04.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:03 smithi191 ceph-mon[36352]: pgmap v405: 129 pgs: 129 active+clean; 329 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 1.1 MiB/s wr, 915 op/s 2024-02-26T04:59:04.666 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-02-26T04:59:04.666 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-02-26T04:59:04.666 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-02-26T04:59:04.666 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-02-26T04:59:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-02-26T04:59:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-02-26T04:59:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-02-26T04:59:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-02-26T04:59:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-02-26T04:59:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-02-26T04:59:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-02-26T04:59:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-02-26T04:59:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-02-26T04:59:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-02-26T04:59:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-02-26T04:59:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-02-26T04:59:04.667 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-02-26T04:59:04.668 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-02-26T04:59:04.668 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-02-26T04:59:04.668 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-02-26T04:59:04.668 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-02-26T04:59:04.668 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-02-26T04:59:04.668 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-02-26T04:59:04.668 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-02-26T04:59:04.668 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-02-26T04:59:04.668 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-02-26T04:59:04.668 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-02-26T04:59:04.669 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-02-26T04:59:04.670 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-02-26T04:59:04.671 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-02-26T04:59:04.672 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-02-26T04:59:04.672 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-02-26T04:59:04.672 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-02-26T04:59:04.672 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-02-26T04:59:04.672 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-02-26T04:59:04.672 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-02-26T04:59:04.672 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-02-26T04:59:04.672 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-02-26T04:59:04.672 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-02-26T04:59:05.730 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-02-26T04:59:05.730 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-02-26T04:59:05.730 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-02-26T04:59:05.730 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-02-26T04:59:05.730 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-02-26T04:59:05.730 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-02-26T04:59:05.730 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-02-26T04:59:05.730 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-02-26T04:59:05.730 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-02-26T04:59:05.730 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-02-26T04:59:05.731 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-02-26T04:59:05.731 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-02-26T04:59:05.731 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-02-26T04:59:05.731 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-02-26T04:59:05.731 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-02-26T04:59:05.731 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-02-26T04:59:05.731 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-02-26T04:59:05.731 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-02-26T04:59:05.731 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-02-26T04:59:05.731 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-02-26T04:59:05.731 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-02-26T04:59:05.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-02-26T04:59:05.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-02-26T04:59:05.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-02-26T04:59:05.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-02-26T04:59:05.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-02-26T04:59:05.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-02-26T04:59:05.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-02-26T04:59:05.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-02-26T04:59:05.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-02-26T04:59:05.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-02-26T04:59:05.732 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-02-26T04:59:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-02-26T04:59:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-02-26T04:59:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-02-26T04:59:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-02-26T04:59:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-02-26T04:59:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-02-26T04:59:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-02-26T04:59:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-02-26T04:59:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-02-26T04:59:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-02-26T04:59:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-02-26T04:59:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-02-26T04:59:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-02-26T04:59:05.733 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-02-26T04:59:05.734 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-02-26T04:59:05.735 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-02-26T04:59:05.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-02-26T04:59:05.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-02-26T04:59:05.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-02-26T04:59:05.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-02-26T04:59:05.736 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-02-26T04:59:06.083 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-02-26T04:59:06.083 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-02-26T04:59:06.083 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-02-26T04:59:06.083 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-02-26T04:59:06.083 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-02-26T04:59:06.084 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-02-26T04:59:06.084 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-02-26T04:59:06.084 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-02-26T04:59:06.084 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-02-26T04:59:06.084 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-02-26T04:59:06.084 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-02-26T04:59:06.084 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-02-26T04:59:06.084 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-02-26T04:59:06.084 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-02-26T04:59:06.084 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-02-26T04:59:06.085 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-02-26T04:59:06.085 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-02-26T04:59:06.085 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-02-26T04:59:06.085 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-02-26T04:59:06.085 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-02-26T04:59:06.085 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-02-26T04:59:06.085 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-02-26T04:59:06.085 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-02-26T04:59:06.085 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-02-26T04:59:06.085 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-02-26T04:59:06.085 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-02-26T04:59:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-02-26T04:59:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-02-26T04:59:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-02-26T04:59:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-02-26T04:59:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-02-26T04:59:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-02-26T04:59:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-02-26T04:59:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-02-26T04:59:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-02-26T04:59:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-02-26T04:59:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-02-26T04:59:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-02-26T04:59:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-02-26T04:59:06.086 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-02-26T04:59:06.087 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-02-26T04:59:06.087 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-02-26T04:59:06.087 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-02-26T04:59:06.087 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-02-26T04:59:06.087 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-02-26T04:59:06.087 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-02-26T04:59:06.087 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-02-26T04:59:06.087 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-02-26T04:59:06.087 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-02-26T04:59:06.087 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-02-26T04:59:06.087 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-02-26T04:59:06.087 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-02-26T04:59:06.088 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-02-26T04:59:06.088 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-02-26T04:59:06.088 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-02-26T04:59:06.088 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-02-26T04:59:06.088 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-02-26T04:59:06.088 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-02-26T04:59:06.088 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-02-26T04:59:06.088 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-02-26T04:59:06.088 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-02-26T04:59:06.088 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-02-26T04:59:06.088 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-02-26T04:59:06.089 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-02-26T04:59:06.089 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-02-26T04:59:06.089 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-02-26T04:59:06.089 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-02-26T04:59:06.089 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-02-26T04:59:06.089 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-02-26T04:59:06.089 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-02-26T04:59:06.089 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-02-26T04:59:06.089 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-02-26T04:59:06.089 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-02-26T04:59:06.089 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-02-26T04:59:06.089 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-02-26T04:59:06.089 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-02-26T04:59:06.090 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-02-26T04:59:06.090 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-02-26T04:59:06.090 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-02-26T04:59:06.090 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-02-26T04:59:06.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:05 smithi156 ceph-mon[35805]: scrub queued for path: / 2024-02-26T04:59:06.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:05 smithi156 ceph-mon[35805]: scrub summary: idle+waiting paths [/] 2024-02-26T04:59:06.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:05 smithi156 ceph-mon[35805]: scrub summary: active paths [/] 2024-02-26T04:59:06.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:05 smithi156 ceph-mon[35805]: scrub summary: idle+waiting paths [/] 2024-02-26T04:59:06.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:05 smithi156 ceph-mon[35805]: pgmap v406: 129 pgs: 129 active+clean; 337 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 1.2 MiB/s wr, 346 op/s 2024-02-26T04:59:06.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:05 smithi060 ceph-mon[32382]: scrub queued for path: / 2024-02-26T04:59:06.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:05 smithi060 ceph-mon[32382]: scrub summary: idle+waiting paths [/] 2024-02-26T04:59:06.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:05 smithi060 ceph-mon[32382]: scrub summary: active paths [/] 2024-02-26T04:59:06.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:05 smithi060 ceph-mon[32382]: scrub summary: idle+waiting paths [/] 2024-02-26T04:59:06.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:05 smithi060 ceph-mon[32382]: pgmap v406: 129 pgs: 129 active+clean; 337 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 1.2 MiB/s wr, 346 op/s 2024-02-26T04:59:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:05 smithi191 ceph-mon[36352]: scrub queued for path: / 2024-02-26T04:59:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:05 smithi191 ceph-mon[36352]: scrub summary: idle+waiting paths [/] 2024-02-26T04:59:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:05 smithi191 ceph-mon[36352]: scrub summary: active paths [/] 2024-02-26T04:59:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:05 smithi191 ceph-mon[36352]: scrub summary: idle+waiting paths [/] 2024-02-26T04:59:06.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:05 smithi191 ceph-mon[36352]: pgmap v406: 129 pgs: 129 active+clean; 337 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 1.2 MiB/s wr, 346 op/s 2024-02-26T04:59:06.393 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-02-26T04:59:06.393 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-02-26T04:59:06.393 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-02-26T04:59:06.393 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-02-26T04:59:06.393 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-02-26T04:59:06.393 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-02-26T04:59:06.393 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-02-26T04:59:06.393 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-02-26T04:59:06.394 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-02-26T04:59:06.395 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-02-26T04:59:06.396 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-02-26T04:59:06.396 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-02-26T04:59:06.396 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-02-26T04:59:06.396 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-02-26T04:59:06.396 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-02-26T04:59:06.396 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-02-26T04:59:06.396 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-02-26T04:59:06.396 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-02-26T04:59:06.396 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-02-26T04:59:06.396 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-02-26T04:59:06.396 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-02-26T04:59:06.396 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-02-26T04:59:06.396 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-02-26T04:59:06.396 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-02-26T04:59:06.397 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-02-26T04:59:06.398 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-02-26T04:59:06.398 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-02-26T04:59:06.398 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-02-26T04:59:06.398 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-02-26T04:59:06.398 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-02-26T04:59:06.398 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-02-26T04:59:06.398 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-02-26T04:59:06.398 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-02-26T04:59:06.398 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-02-26T04:59:06.398 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-02-26T04:59:06.398 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-02-26T04:59:06.398 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-02-26T04:59:06.398 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-02-26T04:59:06.398 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-02-26T04:59:07.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:06 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:59:07.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:06 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:59:07.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:06 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T04:59:07.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-02-26T04:59:07.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-02-26T04:59:07.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-02-26T04:59:07.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-02-26T04:59:07.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-02-26T04:59:07.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-02-26T04:59:07.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-02-26T04:59:07.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-02-26T04:59:07.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-02-26T04:59:07.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-02-26T04:59:07.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-02-26T04:59:07.691 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-02-26T04:59:07.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-02-26T04:59:07.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-02-26T04:59:07.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-02-26T04:59:07.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-02-26T04:59:07.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-02-26T04:59:07.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-02-26T04:59:07.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-02-26T04:59:07.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-02-26T04:59:07.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-02-26T04:59:07.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-02-26T04:59:07.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-02-26T04:59:07.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-02-26T04:59:07.692 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-02-26T04:59:07.693 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-02-26T04:59:07.694 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-02-26T04:59:07.695 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-02-26T04:59:07.696 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-02-26T04:59:07.696 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-02-26T04:59:07.696 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-02-26T04:59:07.696 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-02-26T04:59:07.696 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-02-26T04:59:07.696 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-02-26T04:59:07.696 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-02-26T04:59:07.696 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-02-26T04:59:07.696 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-02-26T04:59:07.696 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-02-26T04:59:08.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:07 smithi156 ceph-mon[35805]: pgmap v407: 129 pgs: 129 active+clean; 316 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 100 MiB/s rd, 1.2 MiB/s wr, 420 op/s 2024-02-26T04:59:08.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:07 smithi156 ceph-mon[35805]: scrub summary: idle 2024-02-26T04:59:08.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:07 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:07 smithi060 ceph-mon[32382]: pgmap v407: 129 pgs: 129 active+clean; 316 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 100 MiB/s rd, 1.2 MiB/s wr, 420 op/s 2024-02-26T04:59:08.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:07 smithi060 ceph-mon[32382]: scrub summary: idle 2024-02-26T04:59:08.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:07 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:07 smithi191 ceph-mon[36352]: pgmap v407: 129 pgs: 129 active+clean; 316 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 100 MiB/s rd, 1.2 MiB/s wr, 420 op/s 2024-02-26T04:59:08.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:07 smithi191 ceph-mon[36352]: scrub summary: idle 2024-02-26T04:59:08.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:07 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:08.869 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-02-26T04:59:08.869 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-02-26T04:59:08.869 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-02-26T04:59:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-02-26T04:59:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-02-26T04:59:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-02-26T04:59:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-02-26T04:59:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-02-26T04:59:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-02-26T04:59:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-02-26T04:59:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-02-26T04:59:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-02-26T04:59:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-02-26T04:59:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-02-26T04:59:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-02-26T04:59:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-02-26T04:59:08.870 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-02-26T04:59:08.871 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-02-26T04:59:08.872 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-02-26T04:59:08.872 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-02-26T04:59:08.872 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-02-26T04:59:08.872 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-02-26T04:59:08.872 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-02-26T04:59:08.872 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-02-26T04:59:08.872 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-02-26T04:59:08.872 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-02-26T04:59:08.872 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-02-26T04:59:08.872 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-02-26T04:59:08.872 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-02-26T04:59:08.872 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-02-26T04:59:08.872 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-02-26T04:59:08.872 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-02-26T04:59:08.873 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-02-26T04:59:08.873 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-02-26T04:59:08.873 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-02-26T04:59:08.873 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-02-26T04:59:08.873 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-02-26T04:59:08.873 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-02-26T04:59:08.873 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-02-26T04:59:08.873 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-02-26T04:59:08.873 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-02-26T04:59:08.873 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-02-26T04:59:08.873 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-02-26T04:59:08.873 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-02-26T04:59:08.873 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-02-26T04:59:08.874 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-02-26T04:59:08.874 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-02-26T04:59:08.874 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-02-26T04:59:08.874 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-02-26T04:59:08.874 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-02-26T04:59:08.874 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-02-26T04:59:08.874 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-02-26T04:59:08.874 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-02-26T04:59:08.874 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-02-26T04:59:08.874 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-02-26T04:59:08.875 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-02-26T04:59:08.875 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-02-26T04:59:08.875 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-02-26T04:59:08.875 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-02-26T04:59:08.875 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-02-26T04:59:08.875 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-02-26T04:59:08.875 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-02-26T04:59:08.875 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-02-26T04:59:08.875 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-02-26T04:59:08.875 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-02-26T04:59:08.875 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-02-26T04:59:08.875 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-02-26T04:59:09.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:08 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:59:09.177 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:08 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:59:09.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:08 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:09.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:59:09.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:59:09.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:09.247 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-02-26T04:59:09.247 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-02-26T04:59:09.247 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-02-26T04:59:09.247 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-02-26T04:59:09.247 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-02-26T04:59:09.247 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-02-26T04:59:09.247 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-02-26T04:59:09.247 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-02-26T04:59:09.247 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-02-26T04:59:09.248 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-02-26T04:59:09.248 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-02-26T04:59:09.248 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-02-26T04:59:09.248 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-02-26T04:59:09.248 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-02-26T04:59:09.248 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-02-26T04:59:09.248 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-02-26T04:59:09.248 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-02-26T04:59:09.248 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-02-26T04:59:09.248 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-02-26T04:59:09.248 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-02-26T04:59:09.248 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-02-26T04:59:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-02-26T04:59:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-02-26T04:59:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-02-26T04:59:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-02-26T04:59:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-02-26T04:59:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-02-26T04:59:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-02-26T04:59:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-02-26T04:59:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-02-26T04:59:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-02-26T04:59:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-02-26T04:59:09.249 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-02-26T04:59:09.250 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-02-26T04:59:09.250 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-02-26T04:59:09.250 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-02-26T04:59:09.250 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-02-26T04:59:09.250 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-02-26T04:59:09.250 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-02-26T04:59:09.250 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-02-26T04:59:09.250 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-02-26T04:59:09.250 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-02-26T04:59:09.250 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-02-26T04:59:09.250 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-02-26T04:59:09.251 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-02-26T04:59:09.251 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-02-26T04:59:09.251 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-02-26T04:59:09.251 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-02-26T04:59:09.251 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-02-26T04:59:09.251 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-02-26T04:59:09.251 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-02-26T04:59:09.251 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-02-26T04:59:09.251 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-02-26T04:59:09.251 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-02-26T04:59:09.251 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-02-26T04:59:09.251 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-02-26T04:59:09.251 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-02-26T04:59:09.252 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-02-26T04:59:09.252 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-02-26T04:59:09.252 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-02-26T04:59:09.252 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-02-26T04:59:09.252 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-02-26T04:59:09.252 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-02-26T04:59:09.252 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-02-26T04:59:09.252 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-02-26T04:59:09.252 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-02-26T04:59:09.252 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-02-26T04:59:09.252 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-02-26T04:59:09.252 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-02-26T04:59:09.253 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-02-26T04:59:09.253 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-02-26T04:59:09.253 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-02-26T04:59:09.253 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-02-26T04:59:09.253 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-02-26T04:59:09.253 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-02-26T04:59:09.253 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-02-26T04:59:09.253 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-02-26T04:59:09.253 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-02-26T04:59:09.253 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-02-26T04:59:09.253 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-02-26T04:59:09.253 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-02-26T04:59:09.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T04:59:09.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T04:59:09.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T04:59:09.762 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-02-26T04:59:09.762 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-02-26T04:59:09.762 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-02-26T04:59:09.762 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-02-26T04:59:09.762 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-02-26T04:59:09.762 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-02-26T04:59:09.762 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-02-26T04:59:09.763 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-02-26T04:59:09.763 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-02-26T04:59:09.763 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-02-26T04:59:09.763 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-02-26T04:59:09.763 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-02-26T04:59:09.763 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-02-26T04:59:09.763 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-02-26T04:59:09.763 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-02-26T04:59:09.763 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-02-26T04:59:09.763 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-02-26T04:59:09.763 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-02-26T04:59:09.763 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-02-26T04:59:09.763 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-02-26T04:59:09.763 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-02-26T04:59:09.764 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-02-26T04:59:09.764 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-02-26T04:59:09.764 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-02-26T04:59:09.764 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-02-26T04:59:09.764 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-02-26T04:59:09.764 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-02-26T04:59:09.764 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-02-26T04:59:09.764 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-02-26T04:59:09.764 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-02-26T04:59:09.764 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-02-26T04:59:09.764 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-02-26T04:59:09.764 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-02-26T04:59:09.764 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-02-26T04:59:09.764 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-02-26T04:59:09.765 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-02-26T04:59:09.766 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-02-26T04:59:09.767 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-02-26T04:59:09.767 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-02-26T04:59:09.767 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-02-26T04:59:09.767 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-02-26T04:59:09.767 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-02-26T04:59:09.767 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-02-26T04:59:09.767 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-02-26T04:59:09.767 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-02-26T04:59:09.767 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-02-26T04:59:09.767 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-02-26T04:59:09.767 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-02-26T04:59:09.767 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-02-26T04:59:09.767 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-02-26T04:59:09.767 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-02-26T04:59:09.768 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-02-26T04:59:10.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:09 smithi060 ceph-mon[32382]: pgmap v408: 129 pgs: 129 active+clean; 336 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 103 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-02-26T04:59:10.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:09 smithi191 ceph-mon[36352]: pgmap v408: 129 pgs: 129 active+clean; 336 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 103 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-02-26T04:59:10.428 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:09 smithi156 ceph-mon[35805]: pgmap v408: 129 pgs: 129 active+clean; 336 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 103 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-02-26T04:59:10.769 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-02-26T04:59:10.769 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-02-26T04:59:10.769 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-02-26T04:59:10.769 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-02-26T04:59:10.769 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-02-26T04:59:10.769 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-02-26T04:59:10.769 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-02-26T04:59:10.769 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-02-26T04:59:10.769 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-02-26T04:59:10.770 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-02-26T04:59:10.770 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-02-26T04:59:10.770 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-02-26T04:59:10.770 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-02-26T04:59:10.770 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-02-26T04:59:10.770 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-02-26T04:59:10.770 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-02-26T04:59:10.771 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-02-26T04:59:10.771 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-02-26T04:59:10.771 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-02-26T04:59:10.771 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-02-26T04:59:10.771 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-02-26T04:59:10.771 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-02-26T04:59:10.771 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-02-26T04:59:10.771 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-02-26T04:59:10.771 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-02-26T04:59:10.771 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-02-26T04:59:10.771 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-02-26T04:59:10.771 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-02-26T04:59:10.771 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-02-26T04:59:10.771 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-02-26T04:59:10.772 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-02-26T04:59:10.772 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-02-26T04:59:10.772 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-02-26T04:59:10.772 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-02-26T04:59:10.772 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-02-26T04:59:10.772 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-02-26T04:59:10.772 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-02-26T04:59:10.772 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-02-26T04:59:10.772 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-02-26T04:59:10.772 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-02-26T04:59:10.772 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-02-26T04:59:10.772 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-02-26T04:59:10.772 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-02-26T04:59:10.773 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-02-26T04:59:10.773 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-02-26T04:59:10.773 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-02-26T04:59:10.773 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-02-26T04:59:10.773 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-02-26T04:59:10.773 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-02-26T04:59:10.773 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-02-26T04:59:10.773 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-02-26T04:59:10.773 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-02-26T04:59:10.773 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-02-26T04:59:10.773 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-02-26T04:59:10.773 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-02-26T04:59:10.773 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-02-26T04:59:10.773 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-02-26T04:59:10.774 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-02-26T04:59:10.774 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-02-26T04:59:10.774 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-02-26T04:59:10.774 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-02-26T04:59:10.774 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-02-26T04:59:10.774 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-02-26T04:59:10.774 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-02-26T04:59:10.774 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-02-26T04:59:10.774 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-02-26T04:59:10.774 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-02-26T04:59:10.774 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-02-26T04:59:10.774 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-02-26T04:59:10.774 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-02-26T04:59:10.774 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-02-26T04:59:10.775 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-02-26T04:59:10.775 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-02-26T04:59:10.775 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-02-26T04:59:10.775 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-02-26T04:59:10.775 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-02-26T04:59:10.775 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-02-26T04:59:10.775 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-02-26T04:59:10.775 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-02-26T04:59:10.775 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-02-26T04:59:10.775 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-02-26T04:59:11.468 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-02-26T04:59:11.468 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-02-26T04:59:11.468 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-02-26T04:59:11.468 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-02-26T04:59:11.469 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-02-26T04:59:11.469 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-02-26T04:59:11.469 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-02-26T04:59:11.469 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-02-26T04:59:11.469 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-02-26T04:59:11.469 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-02-26T04:59:11.469 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-02-26T04:59:11.469 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-02-26T04:59:11.469 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-02-26T04:59:11.469 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-02-26T04:59:11.469 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-02-26T04:59:11.469 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-02-26T04:59:11.469 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-02-26T04:59:11.470 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-02-26T04:59:11.470 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-02-26T04:59:11.470 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-02-26T04:59:11.470 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-02-26T04:59:11.470 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-02-26T04:59:11.470 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-02-26T04:59:11.470 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-02-26T04:59:11.470 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-02-26T04:59:11.470 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-02-26T04:59:11.470 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-02-26T04:59:11.470 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-02-26T04:59:11.470 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-02-26T04:59:11.470 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-02-26T04:59:11.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-02-26T04:59:11.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-02-26T04:59:11.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-02-26T04:59:11.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-02-26T04:59:11.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-02-26T04:59:11.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-02-26T04:59:11.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-02-26T04:59:11.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-02-26T04:59:11.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-02-26T04:59:11.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-02-26T04:59:11.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-02-26T04:59:11.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-02-26T04:59:11.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-02-26T04:59:11.471 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-02-26T04:59:11.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-02-26T04:59:11.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-02-26T04:59:11.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-02-26T04:59:11.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-02-26T04:59:11.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-02-26T04:59:11.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-02-26T04:59:11.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-02-26T04:59:11.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-02-26T04:59:11.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-02-26T04:59:11.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-02-26T04:59:11.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-02-26T04:59:11.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-02-26T04:59:11.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-02-26T04:59:11.472 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-02-26T04:59:11.473 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-02-26T04:59:11.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-02-26T04:59:11.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-02-26T04:59:11.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-02-26T04:59:11.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-02-26T04:59:11.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-02-26T04:59:11.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-02-26T04:59:11.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-02-26T04:59:11.474 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-02-26T04:59:11.794 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-02-26T04:59:11.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-02-26T04:59:11.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-02-26T04:59:11.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-02-26T04:59:11.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-02-26T04:59:11.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-02-26T04:59:11.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-02-26T04:59:11.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-02-26T04:59:11.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-02-26T04:59:11.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-02-26T04:59:11.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-02-26T04:59:11.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-02-26T04:59:11.795 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-02-26T04:59:11.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-02-26T04:59:11.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-02-26T04:59:11.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-02-26T04:59:11.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-02-26T04:59:11.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-02-26T04:59:11.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-02-26T04:59:11.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-02-26T04:59:11.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-02-26T04:59:11.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-02-26T04:59:11.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-02-26T04:59:11.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-02-26T04:59:11.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-02-26T04:59:11.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-02-26T04:59:11.796 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-02-26T04:59:11.797 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-02-26T04:59:11.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-02-26T04:59:11.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-02-26T04:59:11.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-02-26T04:59:11.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-02-26T04:59:11.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-02-26T04:59:11.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-02-26T04:59:11.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-02-26T04:59:11.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-02-26T04:59:11.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-02-26T04:59:11.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-02-26T04:59:11.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-02-26T04:59:11.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-02-26T04:59:11.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-02-26T04:59:11.798 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-02-26T04:59:11.799 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-02-26T04:59:11.800 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-02-26T04:59:11.800 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-02-26T04:59:11.800 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-02-26T04:59:11.800 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-02-26T04:59:11.800 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-02-26T04:59:11.800 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-02-26T04:59:11.800 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-02-26T04:59:11.800 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-02-26T04:59:11.800 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-02-26T04:59:12.229 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:11 smithi156 ceph-mon[35805]: pgmap v409: 129 pgs: 129 active+clean; 336 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s rd, 924 KiB/s wr, 278 op/s 2024-02-26T04:59:12.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:11 smithi060 ceph-mon[32382]: pgmap v409: 129 pgs: 129 active+clean; 336 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s rd, 924 KiB/s wr, 278 op/s 2024-02-26T04:59:12.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:11 smithi191 ceph-mon[36352]: pgmap v409: 129 pgs: 129 active+clean; 336 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s rd, 924 KiB/s wr, 278 op/s 2024-02-26T04:59:12.492 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-02-26T04:59:12.493 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-02-26T04:59:12.493 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-02-26T04:59:12.493 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-02-26T04:59:12.493 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-02-26T04:59:12.493 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-02-26T04:59:12.493 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-02-26T04:59:12.493 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-02-26T04:59:12.493 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-02-26T04:59:12.493 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-02-26T04:59:12.493 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-02-26T04:59:12.494 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-02-26T04:59:12.495 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-02-26T04:59:12.496 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-02-26T04:59:12.497 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-02-26T04:59:12.498 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-02-26T04:59:12.498 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-02-26T04:59:12.498 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-02-26T04:59:12.498 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-02-26T04:59:12.498 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-02-26T04:59:12.498 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-02-26T04:59:12.498 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-02-26T04:59:12.498 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-02-26T04:59:13.480 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-02-26T04:59:13.480 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-02-26T04:59:13.480 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-02-26T04:59:13.480 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-02-26T04:59:13.480 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-02-26T04:59:13.480 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-02-26T04:59:13.480 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-02-26T04:59:13.480 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-02-26T04:59:13.480 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-02-26T04:59:13.481 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-02-26T04:59:13.482 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-02-26T04:59:13.483 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-02-26T04:59:13.484 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-02-26T04:59:13.484 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-02-26T04:59:13.484 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-02-26T04:59:13.484 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-02-26T04:59:13.484 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-02-26T04:59:13.484 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-02-26T04:59:13.484 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-02-26T04:59:13.484 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-02-26T04:59:13.484 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-02-26T04:59:13.484 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-02-26T04:59:13.484 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-02-26T04:59:13.484 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-02-26T04:59:13.484 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-02-26T04:59:13.484 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-02-26T04:59:13.485 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-02-26T04:59:13.485 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-02-26T04:59:13.485 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-02-26T04:59:13.485 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-02-26T04:59:13.485 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-02-26T04:59:13.485 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-02-26T04:59:13.485 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-02-26T04:59:13.485 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-02-26T04:59:13.485 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-02-26T04:59:13.485 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-02-26T04:59:13.485 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-02-26T04:59:13.952 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-02-26T04:59:13.952 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-02-26T04:59:13.952 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-02-26T04:59:13.952 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-02-26T04:59:13.952 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-02-26T04:59:13.952 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-02-26T04:59:13.953 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-02-26T04:59:13.953 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-02-26T04:59:13.953 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-02-26T04:59:13.953 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-02-26T04:59:13.953 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-02-26T04:59:13.953 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-02-26T04:59:13.953 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-02-26T04:59:13.953 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-02-26T04:59:13.953 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-02-26T04:59:13.953 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-02-26T04:59:13.953 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-02-26T04:59:13.953 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-02-26T04:59:13.953 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-02-26T04:59:13.953 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-02-26T04:59:13.954 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-02-26T04:59:13.955 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-02-26T04:59:13.956 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-02-26T04:59:13.956 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-02-26T04:59:13.956 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-02-26T04:59:13.956 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-02-26T04:59:13.956 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-02-26T04:59:13.956 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-02-26T04:59:13.956 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-02-26T04:59:13.956 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-02-26T04:59:13.956 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-02-26T04:59:13.956 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-02-26T04:59:13.956 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-02-26T04:59:13.956 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-02-26T04:59:13.956 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-02-26T04:59:13.956 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-02-26T04:59:13.957 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-02-26T04:59:13.958 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-02-26T04:59:13.958 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-02-26T04:59:14.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:13 smithi060 ceph-mon[32382]: pgmap v410: 129 pgs: 129 active+clean; 358 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 114 MiB/s rd, 863 KiB/s wr, 289 op/s 2024-02-26T04:59:14.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:13 smithi191 ceph-mon[36352]: pgmap v410: 129 pgs: 129 active+clean; 358 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 114 MiB/s rd, 863 KiB/s wr, 289 op/s 2024-02-26T04:59:14.281 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-02-26T04:59:14.281 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-02-26T04:59:14.281 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-02-26T04:59:14.281 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-02-26T04:59:14.281 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-02-26T04:59:14.281 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-02-26T04:59:14.281 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-02-26T04:59:14.281 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-02-26T04:59:14.281 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-02-26T04:59:14.282 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-02-26T04:59:14.283 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-02-26T04:59:14.284 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-02-26T04:59:14.285 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-02-26T04:59:14.286 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-02-26T04:59:14.286 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-02-26T04:59:14.286 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-02-26T04:59:14.286 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-02-26T04:59:14.286 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-02-26T04:59:14.286 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-02-26T04:59:14.286 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-02-26T04:59:14.286 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-02-26T04:59:14.348 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-02-26T04:59:14.348 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-02-26T04:59:14.348 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-02-26T04:59:14.348 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-02-26T04:59:14.348 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-02-26T04:59:14.348 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-02-26T04:59:14.348 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-02-26T04:59:14.348 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-02-26T04:59:14.348 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-02-26T04:59:14.349 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-02-26T04:59:14.349 INFO:tasks.workunit.client.0.smithi060.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-02-26T04:59:14.349 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-02-26T04:59:14.349 INFO:tasks.workunit.client.0.smithi060.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-02-26T04:59:14.349 INFO:tasks.workunit.client.0.smithi060.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-02-26T04:59:14.349 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-26T04:59:14.349 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-26T04:59:14.384 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-02-26T04:59:14.384 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-26T04:59:14.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:13 smithi156 ceph-mon[35805]: pgmap v410: 129 pgs: 129 active+clean; 358 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 114 MiB/s rd, 863 KiB/s wr, 289 op/s 2024-02-26T04:59:14.710 DEBUG:teuthology.parallel:result is None 2024-02-26T04:59:14.710 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-26T04:59:14.744 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-26T04:59:14.745 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-02-26T04:59:14.755 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-02-26T04:59:16.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:15 smithi060 ceph-mon[32382]: pgmap v411: 129 pgs: 129 active+clean; 369 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s rd, 1.0 MiB/s wr, 291 op/s 2024-02-26T04:59:16.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:15 smithi191 ceph-mon[36352]: pgmap v411: 129 pgs: 129 active+clean; 369 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s rd, 1.0 MiB/s wr, 291 op/s 2024-02-26T04:59:16.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:15 smithi156 ceph-mon[35805]: pgmap v411: 129 pgs: 129 active+clean; 369 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s rd, 1.0 MiB/s wr, 291 op/s 2024-02-26T04:59:18.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:17 smithi060 ceph-mon[32382]: pgmap v412: 129 pgs: 129 active+clean; 602 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s rd, 1.0 MiB/s wr, 300 op/s 2024-02-26T04:59:18.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:17 smithi191 ceph-mon[36352]: pgmap v412: 129 pgs: 129 active+clean; 602 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s rd, 1.0 MiB/s wr, 300 op/s 2024-02-26T04:59:18.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:17 smithi156 ceph-mon[35805]: pgmap v412: 129 pgs: 129 active+clean; 602 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 108 MiB/s rd, 1.0 MiB/s wr, 300 op/s 2024-02-26T04:59:20.230 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:19 smithi060 ceph-mon[32382]: pgmap v413: 129 pgs: 129 active+clean; 658 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 726 KiB/s wr, 227 op/s 2024-02-26T04:59:20.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:19 smithi191 ceph-mon[36352]: pgmap v413: 129 pgs: 129 active+clean; 658 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 726 KiB/s wr, 227 op/s 2024-02-26T04:59:20.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:19 smithi156 ceph-mon[35805]: pgmap v413: 129 pgs: 129 active+clean; 658 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 726 KiB/s wr, 227 op/s 2024-02-26T04:59:22.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:21 smithi060 ceph-mon[32382]: pgmap v414: 129 pgs: 129 active+clean; 658 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s rd, 704 KiB/s wr, 198 op/s 2024-02-26T04:59:22.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:21 smithi191 ceph-mon[36352]: pgmap v414: 129 pgs: 129 active+clean; 658 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s rd, 704 KiB/s wr, 198 op/s 2024-02-26T04:59:22.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:21 smithi156 ceph-mon[35805]: pgmap v414: 129 pgs: 129 active+clean; 658 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s rd, 704 KiB/s wr, 198 op/s 2024-02-26T04:59:24.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:23 smithi060 ceph-mon[32382]: pgmap v415: 129 pgs: 129 active+clean; 373 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 627 KiB/s wr, 188 op/s 2024-02-26T04:59:24.259 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:23 smithi191 ceph-mon[36352]: pgmap v415: 129 pgs: 129 active+clean; 373 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 627 KiB/s wr, 188 op/s 2024-02-26T04:59:24.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:23 smithi156 ceph-mon[35805]: pgmap v415: 129 pgs: 129 active+clean; 373 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 627 KiB/s wr, 188 op/s 2024-02-26T04:59:26.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:26 smithi060 ceph-mon[32382]: pgmap v416: 129 pgs: 129 active+clean; 229 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 491 KiB/s wr, 138 op/s 2024-02-26T04:59:26.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:26 smithi191 ceph-mon[36352]: pgmap v416: 129 pgs: 129 active+clean; 229 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 491 KiB/s wr, 138 op/s 2024-02-26T04:59:26.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:26 smithi156 ceph-mon[35805]: pgmap v416: 129 pgs: 129 active+clean; 229 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 491 KiB/s wr, 138 op/s 2024-02-26T04:59:28.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:28 smithi060 ceph-mon[32382]: pgmap v417: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 233 KiB/s wr, 103 op/s 2024-02-26T04:59:28.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:28 smithi191 ceph-mon[36352]: pgmap v417: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 233 KiB/s wr, 103 op/s 2024-02-26T04:59:28.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:28 smithi156 ceph-mon[35805]: pgmap v417: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 233 KiB/s wr, 103 op/s 2024-02-26T04:59:30.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:30 smithi060 ceph-mon[32382]: pgmap v418: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 155 KiB/s wr, 68 op/s 2024-02-26T04:59:30.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:30 smithi191 ceph-mon[36352]: pgmap v418: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 155 KiB/s wr, 68 op/s 2024-02-26T04:59:30.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:30 smithi156 ceph-mon[35805]: pgmap v418: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 155 KiB/s wr, 68 op/s 2024-02-26T04:59:32.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:32 smithi060 ceph-mon[32382]: pgmap v419: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 126 KiB/s wr, 58 op/s 2024-02-26T04:59:32.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:32 smithi191 ceph-mon[36352]: pgmap v419: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 126 KiB/s wr, 58 op/s 2024-02-26T04:59:32.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:32 smithi156 ceph-mon[35805]: pgmap v419: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 126 KiB/s wr, 58 op/s 2024-02-26T04:59:34.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-26T04:59:34.169 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-26T04:59:34.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.285+0000 7ff2a1e60640 1 -- 172.21.15.60:0/716596290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff29c102370 msgr2=0x7ff29c1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.285+0000 7ff2a1e60640 1 --2- 172.21.15.60:0/716596290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff29c102370 0x7ff29c1027f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff290009900 tx=0x7ff29002bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:34.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.285+0000 7ff2a1e60640 1 -- 172.21.15.60:0/716596290 shutdown_connections 2024-02-26T04:59:34.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.285+0000 7ff2a1e60640 1 --2- 172.21.15.60:0/716596290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff29c102d30 0x7ff29c1095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.285+0000 7ff2a1e60640 1 --2- 172.21.15.60:0/716596290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff29c102370 0x7ff29c1027f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.285+0000 7ff2a1e60640 1 --2- 172.21.15.60:0/716596290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff29c101180 0x7ff29c1015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.285+0000 7ff2a1e60640 1 -- 172.21.15.60:0/716596290 >> 172.21.15.60:0/716596290 conn(0x7ff29c0fc6f0 msgr2=0x7ff29c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.285+0000 7ff2a1e60640 1 -- 172.21.15.60:0/716596290 shutdown_connections 2024-02-26T04:59:34.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.285+0000 7ff2a1e60640 1 -- 172.21.15.60:0/716596290 wait complete. 2024-02-26T04:59:34.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.286+0000 7ff2a1e60640 1 Processor -- start 2024-02-26T04:59:34.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.286+0000 7ff2a1e60640 1 -- start start 2024-02-26T04:59:34.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.286+0000 7ff2a1e60640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff29c101180 0x7ff29c197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.286+0000 7ff2a1e60640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff29c102370 0x7ff29c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.286+0000 7ff2a1e60640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff29c102d30 0x7ff29c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.286+0000 7ff2a1e60640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c19ed00 con 0x7ff29c102370 2024-02-26T04:59:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.286+0000 7ff2a1e60640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c19ee70 con 0x7ff29c101180 2024-02-26T04:59:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.286+0000 7ff2a1e60640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c19f030 con 0x7ff29c102d30 2024-02-26T04:59:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.286+0000 7ff29affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff29c102370 0x7ff29c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.286+0000 7ff29affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff29c102370 0x7ff29c197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43346/0 (socket says 172.21.15.60:43346) 2024-02-26T04:59:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.286+0000 7ff29affd640 1 -- 172.21.15.60:0/2218699029 learned_addr learned my addr 172.21.15.60:0/2218699029 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.287+0000 7ff29b7fe640 1 --2- 172.21.15.60:0/2218699029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff29c101180 0x7ff29c197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.287+0000 7ff29affd640 1 -- 172.21.15.60:0/2218699029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff29c102d30 msgr2=0x7ff29c19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.287+0000 7ff29bfff640 1 --2- 172.21.15.60:0/2218699029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff29c102d30 0x7ff29c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.287+0000 7ff29affd640 1 --2- 172.21.15.60:0/2218699029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff29c102d30 0x7ff29c19e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.287+0000 7ff29affd640 1 -- 172.21.15.60:0/2218699029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff29c101180 msgr2=0x7ff29c197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.287+0000 7ff29affd640 1 --2- 172.21.15.60:0/2218699029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff29c101180 0x7ff29c197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.287+0000 7ff29affd640 1 -- 172.21.15.60:0/2218699029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff290009560 con 0x7ff29c102370 2024-02-26T04:59:34.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.287+0000 7ff29affd640 1 --2- 172.21.15.60:0/2218699029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff29c102370 0x7ff29c197750 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff290006010 tx=0x7ff2900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.287+0000 7ff298ff9640 1 -- 172.21.15.60:0/2218699029 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2900044e0 con 0x7ff29c102370 2024-02-26T04:59:34.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.287+0000 7ff2a1e60640 1 -- 172.21.15.60:0/2218699029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29c19f2b0 con 0x7ff29c102370 2024-02-26T04:59:34.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.287+0000 7ff298ff9640 1 -- 172.21.15.60:0/2218699029 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2900054e0 con 0x7ff29c102370 2024-02-26T04:59:34.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.287+0000 7ff298ff9640 1 -- 172.21.15.60:0/2218699029 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29003e450 con 0x7ff29c102370 2024-02-26T04:59:34.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.287+0000 7ff2a1e60640 1 -- 172.21.15.60:0/2218699029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29c19f7c0 con 0x7ff29c102370 2024-02-26T04:59:34.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.290+0000 7ff298ff9640 1 -- 172.21.15.60:0/2218699029 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff2900035f0 con 0x7ff29c102370 2024-02-26T04:59:34.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.290+0000 7ff2a4c39740 1 Processor -- start 2024-02-26T04:59:34.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.290+0000 7ff2a4c39740 1 -- start start 2024-02-26T04:59:34.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.290+0000 7ff2a4c39740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ad6b0 0x55e0a13adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.290+0000 7ff2a4c39740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.290+0000 7ff2a4c39740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13b3210 0x55e0a13b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.290+0000 7ff2a4c39740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a1319250 con 0x55e0a13ae0a0 2024-02-26T04:59:34.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.290+0000 7ff2a4c39740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a1316780 con 0x55e0a13ad6b0 2024-02-26T04:59:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.290+0000 7ff2a4c39740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a1311f60 con 0x55e0a13b3210 2024-02-26T04:59:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.290+0000 7ff29b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.290+0000 7ff29b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43362/0 (socket says 172.21.15.60:43362) 2024-02-26T04:59:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.290+0000 7ff29b7fe640 1 -- 172.21.15.60:0/1951424605 learned_addr learned my addr 172.21.15.60:0/1951424605 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff298ff9640 1 --2- 172.21.15.60:0/2218699029 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff2840a20c0 0x7ff2840a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff29affd640 1 --2- 172.21.15.60:0/1951424605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13b3210 0x55e0a13b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff29bfff640 1 --2- 172.21.15.60:0/1951424605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ad6b0 0x55e0a13adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff29bfff640 1 --2- 172.21.15.60:0/2218699029 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff2840a20c0 0x7ff2840a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff298ff9640 1 -- 172.21.15.60:0/2218699029 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff2900f0540 con 0x7ff29c102370 2024-02-26T04:59:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff29b7fe640 1 -- 172.21.15.60:0/1951424605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13b3210 msgr2=0x55e0a13b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff29b7fe640 1 --2- 172.21.15.60:0/1951424605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13b3210 0x55e0a13b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff29b7fe640 1 -- 172.21.15.60:0/1951424605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ad6b0 msgr2=0x55e0a13adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff29b7fe640 1 --2- 172.21.15.60:0/1951424605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ad6b0 0x55e0a13adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff29b7fe640 1 -- 172.21.15.60:0/1951424605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0a12d0540 con 0x55e0a13ae0a0 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff29b7fe640 1 --2- 172.21.15.60:0/1951424605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b2ba0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00bd10 tx=0x7ff28c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12b6faed0b84d0b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff29bfff640 1 --2- 172.21.15.60:0/2218699029 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff2840a20c0 0x7ff2840a4580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55e0a13ae900 tx=0x7ff294005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff281ffb640 1 -- 172.21.15.60:0/1951424605 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c01a460 con 0x55e0a13ae0a0 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff281ffb640 1 -- 172.21.15.60:0/1951424605 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff28c01aaa0 con 0x55e0a13ae0a0 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.291+0000 7ff281ffb640 1 -- 172.21.15.60:0/1951424605 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c011aa0 con 0x55e0a13ae0a0 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.292+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1951424605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13ae0a0 msgr2=0x55e0a13b2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.292+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1951424605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b2ba0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00bd10 tx=0x7ff28c00f730 comp rx=0 tx=0).stop 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.292+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1951424605 shutdown_connections 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.292+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1951424605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13b3210 0x55e0a13b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.292+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1951424605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b2ba0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.292+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1951424605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ad6b0 0x55e0a13adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.292+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1951424605 >> 172.21.15.60:0/1951424605 conn(0x55e0a13a47c0 msgr2=0x55e0a13a4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.292+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1951424605 shutdown_connections 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.292+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1951424605 wait complete. 2024-02-26T04:59:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.292+0000 7ff2a4c39740 1 -- start start 2024-02-26T04:59:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff2a4c39740 1 Processor -- start 2024-02-26T04:59:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff2a4c39740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ad6b0 0x55e0a13b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff2a4c39740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff2a4c39740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13b3210 0x55e0a13bec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff2a4c39740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a0fe5430 con 0x55e0a13ae0a0 2024-02-26T04:59:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff2a4c39740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a0fa1e00 con 0x55e0a13b3210 2024-02-26T04:59:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff2a4c39740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a1296fa0 con 0x55e0a13ad6b0 2024-02-26T04:59:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff2a4c39740 10 client.0 _ll_register_callbacks cb 0x55e0a12ecfa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff29affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff29affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43366/0 (socket says 172.21.15.60:43366) 2024-02-26T04:59:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff29affd640 1 -- 172.21.15.60:0/1813093014 learned_addr learned my addr 172.21.15.60:0/1813093014 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff29bfff640 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13b3210 0x55e0a13bec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff29b7fe640 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ad6b0 0x55e0a13b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff29affd640 1 -- 172.21.15.60:0/1813093014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ad6b0 msgr2=0x55e0a13b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff29affd640 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ad6b0 0x55e0a13b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff29affd640 1 -- 172.21.15.60:0/1813093014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13b3210 msgr2=0x55e0a13bec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff29affd640 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13b3210 0x55e0a13bec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff29affd640 1 -- 172.21.15.60:0/1813093014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28c00b940 con 0x55e0a13ae0a0 2024-02-26T04:59:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff29b7fe640 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ad6b0 0x55e0a13b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:59:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.293+0000 7ff29bfff640 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13b3210 0x55e0a13bec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:59:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff29affd640 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b7c00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff290048000 tx=0x7ff29008c020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa00cfecaed1076 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff27affd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff27affd640 1 -- 172.21.15.60:0/1813093014 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2900bc790 con 0x55e0a13ae0a0 2024-02-26T04:59:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff27affd640 1 -- 172.21.15.60:0/1813093014 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff2900883e0 con 0x55e0a13ae0a0 2024-02-26T04:59:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1813093014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0a10117e0 con 0x55e0a13ae0a0 2024-02-26T04:59:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff27affd640 1 -- 172.21.15.60:0/1813093014 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2900bd530 con 0x55e0a13ae0a0 2024-02-26T04:59:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff2a4c39740 10 client.15837 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1813093014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e0a13105e0 con 0x55e0a13ae0a0 2024-02-26T04:59:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff27affd640 1 -- 172.21.15.60:0/1813093014 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2900d4020 con 0x55e0a13ae0a0 2024-02-26T04:59:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff27affd640 1 -- 172.21.15.60:0/1813093014 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff29003c070 con 0x55e0a13ae0a0 2024-02-26T04:59:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff2a4c39740 10 client.15837 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff2a4c39740 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:59:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff2a4c39740 20 client.15837 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55e0a13c3560 0x55e0a13c5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff2a4c39740 4 client.15837 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:59:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.294+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1813093014 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0a104e720 con 0x55e0a13c3560 2024-02-26T04:59:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.295+0000 7ff29b7fe640 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55e0a13c3560 0x55e0a13c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.295+0000 7ff29b7fe640 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55e0a13c3560 0x55e0a13c5950 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50ae4f8385149fde server_cookie=d16d32936a758072 in_seq=0 out_seq=0 2024-02-26T04:59:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.295+0000 7ff27affd640 10 client.15837 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff27affd640 1 -- 172.21.15.60:0/1813093014 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e0a104e720 con 0x55e0a13c3560 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff27affd640 10 client.15837 handle_command_reply: tid=0 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff2a4c39740 1 client.15837 shutdown 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1813093014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13ae0a0 msgr2=0x55e0a13b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b7c00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff290048000 tx=0x7ff29008c020 comp rx=0 tx=0).stop 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1813093014 shutdown_connections 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55e0a13c3560 0x55e0a13c5950 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13b3210 0x55e0a13bec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b7c00 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff27affd640 0 client.15837 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1813093014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ad6b0 0x55e0a13b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1813093014 >> 172.21.15.60:0/1813093014 conn(0x55e0a13a47c0 msgr2=0x55e0a13a3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1813093014 shutdown_connections 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1813093014 wait complete. 2024-02-26T04:59:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.296+0000 7ff2a4c39740 20 client.15837 trim_cache size 0 max 16384 2024-02-26T04:59:34.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.311+0000 7ff2a4c39740 1 Processor -- start 2024-02-26T04:59:34.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.311+0000 7ff2a4c39740 1 -- start start 2024-02-26T04:59:34.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.311+0000 7ff2a4c39740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ad6b0 0x55e0a13b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.311+0000 7ff2a4c39740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.311+0000 7ff2a4c39740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13b3210 0x55e0a13bec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.311+0000 7ff2a4c39740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a13b8440 con 0x55e0a13b3210 2024-02-26T04:59:34.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.311+0000 7ff2a4c39740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a13b5d60 con 0x55e0a13ae0a0 2024-02-26T04:59:34.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.311+0000 7ff2a4c39740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a13b7350 con 0x55e0a13ad6b0 2024-02-26T04:59:34.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.312+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13b3210 0x55e0a13bec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.312+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13b3210 0x55e0a13bec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43368/0 (socket says 172.21.15.60:43368) 2024-02-26T04:59:34.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.312+0000 7ff29bfff640 1 -- 172.21.15.60:0/3225140871 learned_addr learned my addr 172.21.15.60:0/3225140871 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.312+0000 7ff29affd640 1 --2- 172.21.15.60:0/3225140871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.312+0000 7ff29b7fe640 1 --2- 172.21.15.60:0/3225140871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ad6b0 0x55e0a13b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.312+0000 7ff29bfff640 1 -- 172.21.15.60:0/3225140871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ad6b0 msgr2=0x55e0a13b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.312+0000 7ff29bfff640 1 --2- 172.21.15.60:0/3225140871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ad6b0 0x55e0a13b76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.312+0000 7ff29bfff640 1 -- 172.21.15.60:0/3225140871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ae0a0 msgr2=0x55e0a13b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.312+0000 7ff29bfff640 1 --2- 172.21.15.60:0/3225140871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.312+0000 7ff29bfff640 1 -- 172.21.15.60:0/3225140871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0a12aca90 con 0x55e0a13b3210 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.312+0000 7ff29bfff640 1 --2- 172.21.15.60:0/3225140871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13b3210 0x55e0a13bec00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff294005f30 tx=0x7ff29400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2580adc6301f3700 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.312+0000 7ff2817fa640 1 -- 172.21.15.60:0/3225140871 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294012ea0 con 0x55e0a13b3210 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.312+0000 7ff2817fa640 1 -- 172.21.15.60:0/3225140871 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff29401de60 con 0x55e0a13b3210 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.312+0000 7ff2817fa640 1 -- 172.21.15.60:0/3225140871 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29401e680 con 0x55e0a13b3210 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.313+0000 7ff2a4c39740 1 -- 172.21.15.60:0/3225140871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13b3210 msgr2=0x55e0a13bec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.313+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/3225140871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13b3210 0x55e0a13bec00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff294005f30 tx=0x7ff29400a430 comp rx=0 tx=0).stop 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.313+0000 7ff2a4c39740 1 -- 172.21.15.60:0/3225140871 shutdown_connections 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.313+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/3225140871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13b3210 0x55e0a13bec00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.313+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/3225140871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ae0a0 0x55e0a13b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.313+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/3225140871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ad6b0 0x55e0a13b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.313+0000 7ff2a4c39740 1 -- 172.21.15.60:0/3225140871 >> 172.21.15.60:0/3225140871 conn(0x55e0a13a47c0 msgr2=0x55e0a13ca2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.313+0000 7ff2a4c39740 1 -- 172.21.15.60:0/3225140871 shutdown_connections 2024-02-26T04:59:34.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.313+0000 7ff2a4c39740 1 -- 172.21.15.60:0/3225140871 wait complete. 2024-02-26T04:59:34.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.313+0000 7ff2a4c39740 1 -- start start 2024-02-26T04:59:34.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.313+0000 7ff2a4c39740 1 Processor -- start 2024-02-26T04:59:34.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff2a4c39740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ad6b0 0x55e0a13beaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff2a4c39740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ae0a0 0x55e0a13bd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff2a4c39740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13b3210 0x55e0a13bd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff2a4c39740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a13bdcc0 con 0x55e0a13b3210 2024-02-26T04:59:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff2a4c39740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a13bde30 con 0x55e0a13ad6b0 2024-02-26T04:59:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff2a4c39740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a13bdff0 con 0x55e0a13ae0a0 2024-02-26T04:59:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff2a4c39740 10 client.0 _ll_register_callbacks cb 0x7ff28c00b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13b3210 0x55e0a13bd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13b3210 0x55e0a13bd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43378/0 (socket says 172.21.15.60:43378) 2024-02-26T04:59:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff29bfff640 1 -- 172.21.15.60:0/1384410881 learned_addr learned my addr 172.21.15.60:0/1384410881 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff29b7fe640 1 --2- 172.21.15.60:0/1384410881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ad6b0 0x55e0a13beaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff29affd640 1 --2- 172.21.15.60:0/1384410881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ae0a0 0x55e0a13bd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff29bfff640 1 -- 172.21.15.60:0/1384410881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ae0a0 msgr2=0x55e0a13bd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff29bfff640 1 --2- 172.21.15.60:0/1384410881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ae0a0 0x55e0a13bd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff29bfff640 1 -- 172.21.15.60:0/1384410881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ad6b0 msgr2=0x55e0a13beaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff29bfff640 1 --2- 172.21.15.60:0/1384410881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ad6b0 0x55e0a13beaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff29bfff640 1 -- 172.21.15.60:0/1384410881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29401d4b0 con 0x55e0a13b3210 2024-02-26T04:59:34.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff29bfff640 1 --2- 172.21.15.60:0/1384410881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13b3210 0x55e0a13bd6b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff29401d7a0 tx=0x7ff29400fdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6240fa5526e38899 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff27affd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:34.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff27affd640 1 -- 172.21.15.60:0/1384410881 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294012ea0 con 0x55e0a13b3210 2024-02-26T04:59:34.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff27affd640 1 -- 172.21.15.60:0/1384410881 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff294027470 con 0x55e0a13b3210 2024-02-26T04:59:34.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.314+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1384410881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff28c01aaa0 con 0x55e0a13b3210 2024-02-26T04:59:34.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.315+0000 7ff27affd640 1 -- 172.21.15.60:0/1384410881 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff294004e50 con 0x55e0a13b3210 2024-02-26T04:59:34.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.315+0000 7ff2a4c39740 10 client.15843 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:34.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.315+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1384410881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e0a12d2670 con 0x55e0a13b3210 2024-02-26T04:59:34.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.315+0000 7ff27affd640 1 -- 172.21.15.60:0/1384410881 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff28c01aaa0 con 0x55e0a13b3210 2024-02-26T04:59:34.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.315+0000 7ff27affd640 1 -- 172.21.15.60:0/1384410881 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff29401a040 con 0x55e0a13b3210 2024-02-26T04:59:34.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.315+0000 7ff2a4c39740 10 client.15843 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:34.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.315+0000 7ff2a4c39740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:59:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.315+0000 7ff2a4c39740 20 client.15843 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.315+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1384410881 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55e0a141b2a0 0x55e0a141d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.315+0000 7ff2a4c39740 4 client.15843 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-26T04:59:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.315+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1384410881 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e0a109d890 con 0x55e0a141b2a0 2024-02-26T04:59:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.315+0000 7ff29b7fe640 1 --2- 172.21.15.60:0/1384410881 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55e0a141b2a0 0x55e0a141d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff29b7fe640 1 --2- 172.21.15.60:0/1384410881 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55e0a141b2a0 0x55e0a141d6c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0b142bde867996b server_cookie=5cd0442167123bfc in_seq=0 out_seq=0 2024-02-26T04:59:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff27affd640 10 client.15843 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff27affd640 1 -- 172.21.15.60:0/1384410881 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e0a109d890 con 0x55e0a141b2a0 2024-02-26T04:59:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff27affd640 10 client.15843 handle_command_reply: tid=0 2024-02-26T04:59:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff2a4c39740 1 client.15843 shutdown 2024-02-26T04:59:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1384410881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13b3210 msgr2=0x55e0a13bd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1384410881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13b3210 0x55e0a13bd6b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff29401d7a0 tx=0x7ff29400fdb0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1384410881 shutdown_connections 2024-02-26T04:59:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1384410881 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x55e0a141b2a0 0x55e0a141d6c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1384410881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0a13b3210 0x55e0a13bd6b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1384410881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e0a13ae0a0 0x55e0a13bd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff27affd640 0 client.15843 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff2a4c39740 1 --2- 172.21.15.60:0/1384410881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0a13ad6b0 0x55e0a13beaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1384410881 >> 172.21.15.60:0/1384410881 conn(0x55e0a13a47c0 msgr2=0x55e0a13b7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1384410881 shutdown_connections 2024-02-26T04:59:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff2a4c39740 1 -- 172.21.15.60:0/1384410881 wait complete. 2024-02-26T04:59:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.316+0000 7ff2a4c39740 20 client.15843 trim_cache size 0 max 16384 2024-02-26T04:59:34.325 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:59:34.326 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-02-26T04:59:34.326 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-02-26T04:59:34.326 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:59:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.317+0000 7ff2a1e60640 1 -- 172.21.15.60:0/2218699029 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff2840a20c0 msgr2=0x7ff2840a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.317+0000 7ff2a1e60640 1 --2- 172.21.15.60:0/2218699029 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff2840a20c0 0x7ff2840a4580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55e0a13ae900 tx=0x7ff294005c80 comp rx=0 tx=0).stop 2024-02-26T04:59:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.317+0000 7ff2a1e60640 1 -- 172.21.15.60:0/2218699029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff29c102370 msgr2=0x7ff29c197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.317+0000 7ff2a1e60640 1 --2- 172.21.15.60:0/2218699029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff29c102370 0x7ff29c197750 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff290006010 tx=0x7ff2900042f0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.317+0000 7ff2a1e60640 1 -- 172.21.15.60:0/2218699029 shutdown_connections 2024-02-26T04:59:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.317+0000 7ff2a1e60640 1 --2- 172.21.15.60:0/2218699029 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff2840a20c0 0x7ff2840a4580 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.317+0000 7ff2a1e60640 1 --2- 172.21.15.60:0/2218699029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff29c102d30 0x7ff29c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.317+0000 7ff2a1e60640 1 --2- 172.21.15.60:0/2218699029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff29c102370 0x7ff29c197750 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.317+0000 7ff2a1e60640 1 --2- 172.21.15.60:0/2218699029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff29c101180 0x7ff29c197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.317+0000 7ff2a1e60640 1 -- 172.21.15.60:0/2218699029 >> 172.21.15.60:0/2218699029 conn(0x7ff29c0fc6f0 msgr2=0x7ff29c1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.317+0000 7ff2a1e60640 1 -- 172.21.15.60:0/2218699029 shutdown_connections 2024-02-26T04:59:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.317+0000 7ff2a1e60640 1 -- 172.21.15.60:0/2218699029 wait complete. 2024-02-26T04:59:34.330 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-26T04:59:34.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T04:59:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.431+0000 7f885b636640 1 -- 172.21.15.60:0/551662328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8854101c50 msgr2=0x7f88541084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.431+0000 7f885b636640 1 --2- 172.21.15.60:0/551662328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8854101c50 0x7f88541084e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f884c009ca0 tx=0x7f884c02c0f0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.431+0000 7f885b636640 1 -- 172.21.15.60:0/551662328 shutdown_connections 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.431+0000 7f885b636640 1 --2- 172.21.15.60:0/551662328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8854101c50 0x7f88541084e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.431+0000 7f885b636640 1 --2- 172.21.15.60:0/551662328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8854101290 0x7f8854101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.431+0000 7f885b636640 1 --2- 172.21.15.60:0/551662328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88541000a0 0x7f88541004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.431+0000 7f885b636640 1 -- 172.21.15.60:0/551662328 >> 172.21.15.60:0/551662328 conn(0x7f88540fb610 msgr2=0x7f88540fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.431+0000 7f885b636640 1 -- 172.21.15.60:0/551662328 shutdown_connections 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.431+0000 7f885b636640 1 -- 172.21.15.60:0/551662328 wait complete. 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.432+0000 7f885b636640 1 Processor -- start 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.432+0000 7f885b636640 1 -- start start 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.432+0000 7f885b636640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88541000a0 0x7f8854103fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.432+0000 7f885b636640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8854101290 0x7f8854102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.432+0000 7f885b636640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8854101c50 0x7f8854102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.432+0000 7f885b636640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8854103340 con 0x7f8854101290 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.432+0000 7f885b636640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88541034b0 con 0x7f8854101c50 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.432+0000 7f885b636640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88541a12b0 con 0x7f88541000a0 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.432+0000 7f8858bd1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8854101290 0x7f8854102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.432+0000 7f8858bd1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8854101290 0x7f8854102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43402/0 (socket says 172.21.15.60:43402) 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.432+0000 7f8858bd1640 1 -- 172.21.15.60:0/3393220573 learned_addr learned my addr 172.21.15.60:0/3393220573 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.433+0000 7f88593d2640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88541000a0 0x7f8854103fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.433+0000 7f8859bd3640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8854101c50 0x7f8854102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.433+0000 7f8858bd1640 1 -- 172.21.15.60:0/3393220573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88541000a0 msgr2=0x7f8854103fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.433+0000 7f8858bd1640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88541000a0 0x7f8854103fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.433+0000 7f8858bd1640 1 -- 172.21.15.60:0/3393220573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8854101c50 msgr2=0x7f8854102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.433+0000 7f8858bd1640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8854101c50 0x7f8854102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.433+0000 7f8858bd1640 1 -- 172.21.15.60:0/3393220573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c009900 con 0x7f8854101290 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.433+0000 7f8859bd3640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8854101c50 0x7f8854102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.433+0000 7f8858bd1640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8854101290 0x7f8854102670 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f884400ea20 tx=0x7f884400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.433+0000 7f884a7fc640 1 -- 172.21.15.60:0/3393220573 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844009720 con 0x7f8854101290 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.433+0000 7f884a7fc640 1 -- 172.21.15.60:0/3393220573 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f884400cd80 con 0x7f8854101290 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.433+0000 7f885b636640 1 -- 172.21.15.60:0/3393220573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88541a1510 con 0x7f8854101290 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.433+0000 7f884a7fc640 1 -- 172.21.15.60:0/3393220573 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844010490 con 0x7f8854101290 2024-02-26T04:59:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.433+0000 7f885b636640 1 -- 172.21.15.60:0/3393220573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88541a19d0 con 0x7f8854101290 2024-02-26T04:59:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.434+0000 7f885b636640 1 -- 172.21.15.60:0/3393220573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885410bab0 con 0x7f8854101290 2024-02-26T04:59:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.435+0000 7f884a7fc640 1 -- 172.21.15.60:0/3393220573 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8844005000 con 0x7f8854101290 2024-02-26T04:59:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.435+0000 7f884a7fc640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f88400a20c0 0x7f88400a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.436+0000 7f88593d2640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f88400a20c0 0x7f88400a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.436+0000 7f884a7fc640 1 -- 172.21.15.60:0/3393220573 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8844014070 con 0x7f8854101290 2024-02-26T04:59:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.436+0000 7f88593d2640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f88400a20c0 0x7f88400a4580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8850005fd0 tx=0x7f8850005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.439+0000 7f884a7fc640 1 -- 172.21.15.60:0/3393220573 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f88440972f0 con 0x7f8854101290 2024-02-26T04:59:34.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:34 smithi060 ceph-mon[32382]: pgmap v420: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 92 KiB/s wr, 52 op/s 2024-02-26T04:59:34.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:34 smithi191 ceph-mon[36352]: pgmap v420: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 92 KiB/s wr, 52 op/s 2024-02-26T04:59:34.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.660+0000 7f885b636640 1 -- 172.21.15.60:0/3393220573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f885410bc40 con 0x7f8854101290 2024-02-26T04:59:34.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.660+0000 7f884a7fc640 1 -- 172.21.15.60:0/3393220573 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11347 (secure 0 0 0) 0x7f884409a9b0 con 0x7f8854101290 2024-02-26T04:59:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:59:34.663 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:57:52.162616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24476,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T04:59:34.663 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-02-26T04:59:34.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.662+0000 7f885b636640 1 -- 172.21.15.60:0/3393220573 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f88400a20c0 msgr2=0x7f88400a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.663+0000 7f885b636640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f88400a20c0 0x7f88400a4580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8850005fd0 tx=0x7f8850005c70 comp rx=0 tx=0).stop 2024-02-26T04:59:34.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.663+0000 7f885b636640 1 -- 172.21.15.60:0/3393220573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8854101290 msgr2=0x7f8854102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.663+0000 7f885b636640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8854101290 0x7f8854102670 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f884400ea20 tx=0x7f884400ee20 comp rx=0 tx=0).stop 2024-02-26T04:59:34.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.663+0000 7f885b636640 1 -- 172.21.15.60:0/3393220573 shutdown_connections 2024-02-26T04:59:34.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.663+0000 7f885b636640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f88400a20c0 0x7f88400a4580 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.663+0000 7f885b636640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8854101c50 0x7f8854102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.663+0000 7f885b636640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8854101290 0x7f8854102670 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.663+0000 7f885b636640 1 --2- 172.21.15.60:0/3393220573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88541000a0 0x7f8854103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.663+0000 7f885b636640 1 -- 172.21.15.60:0/3393220573 >> 172.21.15.60:0/3393220573 conn(0x7f88540fb610 msgr2=0x7f88540fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.663+0000 7f885b636640 1 -- 172.21.15.60:0/3393220573 shutdown_connections 2024-02-26T04:59:34.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.663+0000 7f885b636640 1 -- 172.21.15.60:0/3393220573 wait complete. 2024-02-26T04:59:34.676 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T04:59:34.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-26T04:59:34.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:34 smithi156 ceph-mon[35805]: pgmap v420: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 92 KiB/s wr, 52 op/s 2024-02-26T04:59:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 -- 172.21.15.60:0/859140043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08040ff480 msgr2=0x7f08040ff8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 --2- 172.21.15.60:0/859140043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08040ff480 0x7f08040ff8a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0800009900 tx=0x7f080002bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 -- 172.21.15.60:0/859140043 shutdown_connections 2024-02-26T04:59:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 --2- 172.21.15.60:0/859140043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0804102eb0 0x7f0804109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 --2- 172.21.15.60:0/859140043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08041024f0 0x7f0804102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 --2- 172.21.15.60:0/859140043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08040ff480 0x7f08040ff8a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 -- 172.21.15.60:0/859140043 >> 172.21.15.60:0/859140043 conn(0x7f08040fadb0 msgr2=0x7f08040fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 -- 172.21.15.60:0/859140043 shutdown_connections 2024-02-26T04:59:34.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 -- 172.21.15.60:0/859140043 wait complete. 2024-02-26T04:59:34.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 Processor -- start 2024-02-26T04:59:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 -- start start 2024-02-26T04:59:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08040ff480 0x7f0804197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08041024f0 0x7f0804197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0804102eb0 0x7f080419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080419ed40 con 0x7f08040ff480 2024-02-26T04:59:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080419eeb0 con 0x7f08041024f0 2024-02-26T04:59:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.779+0000 7f080b974640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080419f070 con 0x7f0804102eb0 2024-02-26T04:59:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f0809710640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08040ff480 0x7f0804197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f0809710640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08040ff480 0x7f0804197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43428/0 (socket says 172.21.15.60:43428) 2024-02-26T04:59:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f0809710640 1 -- 172.21.15.60:0/432120541 learned_addr learned my addr 172.21.15.60:0/432120541 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f0808f0f640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08041024f0 0x7f0804197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f0809f11640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0804102eb0 0x7f080419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f0809710640 1 -- 172.21.15.60:0/432120541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0804102eb0 msgr2=0x7f080419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f0809710640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0804102eb0 0x7f080419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f0809710640 1 -- 172.21.15.60:0/432120541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08041024f0 msgr2=0x7f0804197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f0809710640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08041024f0 0x7f0804197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f0809710640 1 -- 172.21.15.60:0/432120541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800009560 con 0x7f08040ff480 2024-02-26T04:59:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f0808f0f640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08041024f0 0x7f0804197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:59:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f0809710640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08040ff480 0x7f0804197250 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0800005ab0 tx=0x7f0800004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f07fa7fc640 1 -- 172.21.15.60:0/432120541 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800004aa0 con 0x7f08040ff480 2024-02-26T04:59:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f080b974640 1 -- 172.21.15.60:0/432120541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080419f2f0 con 0x7f08040ff480 2024-02-26T04:59:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f07fa7fc640 1 -- 172.21.15.60:0/432120541 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08000052d0 con 0x7f08040ff480 2024-02-26T04:59:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f07fa7fc640 1 -- 172.21.15.60:0/432120541 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800003e80 con 0x7f08040ff480 2024-02-26T04:59:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.780+0000 7f080b974640 1 -- 172.21.15.60:0/432120541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080419f800 con 0x7f08040ff480 2024-02-26T04:59:34.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.782+0000 7f07fa7fc640 1 -- 172.21.15.60:0/432120541 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0800005440 con 0x7f08040ff480 2024-02-26T04:59:34.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f080e82f740 1 Processor -- start 2024-02-26T04:59:34.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f080e82f740 1 -- start start 2024-02-26T04:59:34.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f080e82f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedf240 0x56450cedf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f080e82f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedfc30 0x56450cee4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f080e82f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450cee7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f080e82f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450ce9b930 con 0x56450cedf240 2024-02-26T04:59:34.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f080e82f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450ce4f2e0 con 0x56450cee4da0 2024-02-26T04:59:34.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f080e82f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450ce4c810 con 0x56450cedfc30 2024-02-26T04:59:34.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f07fa7fc640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f07f00a20c0 0x7f07f00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f0809f11640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedf240 0x56450cedf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f0809f11640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedf240 0x56450cedf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43432/0 (socket says 172.21.15.60:43432) 2024-02-26T04:59:34.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f0809f11640 1 -- 172.21.15.60:0/2627977287 learned_addr learned my addr 172.21.15.60:0/2627977287 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f0809f11640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f07f00a20c0 0x7f07f00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f0809710640 1 --2- 172.21.15.60:0/2627977287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedfc30 0x56450cee4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f0808f0f640 1 --2- 172.21.15.60:0/2627977287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450cee7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f07fa7fc640 1 -- 172.21.15.60:0/432120541 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f08000f0630 con 0x7f08040ff480 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f0809f11640 1 -- 172.21.15.60:0/2627977287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedfc30 msgr2=0x56450cee4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f0809f11640 1 --2- 172.21.15.60:0/2627977287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedfc30 0x56450cee4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f0809f11640 1 -- 172.21.15.60:0/2627977287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 msgr2=0x56450cee7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f0809f11640 1 --2- 172.21.15.60:0/2627977287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450cee7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f0809f11640 1 -- 172.21.15.60:0/2627977287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56450ce0a030 con 0x56450cedf240 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.783+0000 7f0808f0f640 1 --2- 172.21.15.60:0/2627977287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450cee7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.784+0000 7f0809f11640 1 --2- 172.21.15.60:0/2627977287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedf240 0x56450cedf660 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f07fc014d60 tx=0x7f07fc011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400057c0ad022764 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.784+0000 7f0809f11640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f07f00a20c0 0x7f07f00a4580 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f07fc011c50 tx=0x7f07fc016000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.784+0000 7f07ef7fe640 1 -- 172.21.15.60:0/2627977287 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc00cb70 con 0x56450cedf240 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.784+0000 7f07ef7fe640 1 -- 172.21.15.60:0/2627977287 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07fc01a040 con 0x56450cedf240 2024-02-26T04:59:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.784+0000 7f07ef7fe640 1 -- 172.21.15.60:0/2627977287 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc005460 con 0x56450cedf240 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 -- 172.21.15.60:0/2627977287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedf240 msgr2=0x56450cedf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 --2- 172.21.15.60:0/2627977287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedf240 0x56450cedf660 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f07fc014d60 tx=0x7f07fc011750 comp rx=0 tx=0).stop 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 -- 172.21.15.60:0/2627977287 shutdown_connections 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 --2- 172.21.15.60:0/2627977287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450cee7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 --2- 172.21.15.60:0/2627977287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedfc30 0x56450cee4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 --2- 172.21.15.60:0/2627977287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedf240 0x56450cedf660 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 -- 172.21.15.60:0/2627977287 >> 172.21.15.60:0/2627977287 conn(0x56450ceda7b0 msgr2=0x56450cedabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 -- 172.21.15.60:0/2627977287 shutdown_connections 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 -- 172.21.15.60:0/2627977287 wait complete. 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 -- start start 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 Processor -- start 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedf240 0x56450cee94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedfc30 0x56450cee99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450cef0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450ce47ff0 con 0x56450cedf240 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cb1b430 con 0x56450cee4da0 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cad7e00 con 0x56450cedfc30 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f080e82f740 10 client.0 _ll_register_callbacks cb 0x56450cdefc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f0809710640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedf240 0x56450cee94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f0809710640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedf240 0x56450cee94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43436/0 (socket says 172.21.15.60:43436) 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.785+0000 7f0809710640 1 -- 172.21.15.60:0/758503598 learned_addr learned my addr 172.21.15.60:0/758503598 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f0808f0f640 1 --2- 172.21.15.60:0/758503598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedfc30 0x56450cee99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f0809f11640 1 --2- 172.21.15.60:0/758503598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450cef0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f0809710640 1 -- 172.21.15.60:0/758503598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedfc30 msgr2=0x56450cee99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f0809710640 1 --2- 172.21.15.60:0/758503598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedfc30 0x56450cee99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f0809710640 1 -- 172.21.15.60:0/758503598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 msgr2=0x56450cef0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f0809710640 1 --2- 172.21.15.60:0/758503598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450cef0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f0809710640 1 -- 172.21.15.60:0/758503598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07fc014940 con 0x56450cedf240 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f0809f11640 1 --2- 172.21.15.60:0/758503598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450cef0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f0809710640 1 --2- 172.21.15.60:0/758503598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedf240 0x56450cee94b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f08000b9380 tx=0x7f080002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6320bc8c6820b2e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f07ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f07ecff9640 1 -- 172.21.15.60:0/758503598 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08000bcf20 con 0x56450cedf240 2024-02-26T04:59:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f07ecff9640 1 -- 172.21.15.60:0/758503598 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f08000c1070 con 0x56450cedf240 2024-02-26T04:59:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f080e82f740 1 -- 172.21.15.60:0/758503598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56450cb477e0 con 0x56450cedf240 2024-02-26T04:59:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f07ecff9640 1 -- 172.21.15.60:0/758503598 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08000c56c0 con 0x56450cedf240 2024-02-26T04:59:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f080e82f740 10 client.15861 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f080e82f740 1 -- 172.21.15.60:0/758503598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56450ce46670 con 0x56450cedf240 2024-02-26T04:59:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.786+0000 7f07ecff9640 1 -- 172.21.15.60:0/758503598 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56450cb477e0 con 0x56450cedf240 2024-02-26T04:59:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.787+0000 7f07ecff9640 1 -- 172.21.15.60:0/758503598 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f080003c070 con 0x56450cedf240 2024-02-26T04:59:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.787+0000 7f080e82f740 10 client.15861 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.787+0000 7f080e82f740 10 client.15861 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-02-26T04:59:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.787+0000 7f080e82f740 20 client.15861 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.787+0000 7f080e82f740 1 --2- 172.21.15.60:0/758503598 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x56450cef55d0 0x56450cef79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.787+0000 7f080e82f740 4 client.15861 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:59:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.787+0000 7f080e82f740 1 -- 172.21.15.60:0/758503598 --> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56450cb84720 con 0x56450cef55d0 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.787+0000 7f0808f0f640 1 --2- 172.21.15.60:0/758503598 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x56450cef55d0 0x56450cef79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.787+0000 7f0808f0f640 1 --2- 172.21.15.60:0/758503598 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x56450cef55d0 0x56450cef79c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5395e776dec829bc server_cookie=d9c4751a7ab9d906 in_seq=0 out_seq=0 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.787+0000 7f07ecff9640 10 client.15861 ms_handle_connect on v2:172.21.15.191:6838/640686517 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f07ecff9640 1 -- 172.21.15.60:0/758503598 <== mds.2 v2:172.21.15.191:6838/640686517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56450cb84720 con 0x56450cef55d0 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f07ecff9640 10 client.15861 handle_command_reply: tid=0 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f080e82f740 1 client.15861 shutdown 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f080e82f740 1 -- 172.21.15.60:0/758503598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedf240 msgr2=0x56450cee94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f080e82f740 1 --2- 172.21.15.60:0/758503598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedf240 0x56450cee94b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f08000b9380 tx=0x7f080002bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f080e82f740 1 -- 172.21.15.60:0/758503598 shutdown_connections 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f080e82f740 1 --2- 172.21.15.60:0/758503598 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x56450cef55d0 0x56450cef79c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f080e82f740 1 --2- 172.21.15.60:0/758503598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450cef0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f080e82f740 1 --2- 172.21.15.60:0/758503598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedfc30 0x56450cee99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f07ecff9640 0 client.15861 ms_handle_reset on v2:172.21.15.191:6838/640686517 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f080e82f740 1 --2- 172.21.15.60:0/758503598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedf240 0x56450cee94b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f080e82f740 1 -- 172.21.15.60:0/758503598 >> 172.21.15.60:0/758503598 conn(0x56450ceda7b0 msgr2=0x56450ced9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f080e82f740 1 -- 172.21.15.60:0/758503598 shutdown_connections 2024-02-26T04:59:34.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f080e82f740 1 -- 172.21.15.60:0/758503598 wait complete. 2024-02-26T04:59:34.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.788+0000 7f080e82f740 20 client.15861 trim_cache size 0 max 16384 2024-02-26T04:59:34.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.800+0000 7f080e82f740 1 Processor -- start 2024-02-26T04:59:34.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.800+0000 7f080e82f740 1 -- start start 2024-02-26T04:59:34.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.800+0000 7f080e82f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedf240 0x56450ced92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.800+0000 7f080e82f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedfc30 0x56450cef09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.800+0000 7f080e82f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450ced9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.800+0000 7f080e82f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cdcd100 con 0x56450cedfc30 2024-02-26T04:59:34.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.800+0000 7f080e82f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cb84720 con 0x56450cee4da0 2024-02-26T04:59:34.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.800+0000 7f080e82f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cb477e0 con 0x56450cedf240 2024-02-26T04:59:34.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.800+0000 7f0808f0f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedfc30 0x56450cef09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.800+0000 7f0808f0f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedfc30 0x56450cef09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43440/0 (socket says 172.21.15.60:43440) 2024-02-26T04:59:34.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.800+0000 7f0808f0f640 1 -- 172.21.15.60:0/2236972124 learned_addr learned my addr 172.21.15.60:0/2236972124 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.801+0000 7f0809710640 1 --2- 172.21.15.60:0/2236972124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedf240 0x56450ced92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.801+0000 7f0808f0f640 1 -- 172.21.15.60:0/2236972124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedf240 msgr2=0x56450ced92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.801+0000 7f0808f0f640 1 --2- 172.21.15.60:0/2236972124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedf240 0x56450ced92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.801+0000 7f0808f0f640 1 -- 172.21.15.60:0/2236972124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 msgr2=0x56450ced9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.801+0000 7f0808f0f640 1 --2- 172.21.15.60:0/2236972124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450ced9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.801+0000 7f0808f0f640 1 -- 172.21.15.60:0/2236972124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56450cde2bf0 con 0x56450cedfc30 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.801+0000 7f0809710640 1 --2- 172.21.15.60:0/2236972124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedf240 0x56450ced92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.801+0000 7f0808f0f640 1 --2- 172.21.15.60:0/2236972124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedfc30 0x56450cef09c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x56450cef82f0 tx=0x7f07f4014900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21da33497151d41f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.801+0000 7f07eeffd640 1 -- 172.21.15.60:0/2236972124 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f4018040 con 0x56450cedfc30 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.801+0000 7f07eeffd640 1 -- 172.21.15.60:0/2236972124 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07f4014190 con 0x56450cedfc30 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.801+0000 7f07eeffd640 1 -- 172.21.15.60:0/2236972124 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f40038e0 con 0x56450cedfc30 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.801+0000 7f080e82f740 1 -- 172.21.15.60:0/2236972124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedfc30 msgr2=0x56450cef09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.801+0000 7f080e82f740 1 --2- 172.21.15.60:0/2236972124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedfc30 0x56450cef09c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x56450cef82f0 tx=0x7f07f4014900 comp rx=0 tx=0).stop 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 -- 172.21.15.60:0/2236972124 shutdown_connections 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 --2- 172.21.15.60:0/2236972124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450ced9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 --2- 172.21.15.60:0/2236972124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedfc30 0x56450cef09c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 --2- 172.21.15.60:0/2236972124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedf240 0x56450ced92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 -- 172.21.15.60:0/2236972124 >> 172.21.15.60:0/2236972124 conn(0x56450ceda7b0 msgr2=0x56450cee7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 -- 172.21.15.60:0/2236972124 shutdown_connections 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 -- 172.21.15.60:0/2236972124 wait complete. 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 -- start start 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 Processor -- start 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedf240 0x56450ceef260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedfc30 0x56450ceef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450ceefce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cf4bab0 con 0x56450cedfc30 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cf4bbf0 con 0x56450cee4da0 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cf4bdb0 con 0x56450cedf240 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f080e82f740 10 client.0 _ll_register_callbacks cb 0x7f07fc009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f0808f0f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedfc30 0x56450ceef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f0808f0f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedfc30 0x56450ceef7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43452/0 (socket says 172.21.15.60:43452) 2024-02-26T04:59:34.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.802+0000 7f0808f0f640 1 -- 172.21.15.60:0/2592207255 learned_addr learned my addr 172.21.15.60:0/2592207255 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f0808f0f640 1 -- 172.21.15.60:0/2592207255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedf240 msgr2=0x56450ceef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f0809f11640 1 --2- 172.21.15.60:0/2592207255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450ceefce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f0809710640 1 --2- 172.21.15.60:0/2592207255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedf240 0x56450ceef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f0808f0f640 1 --2- 172.21.15.60:0/2592207255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedf240 0x56450ceef260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f0808f0f640 1 -- 172.21.15.60:0/2592207255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 msgr2=0x56450ceefce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f0808f0f640 1 --2- 172.21.15.60:0/2592207255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450ceefce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f0808f0f640 1 -- 172.21.15.60:0/2592207255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f4009e10 con 0x56450cedfc30 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f0809f11640 1 --2- 172.21.15.60:0/2592207255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450ceefce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f0808f0f640 1 --2- 172.21.15.60:0/2592207255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedfc30 0x56450ceef7a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f07f4005860 tx=0x7f07f401d620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5679aae3419f2f90 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f07ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f07ecff9640 1 -- 172.21.15.60:0/2592207255 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f4018040 con 0x56450cedfc30 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f07ecff9640 1 -- 172.21.15.60:0/2592207255 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f07f40052d0 con 0x56450cedfc30 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f080e82f740 1 -- 172.21.15.60:0/2592207255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07fc01a040 con 0x56450cedfc30 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f07ecff9640 1 -- 172.21.15.60:0/2592207255 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07f4005490 con 0x56450cedfc30 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f080e82f740 10 client.15867 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:34.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f080e82f740 1 -- 172.21.15.60:0/2592207255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56450ce08890 con 0x56450cedfc30 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f07ecff9640 1 -- 172.21.15.60:0/2592207255 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07fc01a040 con 0x56450cedfc30 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.803+0000 7f07ecff9640 1 -- 172.21.15.60:0/2592207255 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07f40268b0 con 0x56450cedfc30 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.804+0000 7f080e82f740 10 client.15867 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.804+0000 7f080e82f740 10 client.15867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.804+0000 7f080e82f740 20 client.15867 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.804+0000 7f080e82f740 1 --2- 172.21.15.60:0/2592207255 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x56450cf4d780 0x56450cf4fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.804+0000 7f080e82f740 4 client.15867 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.804+0000 7f080e82f740 1 -- 172.21.15.60:0/2592207255 --> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56450cbd3890 con 0x56450cf4d780 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.804+0000 7f0809710640 1 --2- 172.21.15.60:0/2592207255 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x56450cf4d780 0x56450cf4fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.804+0000 7f0809710640 1 --2- 172.21.15.60:0/2592207255 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x56450cf4d780 0x56450cf4fba0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a98fb72f8e4afd59 server_cookie=448c67c7a9b5b89 in_seq=0 out_seq=0 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.804+0000 7f07ecff9640 10 client.15867 ms_handle_connect on v2:172.21.15.191:6838/640686517 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f07ecff9640 1 -- 172.21.15.60:0/2592207255 <== mds.2 v2:172.21.15.191:6838/640686517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56450cbd3890 con 0x56450cf4d780 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f07ecff9640 10 client.15867 handle_command_reply: tid=0 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080e82f740 1 client.15867 shutdown 2024-02-26T04:59:34.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080e82f740 1 -- 172.21.15.60:0/2592207255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedfc30 msgr2=0x56450ceef7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080e82f740 1 --2- 172.21.15.60:0/2592207255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedfc30 0x56450ceef7a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f07f4005860 tx=0x7f07f401d620 comp rx=0 tx=0).stop 2024-02-26T04:59:34.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080e82f740 1 -- 172.21.15.60:0/2592207255 shutdown_connections 2024-02-26T04:59:34.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080e82f740 1 --2- 172.21.15.60:0/2592207255 >> [v2:172.21.15.191:6838/640686517,v1:172.21.15.191:6839/640686517] conn(0x56450cf4d780 0x56450cf4fba0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080e82f740 1 --2- 172.21.15.60:0/2592207255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56450cee4da0 0x56450ceefce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080e82f740 1 --2- 172.21.15.60:0/2592207255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56450cedfc30 0x56450ceef7a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f07ecff9640 0 client.15867 ms_handle_reset on v2:172.21.15.191:6838/640686517 2024-02-26T04:59:34.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080e82f740 1 --2- 172.21.15.60:0/2592207255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56450cedf240 0x56450ceef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080e82f740 1 -- 172.21.15.60:0/2592207255 >> 172.21.15.60:0/2592207255 conn(0x56450ceda7b0 msgr2=0x56450cede2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080e82f740 1 -- 172.21.15.60:0/2592207255 shutdown_connections 2024-02-26T04:59:34.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080e82f740 1 -- 172.21.15.60:0/2592207255 wait complete. 2024-02-26T04:59:34.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080e82f740 20 client.15867 trim_cache size 0 max 16384 2024-02-26T04:59:34.813 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T04:59:34.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080b974640 1 -- 172.21.15.60:0/432120541 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f07f00a20c0 msgr2=0x7f07f00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080b974640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f07f00a20c0 0x7f07f00a4580 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f07fc011c50 tx=0x7f07fc016000 comp rx=0 tx=0).stop 2024-02-26T04:59:34.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080b974640 1 -- 172.21.15.60:0/432120541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08040ff480 msgr2=0x7f0804197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080b974640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08040ff480 0x7f0804197250 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0800005ab0 tx=0x7f0800004270 comp rx=0 tx=0).stop 2024-02-26T04:59:34.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080b974640 1 -- 172.21.15.60:0/432120541 shutdown_connections 2024-02-26T04:59:34.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080b974640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f07f00a20c0 0x7f07f00a4580 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080b974640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0804102eb0 0x7f080419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080b974640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08041024f0 0x7f0804197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080b974640 1 --2- 172.21.15.60:0/432120541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08040ff480 0x7f0804197250 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.805+0000 7f080b974640 1 -- 172.21.15.60:0/432120541 >> 172.21.15.60:0/432120541 conn(0x7f08040fadb0 msgr2=0x7f08041057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.806+0000 7f080b974640 1 -- 172.21.15.60:0/432120541 shutdown_connections 2024-02-26T04:59:34.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.806+0000 7f080b974640 1 -- 172.21.15.60:0/432120541 wait complete. 2024-02-26T04:59:34.819 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T04:59:34.819 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-26T04:59:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.921+0000 7fb7fb808640 1 -- 172.21.15.60:0/2215398615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f4100090 msgr2=0x7fb7f41004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.921+0000 7fb7fb808640 1 --2- 172.21.15.60:0/2215398615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f4100090 0x7fb7f41004b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009900 tx=0x7fb7e402bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.921+0000 7fb7fb808640 1 -- 172.21.15.60:0/2215398615 shutdown_connections 2024-02-26T04:59:34.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.921+0000 7fb7fb808640 1 --2- 172.21.15.60:0/2215398615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7f4101c40 0x7fb7f41084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.921+0000 7fb7fb808640 1 --2- 172.21.15.60:0/2215398615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7f4101280 0x7fb7f4101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.921+0000 7fb7fb808640 1 --2- 172.21.15.60:0/2215398615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f4100090 0x7fb7f41004b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.921+0000 7fb7fb808640 1 -- 172.21.15.60:0/2215398615 >> 172.21.15.60:0/2215398615 conn(0x7fb7f40fb620 msgr2=0x7fb7f40fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.921+0000 7fb7fb808640 1 -- 172.21.15.60:0/2215398615 shutdown_connections 2024-02-26T04:59:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.921+0000 7fb7fb808640 1 -- 172.21.15.60:0/2215398615 wait complete. 2024-02-26T04:59:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7fb808640 1 Processor -- start 2024-02-26T04:59:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7fb808640 1 -- start start 2024-02-26T04:59:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7fb808640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7f4100090 0x7fb7f4199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7fb808640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f4101280 0x7fb7f4199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7fb808640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7f4101c40 0x7fb7f41a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7fb808640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f41a0f80 con 0x7fb7f4101280 2024-02-26T04:59:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7fb808640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f41a10f0 con 0x7fb7f4101c40 2024-02-26T04:59:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7fb808640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f41a12b0 con 0x7fb7f4100090 2024-02-26T04:59:34.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7f8da3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f4101280 0x7fb7f4199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7f8da3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f4101280 0x7fb7f4199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43460/0 (socket says 172.21.15.60:43460) 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7f8da3640 1 -- 172.21.15.60:0/1549306700 learned_addr learned my addr 172.21.15.60:0/1549306700 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7f95a4640 1 --2- 172.21.15.60:0/1549306700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7f4100090 0x7fb7f4199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7f9da5640 1 --2- 172.21.15.60:0/1549306700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7f4101c40 0x7fb7f41a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7f8da3640 1 -- 172.21.15.60:0/1549306700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7f4100090 msgr2=0x7fb7f4199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7f8da3640 1 --2- 172.21.15.60:0/1549306700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7f4100090 0x7fb7f4199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7f8da3640 1 -- 172.21.15.60:0/1549306700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7f4101c40 msgr2=0x7fb7f41a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7f8da3640 1 --2- 172.21.15.60:0/1549306700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7f4101c40 0x7fb7f41a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7f8da3640 1 -- 172.21.15.60:0/1549306700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e4009560 con 0x7fb7f4101280 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.922+0000 7fb7f8da3640 1 --2- 172.21.15.60:0/1549306700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f4101280 0x7fb7f4199a40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000b4a0 tx=0x7fb7f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.923+0000 7fb7ea7fc640 1 -- 172.21.15.60:0/1549306700 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0010610 con 0x7fb7f4101280 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.923+0000 7fb7ea7fc640 1 -- 172.21.15.60:0/1549306700 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7f0010c50 con 0x7fb7f4101280 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.923+0000 7fb7fb808640 1 -- 172.21.15.60:0/1549306700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f41a1590 con 0x7fb7f4101280 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.923+0000 7fb7ea7fc640 1 -- 172.21.15.60:0/1549306700 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0017e20 con 0x7fb7f4101280 2024-02-26T04:59:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.923+0000 7fb7fb808640 1 -- 172.21.15.60:0/1549306700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f41a1b20 con 0x7fb7f4101280 2024-02-26T04:59:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.925+0000 7fb7ea7fc640 1 -- 172.21.15.60:0/1549306700 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb7f0017480 con 0x7fb7f4101280 2024-02-26T04:59:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.925+0000 7fb7fe5dc740 1 Processor -- start 2024-02-26T04:59:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.925+0000 7fb7fe5dc740 1 -- start start 2024-02-26T04:59:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.925+0000 7fb7fe5dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8240 0x556d83fd8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.925+0000 7fb7fe5dc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8c30 0x556d83fdd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.925+0000 7fb7fe5dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 0x556d83fe0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.925+0000 7fb7fe5dc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d83f94930 con 0x556d83fd8c30 2024-02-26T04:59:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.925+0000 7fb7fe5dc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d83f482e0 con 0x556d83fddda0 2024-02-26T04:59:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.925+0000 7fb7fe5dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d83f45810 con 0x556d83fd8240 2024-02-26T04:59:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.925+0000 7fb7f95a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8c30 0x556d83fdd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7ea7fc640 1 --2- 172.21.15.60:0/1549306700 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb7e00a20c0 0x7fb7e00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7f95a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8c30 0x556d83fdd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43472/0 (socket says 172.21.15.60:43472) 2024-02-26T04:59:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7f95a4640 1 -- 172.21.15.60:0/763478681 learned_addr learned my addr 172.21.15.60:0/763478681 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7f9da5640 1 --2- 172.21.15.60:0/763478681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8240 0x556d83fd8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7f9da5640 1 --2- 172.21.15.60:0/1549306700 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb7e00a20c0 0x7fb7e00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7ea7fc640 1 -- 172.21.15.60:0/1549306700 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fb7f00cd7d0 con 0x7fb7f4101280 2024-02-26T04:59:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7f8da3640 1 --2- 172.21.15.60:0/763478681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 0x556d83fe0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7f95a4640 1 -- 172.21.15.60:0/763478681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8240 msgr2=0x556d83fd8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7f95a4640 1 --2- 172.21.15.60:0/763478681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8240 0x556d83fd8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7f95a4640 1 -- 172.21.15.60:0/763478681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 msgr2=0x556d83fe0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7f95a4640 1 --2- 172.21.15.60:0/763478681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 0x556d83fe0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7f95a4640 1 -- 172.21.15.60:0/763478681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d83f03030 con 0x556d83fd8c30 2024-02-26T04:59:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7f8da3640 1 --2- 172.21.15.60:0/763478681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 0x556d83fe0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:59:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7f95a4640 1 --2- 172.21.15.60:0/763478681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8c30 0x556d83fdd730 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009900 tx=0x7fb7e402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3ee48dd9262d74 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7f9da5640 1 --2- 172.21.15.60:0/1549306700 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb7e00a20c0 0x7fb7e00a4580 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x556d83fd9490 tx=0x7fb7ec004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7df7fe640 1 -- 172.21.15.60:0/763478681 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40048c0 con 0x556d83fd8c30 2024-02-26T04:59:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7df7fe640 1 -- 172.21.15.60:0/763478681 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7e4004510 con 0x556d83fd8c30 2024-02-26T04:59:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.926+0000 7fb7df7fe640 1 -- 172.21.15.60:0/763478681 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40055a0 con 0x556d83fd8c30 2024-02-26T04:59:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.927+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/763478681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8c30 msgr2=0x556d83fdd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.927+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/763478681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8c30 0x556d83fdd730 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009900 tx=0x7fb7e402ca10 comp rx=0 tx=0).stop 2024-02-26T04:59:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.927+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/763478681 shutdown_connections 2024-02-26T04:59:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.927+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/763478681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 0x556d83fe0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.927+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/763478681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8c30 0x556d83fdd730 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.927+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/763478681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8240 0x556d83fd8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.927+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/763478681 >> 172.21.15.60:0/763478681 conn(0x556d83fd37b0 msgr2=0x556d83fd3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.927+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/763478681 shutdown_connections 2024-02-26T04:59:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.927+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/763478681 wait complete. 2024-02-26T04:59:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.927+0000 7fb7fe5dc740 1 -- start start 2024-02-26T04:59:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.927+0000 7fb7fe5dc740 1 Processor -- start 2024-02-26T04:59:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7fe5dc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8240 0x556d83fe23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7fe5dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8c30 0x556d83fe28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7fe5dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 0x556d83fe9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7fe5dc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d83f40ff0 con 0x556d83fd8240 2024-02-26T04:59:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7fe5dc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d83c14430 con 0x556d83fddda0 2024-02-26T04:59:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7fe5dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d83bd0e00 con 0x556d83fd8c30 2024-02-26T04:59:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7fe5dc740 10 client.0 _ll_register_callbacks cb 0x556d83ee8c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7f95a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8240 0x556d83fe23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7f95a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8240 0x556d83fe23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43486/0 (socket says 172.21.15.60:43486) 2024-02-26T04:59:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7f95a4640 1 -- 172.21.15.60:0/504300779 learned_addr learned my addr 172.21.15.60:0/504300779 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7f8da3640 1 --2- 172.21.15.60:0/504300779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8c30 0x556d83fe28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7f9da5640 1 --2- 172.21.15.60:0/504300779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 0x556d83fe9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7f95a4640 1 -- 172.21.15.60:0/504300779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8c30 msgr2=0x556d83fe28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7f95a4640 1 --2- 172.21.15.60:0/504300779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8c30 0x556d83fe28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7f95a4640 1 -- 172.21.15.60:0/504300779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 msgr2=0x556d83fe9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7f95a4640 1 --2- 172.21.15.60:0/504300779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 0x556d83fe9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7f95a4640 1 -- 172.21.15.60:0/504300779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f4108dd0 con 0x556d83fd8240 2024-02-26T04:59:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7f9da5640 1 --2- 172.21.15.60:0/504300779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 0x556d83fe9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:59:34.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7f95a4640 1 --2- 172.21.15.60:0/504300779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8240 0x556d83fe23a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4036040 tx=0x7fb7e4005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb73501d1157c308 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7dcff9640 1 -- 172.21.15.60:0/504300779 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4034640 con 0x556d83fd8240 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7dcff9640 1 -- 172.21.15.60:0/504300779 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fb7e4004a10 con 0x556d83fd8240 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.928+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/504300779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d83c407e0 con 0x556d83fd8240 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.929+0000 7fb7dcff9640 1 -- 172.21.15.60:0/504300779 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7e40347a0 con 0x556d83fd8240 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.929+0000 7fb7fe5dc740 10 client.15879 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.929+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/504300779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556d83f3f670 con 0x556d83fd8240 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.929+0000 7fb7dcff9640 1 -- 172.21.15.60:0/504300779 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556d83c407e0 con 0x556d83fd8240 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.929+0000 7fb7dcff9640 1 -- 172.21.15.60:0/504300779 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb7e403c040 con 0x556d83fd8240 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.929+0000 7fb7fe5dc740 10 client.15879 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.929+0000 7fb7fe5dc740 10 client.15879 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.929+0000 7fb7fe5dc740 20 client.15879 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.929+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/504300779 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x556d83fee110 0x556d83ff0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.929+0000 7fb7fe5dc740 4 client.15879 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:59:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.929+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/504300779 --> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d83c7d720 con 0x556d83fee110 2024-02-26T04:59:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.929+0000 7fb7f8da3640 1 --2- 172.21.15.60:0/504300779 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x556d83fee110 0x556d83ff0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.930+0000 7fb7dcff9640 10 client.15879 ms_handle_connect on v2:172.21.15.156:6836/707575792 2024-02-26T04:59:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.930+0000 7fb7f8da3640 1 --2- 172.21.15.60:0/504300779 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x556d83fee110 0x556d83ff0500 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7881a4a32cbb059e server_cookie=d1c3145c75d29fe4 in_seq=0 out_seq=0 2024-02-26T04:59:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.930+0000 7fb7dcff9640 1 -- 172.21.15.60:0/504300779 <== mds.1 v2:172.21.15.156:6836/707575792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556d83c7d720 con 0x556d83fee110 2024-02-26T04:59:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.930+0000 7fb7dcff9640 10 client.15879 handle_command_reply: tid=0 2024-02-26T04:59:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.930+0000 7fb7fe5dc740 1 client.15879 shutdown 2024-02-26T04:59:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.931+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/504300779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8240 msgr2=0x556d83fe23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.931+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/504300779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8240 0x556d83fe23a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4036040 tx=0x7fb7e4005000 comp rx=0 tx=0).stop 2024-02-26T04:59:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.931+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/504300779 shutdown_connections 2024-02-26T04:59:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.931+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/504300779 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x556d83fee110 0x556d83ff0500 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.931+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/504300779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 0x556d83fe9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.931+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/504300779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8c30 0x556d83fe28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.931+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/504300779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8240 0x556d83fe23a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.931+0000 7fb7dcff9640 0 client.15879 ms_handle_reset on v2:172.21.15.156:6836/707575792 2024-02-26T04:59:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.931+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/504300779 >> 172.21.15.60:0/504300779 conn(0x556d83fd37b0 msgr2=0x556d83fd2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.931+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/504300779 shutdown_connections 2024-02-26T04:59:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.931+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/504300779 wait complete. 2024-02-26T04:59:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.931+0000 7fb7fe5dc740 20 client.15879 trim_cache size 0 max 16384 2024-02-26T04:59:34.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.942+0000 7fb7fe5dc740 1 Processor -- start 2024-02-26T04:59:34.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.942+0000 7fb7fe5dc740 1 -- start start 2024-02-26T04:59:34.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7fe5dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8240 0x556d83fd22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7fe5dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fd8c30 0x556d83fe9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7fe5dc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fddda0 0x556d83fd2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7fe5dc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d83ec6100 con 0x556d83fddda0 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7fe5dc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d83fd7e00 con 0x556d83fd8c30 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7fe5dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d83fd34d0 con 0x556d83fd8240 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7f9da5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fddda0 0x556d83fd2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7f9da5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fddda0 0x556d83fd2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43492/0 (socket says 172.21.15.60:43492) 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7f9da5640 1 -- 172.21.15.60:0/2693659976 learned_addr learned my addr 172.21.15.60:0/2693659976 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7f8da3640 1 --2- 172.21.15.60:0/2693659976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fd8c30 0x556d83fe9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7f95a4640 1 --2- 172.21.15.60:0/2693659976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8240 0x556d83fd22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7f9da5640 1 -- 172.21.15.60:0/2693659976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8240 msgr2=0x556d83fd22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7f9da5640 1 --2- 172.21.15.60:0/2693659976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8240 0x556d83fd22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7f9da5640 1 -- 172.21.15.60:0/2693659976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fd8c30 msgr2=0x556d83fe9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7f9da5640 1 --2- 172.21.15.60:0/2693659976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fd8c30 0x556d83fe9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7f9da5640 1 -- 172.21.15.60:0/2693659976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d83edbbf0 con 0x556d83fddda0 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7f9da5640 1 --2- 172.21.15.60:0/2693659976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fddda0 0x556d83fd2800 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec010d00 tx=0x7fb7ec015070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce504a34b93c386 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7deffd640 1 -- 172.21.15.60:0/2693659976 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec0073b0 con 0x556d83fddda0 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7deffd640 1 -- 172.21.15.60:0/2693659976 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7ec023070 con 0x556d83fddda0 2024-02-26T04:59:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.943+0000 7fb7deffd640 1 -- 172.21.15.60:0/2693659976 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec01f680 con 0x556d83fddda0 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.944+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/2693659976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fddda0 msgr2=0x556d83fd2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.944+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/2693659976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fddda0 0x556d83fd2800 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec010d00 tx=0x7fb7ec015070 comp rx=0 tx=0).stop 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.944+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/2693659976 shutdown_connections 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.944+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/2693659976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fddda0 0x556d83fd2800 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.944+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/2693659976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fd8c30 0x556d83fe9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.944+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/2693659976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8240 0x556d83fd22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.944+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/2693659976 >> 172.21.15.60:0/2693659976 conn(0x556d83fd37b0 msgr2=0x556d83fd3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.944+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/2693659976 shutdown_connections 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.944+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/2693659976 wait complete. 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.944+0000 7fb7fe5dc740 1 -- start start 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.944+0000 7fb7fe5dc740 1 Processor -- start 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7fe5dc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8240 0x556d83fe8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7fe5dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8c30 0x556d83fe8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7fe5dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 0x556d83fe8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7fe5dc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d840446b0 con 0x556d83fd8240 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7fe5dc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d84044820 con 0x556d83fddda0 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7fe5dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d840449e0 con 0x556d83fd8c30 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7fe5dc740 10 client.0 _ll_register_callbacks cb 0x7fb7e4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7f95a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8240 0x556d83fe8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7f95a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8240 0x556d83fe8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43498/0 (socket says 172.21.15.60:43498) 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7f95a4640 1 -- 172.21.15.60:0/3746600091 learned_addr learned my addr 172.21.15.60:0/3746600091 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7f8da3640 1 --2- 172.21.15.60:0/3746600091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8c30 0x556d83fe8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7f9da5640 1 --2- 172.21.15.60:0/3746600091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 0x556d83fe8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7f95a4640 1 -- 172.21.15.60:0/3746600091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8c30 msgr2=0x556d83fe8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7f95a4640 1 --2- 172.21.15.60:0/3746600091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8c30 0x556d83fe8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7f95a4640 1 -- 172.21.15.60:0/3746600091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 msgr2=0x556d83fe8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7f95a4640 1 --2- 172.21.15.60:0/3746600091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 0x556d83fe8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7f95a4640 1 -- 172.21.15.60:0/3746600091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ec0109f0 con 0x556d83fd8240 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7f8da3640 1 --2- 172.21.15.60:0/3746600091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8c30 0x556d83fe8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:59:34.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7f95a4640 1 --2- 172.21.15.60:0/3746600091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8240 0x556d83fe8230 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009900 tx=0x7fb7e402c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee01416571336d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7dcff9640 1 -- 172.21.15.60:0/3746600091 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e403e840 con 0x556d83fd8240 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7dcff9640 1 -- 172.21.15.60:0/3746600091 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fb7e4000e40 con 0x556d83fd8240 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.945+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/3746600091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7e4004510 con 0x556d83fd8240 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.946+0000 7fb7dcff9640 1 -- 172.21.15.60:0/3746600091 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7e403e9a0 con 0x556d83fd8240 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.946+0000 7fb7fe5dc740 10 client.15885 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.946+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/3746600091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556d83f01890 con 0x556d83fd8240 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.946+0000 7fb7dcff9640 1 -- 172.21.15.60:0/3746600091 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7e4004510 con 0x556d83fd8240 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.946+0000 7fb7dcff9640 1 -- 172.21.15.60:0/3746600091 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb7e4050420 con 0x556d83fd8240 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.946+0000 7fb7fe5dc740 10 client.15885 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.946+0000 7fb7fe5dc740 10 client.15885 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.946+0000 7fb7fe5dc740 20 client.15885 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.946+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/3746600091 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x556d840460d0 0x556d840484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.946+0000 7fb7fe5dc740 4 client.15885 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.946+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/3746600091 --> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556d83ccc890 con 0x556d840460d0 2024-02-26T04:59:34.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.946+0000 7fb7f8da3640 1 --2- 172.21.15.60:0/3746600091 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x556d840460d0 0x556d840484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7f8da3640 1 --2- 172.21.15.60:0/3746600091 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x556d840460d0 0x556d840484f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0a26263274bdbfd server_cookie=e03f9811dd36a1b7 in_seq=0 out_seq=0 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7dcff9640 10 client.15885 ms_handle_connect on v2:172.21.15.156:6836/707575792 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7dcff9640 1 -- 172.21.15.60:0/3746600091 <== mds.1 v2:172.21.15.156:6836/707575792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556d83ccc890 con 0x556d840460d0 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7dcff9640 10 client.15885 handle_command_reply: tid=0 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7fe5dc740 1 client.15885 shutdown 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/3746600091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8240 msgr2=0x556d83fe8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/3746600091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8240 0x556d83fe8230 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009900 tx=0x7fb7e402c320 comp rx=0 tx=0).stop 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/3746600091 shutdown_connections 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/3746600091 >> [v2:172.21.15.156:6836/707575792,v1:172.21.15.156:6837/707575792] conn(0x556d840460d0 0x556d840484f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/3746600091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556d83fddda0 0x556d83fe8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/3746600091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556d83fd8c30 0x556d83fe8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7dcff9640 0 client.15885 ms_handle_reset on v2:172.21.15.156:6836/707575792 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7fe5dc740 1 --2- 172.21.15.60:0/3746600091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556d83fd8240 0x556d83fe8230 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/3746600091 >> 172.21.15.60:0/3746600091 conn(0x556d83fd37b0 msgr2=0x556d83fd22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/3746600091 shutdown_connections 2024-02-26T04:59:34.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7fe5dc740 1 -- 172.21.15.60:0/3746600091 wait complete. 2024-02-26T04:59:34.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.947+0000 7fb7fe5dc740 20 client.15885 trim_cache size 0 max 16384 2024-02-26T04:59:34.956 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T04:59:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.948+0000 7fb7fb808640 1 -- 172.21.15.60:0/1549306700 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb7e00a20c0 msgr2=0x7fb7e00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.948+0000 7fb7fb808640 1 --2- 172.21.15.60:0/1549306700 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb7e00a20c0 0x7fb7e00a4580 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x556d83fd9490 tx=0x7fb7ec004a40 comp rx=0 tx=0).stop 2024-02-26T04:59:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.948+0000 7fb7fb808640 1 -- 172.21.15.60:0/1549306700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f4101280 msgr2=0x7fb7f4199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.948+0000 7fb7fb808640 1 --2- 172.21.15.60:0/1549306700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f4101280 0x7fb7f4199a40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000b4a0 tx=0x7fb7f000b960 comp rx=0 tx=0).stop 2024-02-26T04:59:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.948+0000 7fb7fb808640 1 -- 172.21.15.60:0/1549306700 shutdown_connections 2024-02-26T04:59:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.948+0000 7fb7fb808640 1 --2- 172.21.15.60:0/1549306700 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fb7e00a20c0 0x7fb7e00a4580 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.948+0000 7fb7fb808640 1 --2- 172.21.15.60:0/1549306700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7f4101c40 0x7fb7f41a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.948+0000 7fb7fb808640 1 --2- 172.21.15.60:0/1549306700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7f4101280 0x7fb7f4199a40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.948+0000 7fb7fb808640 1 --2- 172.21.15.60:0/1549306700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7f4100090 0x7fb7f4199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.948+0000 7fb7fb808640 1 -- 172.21.15.60:0/1549306700 >> 172.21.15.60:0/1549306700 conn(0x7fb7f40fb620 msgr2=0x7fb7f40fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.948+0000 7fb7fb808640 1 -- 172.21.15.60:0/1549306700 shutdown_connections 2024-02-26T04:59:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:34.948+0000 7fb7fb808640 1 -- 172.21.15.60:0/1549306700 wait complete. 2024-02-26T04:59:34.961 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-26T04:59:34.961 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-26T04:59:35.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.063+0000 7f8b362de640 1 -- 172.21.15.60:0/1892483146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b30101290 msgr2=0x7f8b30101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:35.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.063+0000 7f8b362de640 1 --2- 172.21.15.60:0/1892483146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b30101290 0x7f8b30101710 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009900 tx=0x7f8b2402bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:35.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.063+0000 7f8b362de640 1 -- 172.21.15.60:0/1892483146 shutdown_connections 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.064+0000 7f8b362de640 1 --2- 172.21.15.60:0/1892483146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b30101c50 0x7f8b301084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.064+0000 7f8b362de640 1 --2- 172.21.15.60:0/1892483146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b30101290 0x7f8b30101710 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.064+0000 7f8b362de640 1 --2- 172.21.15.60:0/1892483146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b301000a0 0x7f8b301004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.064+0000 7f8b362de640 1 -- 172.21.15.60:0/1892483146 >> 172.21.15.60:0/1892483146 conn(0x7f8b300fb630 msgr2=0x7f8b300fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.064+0000 7f8b362de640 1 -- 172.21.15.60:0/1892483146 shutdown_connections 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.064+0000 7f8b362de640 1 -- 172.21.15.60:0/1892483146 wait complete. 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.064+0000 7f8b362de640 1 Processor -- start 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.064+0000 7f8b362de640 1 -- start start 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.064+0000 7f8b362de640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b301000a0 0x7f8b301960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.064+0000 7f8b362de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b30101290 0x7f8b30196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.064+0000 7f8b362de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b30101c50 0x7f8b3019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.064+0000 7f8b362de640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3019dbd0 con 0x7f8b30101290 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.064+0000 7f8b362de640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3019dd40 con 0x7f8b301000a0 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.064+0000 7f8b362de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3019df00 con 0x7f8b30101c50 2024-02-26T04:59:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b2f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b30101290 0x7f8b30196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b2f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b30101290 0x7f8b30196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43524/0 (socket says 172.21.15.60:43524) 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b2f7fe640 1 -- 172.21.15.60:0/3802741424 learned_addr learned my addr 172.21.15.60:0/3802741424 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b2ffff640 1 --2- 172.21.15.60:0/3802741424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b301000a0 0x7f8b301960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b3487b640 1 --2- 172.21.15.60:0/3802741424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b30101c50 0x7f8b3019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b2f7fe640 1 -- 172.21.15.60:0/3802741424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b30101c50 msgr2=0x7f8b3019d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b2f7fe640 1 --2- 172.21.15.60:0/3802741424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b30101c50 0x7f8b3019d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b2f7fe640 1 -- 172.21.15.60:0/3802741424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b301000a0 msgr2=0x7f8b301960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b2f7fe640 1 --2- 172.21.15.60:0/3802741424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b301000a0 0x7f8b301960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b2f7fe640 1 -- 172.21.15.60:0/3802741424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b24009560 con 0x7f8b30101290 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b2f7fe640 1 --2- 172.21.15.60:0/3802741424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b30101290 0x7f8b30196620 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8b24005ab0 tx=0x7f8b240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b2d7fa640 1 -- 172.21.15.60:0/3802741424 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b240044e0 con 0x7f8b30101290 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b2d7fa640 1 -- 172.21.15.60:0/3802741424 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b240054e0 con 0x7f8b30101290 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b362de640 1 -- 172.21.15.60:0/3802741424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3019e180 con 0x7f8b30101290 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b2d7fa640 1 -- 172.21.15.60:0/3802741424 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2403e450 con 0x7f8b30101290 2024-02-26T04:59:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.065+0000 7f8b362de640 1 -- 172.21.15.60:0/3802741424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3019e690 con 0x7f8b30101290 2024-02-26T04:59:35.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.068+0000 7f8b2d7fa640 1 -- 172.21.15.60:0/3802741424 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8b240035f0 con 0x7f8b30101290 2024-02-26T04:59:35.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.068+0000 7f8b391a9740 1 Processor -- start 2024-02-26T04:59:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.068+0000 7f8b391a9740 1 -- start start 2024-02-26T04:59:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.068+0000 7f8b391a9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f86450 0x5584f3f86870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.068+0000 7f8b391a9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f8b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.068+0000 7f8b391a9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f8bfb0 0x5584f3f8e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.068+0000 7f8b391a9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584f3f40930 con 0x5584f3f86450 2024-02-26T04:59:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.068+0000 7f8b391a9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584f3ef42e0 con 0x5584f3f8bfb0 2024-02-26T04:59:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.068+0000 7f8b391a9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584f3ef1810 con 0x5584f3f86e40 2024-02-26T04:59:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.068+0000 7f8b3487b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f86450 0x5584f3f86870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.068+0000 7f8b3487b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f86450 0x5584f3f86870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43534/0 (socket says 172.21.15.60:43534) 2024-02-26T04:59:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.068+0000 7f8b3487b640 1 -- 172.21.15.60:0/1439021936 learned_addr learned my addr 172.21.15.60:0/1439021936 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b2d7fa640 1 --2- 172.21.15.60:0/3802741424 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8b180a20c0 0x7f8b180a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b2ffff640 1 --2- 172.21.15.60:0/1439021936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f8b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b2f7fe640 1 --2- 172.21.15.60:0/1439021936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f8bfb0 0x5584f3f8e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b3487b640 1 --2- 172.21.15.60:0/3802741424 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8b180a20c0 0x7f8b180a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b2d7fa640 1 -- 172.21.15.60:0/3802741424 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8b240f0730 con 0x7f8b30101290 2024-02-26T04:59:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b3487b640 1 -- 172.21.15.60:0/1439021936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 msgr2=0x5584f3f8b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b3487b640 1 --2- 172.21.15.60:0/1439021936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f8b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b3487b640 1 -- 172.21.15.60:0/1439021936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f8bfb0 msgr2=0x5584f3f8e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b3487b640 1 --2- 172.21.15.60:0/1439021936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f8bfb0 0x5584f3f8e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b3487b640 1 -- 172.21.15.60:0/1439021936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584f3eaf030 con 0x5584f3f86450 2024-02-26T04:59:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b2f7fe640 1 --2- 172.21.15.60:0/1439021936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f8bfb0 0x5584f3f8e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:59:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b3487b640 1 --2- 172.21.15.60:0/1439021936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f86450 0x5584f3f86870 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8b28012890 tx=0x7f8b28012d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd13c0a6c34efb01 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b167fc640 1 -- 172.21.15.60:0/1439021936 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28020070 con 0x5584f3f86450 2024-02-26T04:59:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b167fc640 1 -- 172.21.15.60:0/1439021936 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b28018070 con 0x5584f3f86450 2024-02-26T04:59:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b167fc640 1 -- 172.21.15.60:0/1439021936 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2801c650 con 0x5584f3f86450 2024-02-26T04:59:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.069+0000 7f8b3487b640 1 --2- 172.21.15.60:0/3802741424 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8b180a20c0 0x7f8b180a4580 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8b2801f000 tx=0x7f8b2801e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.070+0000 7f8b391a9740 1 -- 172.21.15.60:0/1439021936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f86450 msgr2=0x5584f3f86870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.070+0000 7f8b391a9740 1 --2- 172.21.15.60:0/1439021936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f86450 0x5584f3f86870 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8b28012890 tx=0x7f8b28012d50 comp rx=0 tx=0).stop 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.070+0000 7f8b391a9740 1 -- 172.21.15.60:0/1439021936 shutdown_connections 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.070+0000 7f8b391a9740 1 --2- 172.21.15.60:0/1439021936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f8bfb0 0x5584f3f8e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.070+0000 7f8b391a9740 1 --2- 172.21.15.60:0/1439021936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f8b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.070+0000 7f8b391a9740 1 --2- 172.21.15.60:0/1439021936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f86450 0x5584f3f86870 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.070+0000 7f8b391a9740 1 -- 172.21.15.60:0/1439021936 >> 172.21.15.60:0/1439021936 conn(0x5584f3f819c0 msgr2=0x5584f3f81dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.070+0000 7f8b391a9740 1 -- 172.21.15.60:0/1439021936 shutdown_connections 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.070+0000 7f8b391a9740 1 -- 172.21.15.60:0/1439021936 wait complete. 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.070+0000 7f8b391a9740 1 -- start start 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b391a9740 1 Processor -- start 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b391a9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f86450 0x5584f3f90690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b391a9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f90bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b391a9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f8bfb0 0x5584f3f97c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b391a9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584f3eecff0 con 0x5584f3f86450 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b391a9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584f3bc0430 con 0x5584f3f8bfb0 2024-02-26T04:59:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b391a9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584f3b7ce00 con 0x5584f3f86e40 2024-02-26T04:59:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b391a9740 10 client.0 _ll_register_callbacks cb 0x5584f3e94c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b2ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f86450 0x5584f3f90690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b2ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f86450 0x5584f3f90690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43544/0 (socket says 172.21.15.60:43544) 2024-02-26T04:59:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b2ffff640 1 -- 172.21.15.60:0/660916506 learned_addr learned my addr 172.21.15.60:0/660916506 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b3487b640 1 --2- 172.21.15.60:0/660916506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f8bfb0 0x5584f3f97c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b2f7fe640 1 --2- 172.21.15.60:0/660916506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f90bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b2ffff640 1 -- 172.21.15.60:0/660916506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 msgr2=0x5584f3f90bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b2ffff640 1 --2- 172.21.15.60:0/660916506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f90bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b2ffff640 1 -- 172.21.15.60:0/660916506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f8bfb0 msgr2=0x5584f3f97c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b2ffff640 1 --2- 172.21.15.60:0/660916506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f8bfb0 0x5584f3f97c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b2ffff640 1 -- 172.21.15.60:0/660916506 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b280124d0 con 0x5584f3f86450 2024-02-26T04:59:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b2ffff640 1 --2- 172.21.15.60:0/660916506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f86450 0x5584f3f90690 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8b20011760 tx=0x7f8b20011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6d8e62db893913 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b0f7fe640 1 -- 172.21.15.60:0/660916506 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2000fe10 con 0x5584f3f86450 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b0f7fe640 1 -- 172.21.15.60:0/660916506 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8b20005020 con 0x5584f3f86450 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.071+0000 7f8b391a9740 1 -- 172.21.15.60:0/660916506 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584f3bec7e0 con 0x5584f3f86450 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.072+0000 7f8b0f7fe640 1 -- 172.21.15.60:0/660916506 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b200051e0 con 0x5584f3f86450 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.072+0000 7f8b391a9740 10 client.15897 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.072+0000 7f8b391a9740 1 -- 172.21.15.60:0/660916506 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5584f3eeb670 con 0x5584f3f86450 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.072+0000 7f8b0f7fe640 1 -- 172.21.15.60:0/660916506 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5584f3bec7e0 con 0x5584f3f86450 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.072+0000 7f8b0f7fe640 1 -- 172.21.15.60:0/660916506 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8b200193f0 con 0x5584f3f86450 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.072+0000 7f8b391a9740 10 client.15897 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.072+0000 7f8b391a9740 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.072+0000 7f8b391a9740 20 client.15897 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.072+0000 7f8b391a9740 1 --2- 172.21.15.60:0/660916506 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5584f3f9c900 0x5584f3f9ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.072+0000 7f8b391a9740 4 client.15897 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.072+0000 7f8b391a9740 1 -- 172.21.15.60:0/660916506 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584f3c29720 con 0x5584f3f9c900 2024-02-26T04:59:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.072+0000 7f8b2f7fe640 1 --2- 172.21.15.60:0/660916506 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5584f3f9c900 0x5584f3f9ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.073+0000 7f8b0f7fe640 10 client.15897 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.073+0000 7f8b2f7fe640 1 --2- 172.21.15.60:0/660916506 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5584f3f9c900 0x5584f3f9ed20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb4f24b2a4ab0211 server_cookie=25d37e4828c202e9 in_seq=0 out_seq=0 2024-02-26T04:59:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.073+0000 7f8b0f7fe640 1 -- 172.21.15.60:0/660916506 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5584f3c29720 con 0x5584f3f9c900 2024-02-26T04:59:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.073+0000 7f8b0f7fe640 10 client.15897 handle_command_reply: tid=0 2024-02-26T04:59:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.073+0000 7f8b391a9740 1 client.15897 shutdown 2024-02-26T04:59:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.074+0000 7f8b391a9740 1 -- 172.21.15.60:0/660916506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f86450 msgr2=0x5584f3f90690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.074+0000 7f8b391a9740 1 --2- 172.21.15.60:0/660916506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f86450 0x5584f3f90690 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8b20011760 tx=0x7f8b20011c20 comp rx=0 tx=0).stop 2024-02-26T04:59:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.074+0000 7f8b391a9740 1 -- 172.21.15.60:0/660916506 shutdown_connections 2024-02-26T04:59:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.074+0000 7f8b391a9740 1 --2- 172.21.15.60:0/660916506 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5584f3f9c900 0x5584f3f9ed20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.074+0000 7f8b391a9740 1 --2- 172.21.15.60:0/660916506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f8bfb0 0x5584f3f97c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.074+0000 7f8b391a9740 1 --2- 172.21.15.60:0/660916506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f90bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.074+0000 7f8b0f7fe640 0 client.15897 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.074+0000 7f8b391a9740 1 --2- 172.21.15.60:0/660916506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f86450 0x5584f3f90690 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.074+0000 7f8b391a9740 1 -- 172.21.15.60:0/660916506 >> 172.21.15.60:0/660916506 conn(0x5584f3f819c0 msgr2=0x5584f3f82c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:35.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.074+0000 7f8b391a9740 1 -- 172.21.15.60:0/660916506 shutdown_connections 2024-02-26T04:59:35.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.074+0000 7f8b391a9740 1 -- 172.21.15.60:0/660916506 wait complete. 2024-02-26T04:59:35.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.074+0000 7f8b391a9740 20 client.15897 trim_cache size 0 max 16384 2024-02-26T04:59:35.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.085+0000 7f8b391a9740 1 Processor -- start 2024-02-26T04:59:35.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.085+0000 7f8b391a9740 1 -- start start 2024-02-26T04:59:35.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b391a9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f86450 0x5584f3f7e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b391a9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f97bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b391a9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f8bfb0 0x5584f3f7e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b391a9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584f3e72100 con 0x5584f3f8bfb0 2024-02-26T04:59:35.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b391a9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584f3c29720 con 0x5584f3f86450 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b391a9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584f3f71a10 con 0x5584f3f86e40 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b3487b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f8bfb0 0x5584f3f7e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b3487b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f8bfb0 0x5584f3f7e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43558/0 (socket says 172.21.15.60:43558) 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b3487b640 1 -- 172.21.15.60:0/1464839404 learned_addr learned my addr 172.21.15.60:0/1464839404 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b2ffff640 1 --2- 172.21.15.60:0/1464839404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f86450 0x5584f3f7e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b2f7fe640 1 --2- 172.21.15.60:0/1464839404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f97bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b3487b640 1 -- 172.21.15.60:0/1464839404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 msgr2=0x5584f3f97bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b3487b640 1 --2- 172.21.15.60:0/1464839404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f97bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b3487b640 1 -- 172.21.15.60:0/1464839404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f86450 msgr2=0x5584f3f7e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b3487b640 1 --2- 172.21.15.60:0/1464839404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f86450 0x5584f3f7e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b3487b640 1 -- 172.21.15.60:0/1464839404 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584f3e87bf0 con 0x5584f3f8bfb0 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b2ffff640 1 --2- 172.21.15.60:0/1464839404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f86450 0x5584f3f7e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b3487b640 1 --2- 172.21.15.60:0/1464839404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f8bfb0 0x5584f3f7e800 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8b28010ae0 tx=0x7f8b28005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f64b6323c827f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b15ffb640 1 -- 172.21.15.60:0/1464839404 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28020040 con 0x5584f3f8bfb0 2024-02-26T04:59:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b15ffb640 1 -- 172.21.15.60:0/1464839404 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b2802c040 con 0x5584f3f8bfb0 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.086+0000 7f8b15ffb640 1 -- 172.21.15.60:0/1464839404 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b280256d0 con 0x5584f3f8bfb0 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.087+0000 7f8b391a9740 1 -- 172.21.15.60:0/1464839404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f8bfb0 msgr2=0x5584f3f7e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.087+0000 7f8b391a9740 1 --2- 172.21.15.60:0/1464839404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f8bfb0 0x5584f3f7e800 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8b28010ae0 tx=0x7f8b28005b70 comp rx=0 tx=0).stop 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.087+0000 7f8b391a9740 1 -- 172.21.15.60:0/1464839404 shutdown_connections 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.087+0000 7f8b391a9740 1 --2- 172.21.15.60:0/1464839404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f8bfb0 0x5584f3f7e800 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.087+0000 7f8b391a9740 1 --2- 172.21.15.60:0/1464839404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f97bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.087+0000 7f8b391a9740 1 --2- 172.21.15.60:0/1464839404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f86450 0x5584f3f7e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.087+0000 7f8b391a9740 1 -- 172.21.15.60:0/1464839404 >> 172.21.15.60:0/1464839404 conn(0x5584f3f819c0 msgr2=0x5584f3f88360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.087+0000 7f8b391a9740 1 -- 172.21.15.60:0/1464839404 shutdown_connections 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.087+0000 7f8b391a9740 1 -- 172.21.15.60:0/1464839404 wait complete. 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.087+0000 7f8b391a9740 1 -- start start 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b391a9740 1 Processor -- start 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b391a9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f86450 0x5584f3f80d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b391a9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f97a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b391a9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f8bfb0 0x5584f3f7f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b391a9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584f3ff0e20 con 0x5584f3f8bfb0 2024-02-26T04:59:35.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b391a9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584f3ff0f90 con 0x5584f3f86450 2024-02-26T04:59:35.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b391a9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584f3f7f8c0 con 0x5584f3f86e40 2024-02-26T04:59:35.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b3487b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f8bfb0 0x5584f3f7f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b391a9740 10 client.0 _ll_register_callbacks cb 0x7f8b28008610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T04:59:35.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b3487b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f8bfb0 0x5584f3f7f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43560/0 (socket says 172.21.15.60:43560) 2024-02-26T04:59:35.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b3487b640 1 -- 172.21.15.60:0/622273141 learned_addr learned my addr 172.21.15.60:0/622273141 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:35.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b2f7fe640 1 --2- 172.21.15.60:0/622273141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f97a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b2ffff640 1 --2- 172.21.15.60:0/622273141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f86450 0x5584f3f80d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b3487b640 1 -- 172.21.15.60:0/622273141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 msgr2=0x5584f3f97a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:35.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b3487b640 1 --2- 172.21.15.60:0/622273141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f97a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b3487b640 1 -- 172.21.15.60:0/622273141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f86450 msgr2=0x5584f3f80d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T04:59:35.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b3487b640 1 --2- 172.21.15.60:0/622273141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f86450 0x5584f3f80d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.088+0000 7f8b3487b640 1 -- 172.21.15.60:0/622273141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584f3eaf030 con 0x5584f3f8bfb0 2024-02-26T04:59:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b3487b640 1 --2- 172.21.15.60:0/622273141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f8bfb0 0x5584f3f7f380 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8b28010740 tx=0x7f8b2801cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b93904adbbdbb6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-02-26T04:59:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b0f7fe640 1 -- 172.21.15.60:0/622273141 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28020070 con 0x5584f3f8bfb0 2024-02-26T04:59:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b0f7fe640 1 -- 172.21.15.60:0/622273141 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f8b28025670 con 0x5584f3f8bfb0 2024-02-26T04:59:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b391a9740 1 -- 172.21.15.60:0/622273141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b28018070 con 0x5584f3f8bfb0 2024-02-26T04:59:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b0f7fe640 1 -- 172.21.15.60:0/622273141 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b28008a30 con 0x5584f3f8bfb0 2024-02-26T04:59:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b391a9740 10 client.15903 fetch_fsmap learned FSMap version 23 2024-02-26T04:59:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b391a9740 1 -- 172.21.15.60:0/622273141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5584f3ead890 con 0x5584f3f8bfb0 2024-02-26T04:59:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b0f7fe640 1 -- 172.21.15.60:0/622273141 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b28018070 con 0x5584f3f8bfb0 2024-02-26T04:59:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b0f7fe640 1 -- 172.21.15.60:0/622273141 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8b2801b070 con 0x5584f3f8bfb0 2024-02-26T04:59:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b391a9740 10 client.15903 fetch_fsmap finished waiting for FSMap version 23 2024-02-26T04:59:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b391a9740 10 client.15903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-26T04:59:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b391a9740 20 client.15903 populate_metadata read hostname 'smithi060' 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b391a9740 1 --2- 172.21.15.60:0/622273141 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5584f3ffa590 0x5584f3ffc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b391a9740 4 client.15903 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.089+0000 7f8b391a9740 1 -- 172.21.15.60:0/622273141 --> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5584f3c78890 con 0x5584f3ffa590 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b2ffff640 1 --2- 172.21.15.60:0/622273141 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5584f3ffa590 0x5584f3ffc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b2ffff640 1 --2- 172.21.15.60:0/622273141 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5584f3ffa590 0x5584f3ffc9b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e44696981bb4c0 server_cookie=ebe2fe03579cfb9c in_seq=0 out_seq=0 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b0f7fe640 10 client.15903 ms_handle_connect on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b0f7fe640 1 -- 172.21.15.60:0/622273141 <== mds.0 v2:172.21.15.156:6838/1768629491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5584f3c78890 con 0x5584f3ffa590 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b0f7fe640 10 client.15903 handle_command_reply: tid=0 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b391a9740 1 client.15903 shutdown 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b391a9740 1 -- 172.21.15.60:0/622273141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f8bfb0 msgr2=0x5584f3f7f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b391a9740 1 --2- 172.21.15.60:0/622273141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f8bfb0 0x5584f3f7f380 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8b28010740 tx=0x7f8b2801cf80 comp rx=0 tx=0).stop 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b391a9740 1 -- 172.21.15.60:0/622273141 shutdown_connections 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b391a9740 1 --2- 172.21.15.60:0/622273141 >> [v2:172.21.15.156:6838/1768629491,v1:172.21.15.156:6839/1768629491] conn(0x5584f3ffa590 0x5584f3ffc9b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b391a9740 1 --2- 172.21.15.60:0/622273141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584f3f8bfb0 0x5584f3f7f380 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b391a9740 1 --2- 172.21.15.60:0/622273141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584f3f86e40 0x5584f3f97a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b0f7fe640 0 client.15903 ms_handle_reset on v2:172.21.15.156:6838/1768629491 2024-02-26T04:59:35.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b391a9740 1 --2- 172.21.15.60:0/622273141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584f3f86450 0x5584f3f80d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b391a9740 1 -- 172.21.15.60:0/622273141 >> 172.21.15.60:0/622273141 conn(0x5584f3f819c0 msgr2=0x5584f3f7e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:35.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b391a9740 1 -- 172.21.15.60:0/622273141 shutdown_connections 2024-02-26T04:59:35.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.090+0000 7f8b391a9740 1 -- 172.21.15.60:0/622273141 wait complete. 2024-02-26T04:59:35.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.091+0000 7f8b391a9740 20 client.15903 trim_cache size 0 max 16384 2024-02-26T04:59:35.099 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-02-26T04:59:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.091+0000 7f8b362de640 1 -- 172.21.15.60:0/3802741424 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8b180a20c0 msgr2=0x7f8b180a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.091+0000 7f8b362de640 1 --2- 172.21.15.60:0/3802741424 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8b180a20c0 0x7f8b180a4580 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8b2801f000 tx=0x7f8b2801e000 comp rx=0 tx=0).stop 2024-02-26T04:59:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.091+0000 7f8b362de640 1 -- 172.21.15.60:0/3802741424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b30101290 msgr2=0x7f8b30196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.091+0000 7f8b362de640 1 --2- 172.21.15.60:0/3802741424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b30101290 0x7f8b30196620 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8b24005ab0 tx=0x7f8b240042f0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.091+0000 7f8b362de640 1 -- 172.21.15.60:0/3802741424 shutdown_connections 2024-02-26T04:59:35.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.091+0000 7f8b362de640 1 --2- 172.21.15.60:0/3802741424 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f8b180a20c0 0x7f8b180a4580 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.091+0000 7f8b362de640 1 --2- 172.21.15.60:0/3802741424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b30101c50 0x7f8b3019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.091+0000 7f8b362de640 1 --2- 172.21.15.60:0/3802741424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b30101290 0x7f8b30196620 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.091+0000 7f8b362de640 1 --2- 172.21.15.60:0/3802741424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b301000a0 0x7f8b301960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:35.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.091+0000 7f8b362de640 1 -- 172.21.15.60:0/3802741424 >> 172.21.15.60:0/3802741424 conn(0x7f8b300fb630 msgr2=0x7f8b30104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:35.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.091+0000 7f8b362de640 1 -- 172.21.15.60:0/3802741424 shutdown_connections 2024-02-26T04:59:35.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:35.091+0000 7f8b362de640 1 -- 172.21.15.60:0/3802741424 wait complete. 2024-02-26T04:59:35.229 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:35 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3393220573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:59:35.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:35 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3393220573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:59:35.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:35 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3393220573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T04:59:36.105 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-02-26T04:59:36.106 INFO:tasks.fwd_scrub:done joining 2024-02-26T04:59:36.106 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-02-26T04:59:36.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T04:59:36.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.234+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3191003351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3dc102430 msgr2=0x7fe3dc1028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.234+0000 7fe3e30b7640 1 --2- 172.21.15.60:0/3191003351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3dc102430 0x7fe3dc1028b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009900 tx=0x7fe3c402bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.234+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3191003351 shutdown_connections 2024-02-26T04:59:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.234+0000 7fe3e30b7640 1 --2- 172.21.15.60:0/3191003351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3dc102df0 0x7fe3dc109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.234+0000 7fe3e30b7640 1 --2- 172.21.15.60:0/3191003351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3dc102430 0x7fe3dc1028b0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.234+0000 7fe3e30b7640 1 --2- 172.21.15.60:0/3191003351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc0ff460 0x7fe3dc0ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.234+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3191003351 >> 172.21.15.60:0/3191003351 conn(0x7fe3dc0fadb0 msgr2=0x7fe3dc0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.234+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3191003351 shutdown_connections 2024-02-26T04:59:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.234+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3191003351 wait complete. 2024-02-26T04:59:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e30b7640 1 Processor -- start 2024-02-26T04:59:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e30b7640 1 -- start start 2024-02-26T04:59:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e30b7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc0ff460 0x7fe3dc197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e30b7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3dc102430 0x7fe3dc1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e30b7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3dc102df0 0x7fe3dc19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e30b7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc19ed50 con 0x7fe3dc102df0 2024-02-26T04:59:36.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e30b7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc19eec0 con 0x7fe3dc0ff460 2024-02-26T04:59:36.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e30b7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc19f080 con 0x7fe3dc102430 2024-02-26T04:59:36.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e1654640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3dc102df0 0x7fe3dc19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:36.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e1654640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3dc102df0 0x7fe3dc19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43580/0 (socket says 172.21.15.60:43580) 2024-02-26T04:59:36.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e1654640 1 -- 172.21.15.60:0/3916378541 learned_addr learned my addr 172.21.15.60:0/3916378541 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:36.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3dbfff640 1 --2- 172.21.15.60:0/3916378541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3dc102430 0x7fe3dc1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:36.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e0e53640 1 --2- 172.21.15.60:0/3916378541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc0ff460 0x7fe3dc197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:36.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e1654640 1 -- 172.21.15.60:0/3916378541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3dc102430 msgr2=0x7fe3dc1977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:36.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e1654640 1 --2- 172.21.15.60:0/3916378541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3dc102430 0x7fe3dc1977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e1654640 1 -- 172.21.15.60:0/3916378541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc0ff460 msgr2=0x7fe3dc197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:36.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e1654640 1 --2- 172.21.15.60:0/3916378541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc0ff460 0x7fe3dc197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e1654640 1 -- 172.21.15.60:0/3916378541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c4009560 con 0x7fe3dc102df0 2024-02-26T04:59:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.235+0000 7fe3e1654640 1 --2- 172.21.15.60:0/3916378541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3dc102df0 0x7fe3dc19e810 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009fd0 tx=0x7fe3d400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.236+0000 7fe3d9ffb640 1 -- 172.21.15.60:0/3916378541 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d400bea0 con 0x7fe3dc102df0 2024-02-26T04:59:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.236+0000 7fe3d9ffb640 1 -- 172.21.15.60:0/3916378541 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3d4009940 con 0x7fe3dc102df0 2024-02-26T04:59:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.236+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3916378541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc19f360 con 0x7fe3dc102df0 2024-02-26T04:59:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.236+0000 7fe3d9ffb640 1 -- 172.21.15.60:0/3916378541 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d40106c0 con 0x7fe3dc102df0 2024-02-26T04:59:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.236+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3916378541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc19f840 con 0x7fe3dc102df0 2024-02-26T04:59:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.238+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3916378541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3dc108630 con 0x7fe3dc102df0 2024-02-26T04:59:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.238+0000 7fe3d9ffb640 1 -- 172.21.15.60:0/3916378541 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe3d4004240 con 0x7fe3dc102df0 2024-02-26T04:59:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.239+0000 7fe3d9ffb640 1 --2- 172.21.15.60:0/3916378541 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe3c80a2140 0x7fe3c80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.239+0000 7fe3e0e53640 1 --2- 172.21.15.60:0/3916378541 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe3c80a2140 0x7fe3c80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.239+0000 7fe3d9ffb640 1 -- 172.21.15.60:0/3916378541 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fe3d40cdce0 con 0x7fe3dc102df0 2024-02-26T04:59:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.239+0000 7fe3e0e53640 1 --2- 172.21.15.60:0/3916378541 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe3c80a2140 0x7fe3c80a4600 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc069650 tx=0x7fe3cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:36.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.243+0000 7fe3d9ffb640 1 -- 172.21.15.60:0/3916378541 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe3d4096920 con 0x7fe3dc102df0 2024-02-26T04:59:36.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.425+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3916378541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3dc1080a0 con 0x7fe3dc102df0 2024-02-26T04:59:36.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.426+0000 7fe3d9ffb640 1 -- 172.21.15.60:0/3916378541 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24972 (secure 0 0 0) 0x7fe3d4099fe0 con 0x7fe3dc102df0 2024-02-26T04:59:36.430 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:59:36.430 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:57:21.168776+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T04:59:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.428+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3916378541 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe3c80a2140 msgr2=0x7fe3c80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.428+0000 7fe3e30b7640 1 --2- 172.21.15.60:0/3916378541 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe3c80a2140 0x7fe3c80a4600 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc069650 tx=0x7fe3cc005f90 comp rx=0 tx=0).stop 2024-02-26T04:59:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.429+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3916378541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3dc102df0 msgr2=0x7fe3dc19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.429+0000 7fe3e30b7640 1 --2- 172.21.15.60:0/3916378541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3dc102df0 0x7fe3dc19e810 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009fd0 tx=0x7fe3d400b260 comp rx=0 tx=0).stop 2024-02-26T04:59:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.429+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3916378541 shutdown_connections 2024-02-26T04:59:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.429+0000 7fe3e30b7640 1 --2- 172.21.15.60:0/3916378541 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe3c80a2140 0x7fe3c80a4600 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.429+0000 7fe3e30b7640 1 --2- 172.21.15.60:0/3916378541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3dc102df0 0x7fe3dc19e810 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.429+0000 7fe3e30b7640 1 --2- 172.21.15.60:0/3916378541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3dc102430 0x7fe3dc1977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.429+0000 7fe3e30b7640 1 --2- 172.21.15.60:0/3916378541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3dc0ff460 0x7fe3dc197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.429+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3916378541 >> 172.21.15.60:0/3916378541 conn(0x7fe3dc0fadb0 msgr2=0x7fe3dc105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.429+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3916378541 shutdown_connections 2024-02-26T04:59:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.429+0000 7fe3e30b7640 1 -- 172.21.15.60:0/3916378541 wait complete. 2024-02-26T04:59:36.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:36 smithi060 ceph-mon[32382]: pgmap v421: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 92 KiB/s wr, 34 op/s 2024-02-26T04:59:36.443 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T04:50:44.572921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T04:55:25.530720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T04:55:26.741604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T04:55:30.453569+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T04:59:36.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T04:59:36.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:36 smithi191 ceph-mon[36352]: pgmap v421: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 92 KiB/s wr, 34 op/s 2024-02-26T04:59:36.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.546+0000 7f666ea36640 1 -- 172.21.15.60:0/3049727168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6668101270 msgr2=0x7f66681016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:36.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.546+0000 7f666ea36640 1 --2- 172.21.15.60:0/3049727168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6668101270 0x7f66681016f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f665c009900 tx=0x7f665c02bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.547+0000 7f666ea36640 1 -- 172.21.15.60:0/3049727168 shutdown_connections 2024-02-26T04:59:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.547+0000 7f666ea36640 1 --2- 172.21.15.60:0/3049727168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6668101c30 0x7f66681084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.547+0000 7f666ea36640 1 --2- 172.21.15.60:0/3049727168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6668101270 0x7f66681016f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.547+0000 7f666ea36640 1 --2- 172.21.15.60:0/3049727168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6668100080 0x7f66681004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.547+0000 7f666ea36640 1 -- 172.21.15.60:0/3049727168 >> 172.21.15.60:0/3049727168 conn(0x7f66680fb610 msgr2=0x7f66680fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.547+0000 7f666ea36640 1 -- 172.21.15.60:0/3049727168 shutdown_connections 2024-02-26T04:59:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.547+0000 7f666ea36640 1 -- 172.21.15.60:0/3049727168 wait complete. 2024-02-26T04:59:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.547+0000 7f666ea36640 1 Processor -- start 2024-02-26T04:59:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.547+0000 7f666ea36640 1 -- start start 2024-02-26T04:59:36.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.547+0000 7f666ea36640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6668100080 0x7f6668104010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:36.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f666ea36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6668101270 0x7f6668102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:36.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f666ea36640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6668101c30 0x7f6668102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:36.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f666ea36640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66681032f0 con 0x7f6668101270 2024-02-26T04:59:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f666ea36640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6668103460 con 0x7f6668100080 2024-02-26T04:59:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f666ea36640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66681a12c0 con 0x7f6668101c30 2024-02-26T04:59:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f66677fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6668101270 0x7f6668102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f66677fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6668101270 0x7f6668102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43604/0 (socket says 172.21.15.60:43604) 2024-02-26T04:59:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f66677fe640 1 -- 172.21.15.60:0/202612877 learned_addr learned my addr 172.21.15.60:0/202612877 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f6667fff640 1 --2- 172.21.15.60:0/202612877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6668100080 0x7f6668104010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f666cfd3640 1 --2- 172.21.15.60:0/202612877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6668101c30 0x7f6668102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f66677fe640 1 -- 172.21.15.60:0/202612877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6668101c30 msgr2=0x7f6668102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f66677fe640 1 --2- 172.21.15.60:0/202612877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6668101c30 0x7f6668102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f66677fe640 1 -- 172.21.15.60:0/202612877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6668100080 msgr2=0x7f6668104010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f66677fe640 1 --2- 172.21.15.60:0/202612877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6668100080 0x7f6668104010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f66677fe640 1 -- 172.21.15.60:0/202612877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c009560 con 0x7f6668101270 2024-02-26T04:59:36.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f66677fe640 1 --2- 172.21.15.60:0/202612877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6668101270 0x7f6668102650 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f665c036040 tx=0x7f665c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:36.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f66657fa640 1 -- 172.21.15.60:0/202612877 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c0044e0 con 0x7f6668101270 2024-02-26T04:59:36.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f66657fa640 1 -- 172.21.15.60:0/202612877 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f665c0054e0 con 0x7f6668101270 2024-02-26T04:59:36.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f666ea36640 1 -- 172.21.15.60:0/202612877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66681a1510 con 0x7f6668101270 2024-02-26T04:59:36.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f66657fa640 1 -- 172.21.15.60:0/202612877 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c003ec0 con 0x7f6668101270 2024-02-26T04:59:36.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.548+0000 7f666ea36640 1 -- 172.21.15.60:0/202612877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66681a1a20 con 0x7f6668101270 2024-02-26T04:59:36.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.549+0000 7f666ea36640 1 -- 172.21.15.60:0/202612877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666810ba90 con 0x7f6668101270 2024-02-26T04:59:36.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.550+0000 7f66657fa640 1 -- 172.21.15.60:0/202612877 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f665c0035f0 con 0x7f6668101270 2024-02-26T04:59:36.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.551+0000 7f66657fa640 1 --2- 172.21.15.60:0/202612877 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f66500a20c0 0x7f66500a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:36.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.551+0000 7f6667fff640 1 --2- 172.21.15.60:0/202612877 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f66500a20c0 0x7f66500a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:36.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.551+0000 7f66657fa640 1 -- 172.21.15.60:0/202612877 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f665c0f0950 con 0x7f6668101270 2024-02-26T04:59:36.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.552+0000 7f6667fff640 1 --2- 172.21.15.60:0/202612877 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f66500a20c0 0x7f66500a4580 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6658005fd0 tx=0x7f665800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:36.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.555+0000 7f66657fa640 1 -- 172.21.15.60:0/202612877 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f665c0b94e0 con 0x7f6668101270 2024-02-26T04:59:36.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:36 smithi156 ceph-mon[35805]: pgmap v421: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 92 KiB/s wr, 34 op/s 2024-02-26T04:59:36.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.736+0000 7f666ea36640 1 -- 172.21.15.60:0/202612877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66681a1eb0 con 0x7f6668101270 2024-02-26T04:59:36.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.737+0000 7f66657fa640 1 -- 172.21.15.60:0/202612877 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f665c0bcba0 con 0x7f6668101270 2024-02-26T04:59:36.739 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T04:59:36.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.738+0000 7f666ea36640 1 -- 172.21.15.60:0/202612877 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f66500a20c0 msgr2=0x7f66500a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:36.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.738+0000 7f666ea36640 1 --2- 172.21.15.60:0/202612877 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f66500a20c0 0x7f66500a4580 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6658005fd0 tx=0x7f665800b410 comp rx=0 tx=0).stop 2024-02-26T04:59:36.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.738+0000 7f666ea36640 1 -- 172.21.15.60:0/202612877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6668101270 msgr2=0x7f6668102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:36.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.738+0000 7f666ea36640 1 --2- 172.21.15.60:0/202612877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6668101270 0x7f6668102650 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f665c036040 tx=0x7f665c0042f0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.739+0000 7f666ea36640 1 -- 172.21.15.60:0/202612877 shutdown_connections 2024-02-26T04:59:36.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.739+0000 7f666ea36640 1 --2- 172.21.15.60:0/202612877 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f66500a20c0 0x7f66500a4580 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.739+0000 7f666ea36640 1 --2- 172.21.15.60:0/202612877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6668101c30 0x7f6668102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.739+0000 7f666ea36640 1 --2- 172.21.15.60:0/202612877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6668101270 0x7f6668102650 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.739+0000 7f666ea36640 1 --2- 172.21.15.60:0/202612877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6668100080 0x7f6668104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.739+0000 7f666ea36640 1 -- 172.21.15.60:0/202612877 >> 172.21.15.60:0/202612877 conn(0x7f66680fb610 msgr2=0x7f66680fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:36.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.739+0000 7f666ea36640 1 -- 172.21.15.60:0/202612877 shutdown_connections 2024-02-26T04:59:36.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.739+0000 7f666ea36640 1 -- 172.21.15.60:0/202612877 wait complete. 2024-02-26T04:59:36.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T04:59:36.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.855+0000 7f581ae36640 1 -- 172.21.15.60:0/580034468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58141000a0 msgr2=0x7f58141004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:36.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.855+0000 7f581ae36640 1 --2- 172.21.15.60:0/580034468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58141000a0 0x7f58141004c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f580c009ca0 tx=0x7f580c02c0f0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.855+0000 7f581ae36640 1 -- 172.21.15.60:0/580034468 shutdown_connections 2024-02-26T04:59:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.855+0000 7f581ae36640 1 --2- 172.21.15.60:0/580034468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5814101c50 0x7f58141084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.855+0000 7f581ae36640 1 --2- 172.21.15.60:0/580034468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5814101290 0x7f5814101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.855+0000 7f581ae36640 1 --2- 172.21.15.60:0/580034468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58141000a0 0x7f58141004c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.855+0000 7f581ae36640 1 -- 172.21.15.60:0/580034468 >> 172.21.15.60:0/580034468 conn(0x7f58140fb630 msgr2=0x7f58140fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.855+0000 7f581ae36640 1 -- 172.21.15.60:0/580034468 shutdown_connections 2024-02-26T04:59:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.855+0000 7f581ae36640 1 -- 172.21.15.60:0/580034468 wait complete. 2024-02-26T04:59:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f581ae36640 1 Processor -- start 2024-02-26T04:59:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f581ae36640 1 -- start start 2024-02-26T04:59:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f581ae36640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58141000a0 0x7f58141961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f581ae36640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5814101290 0x7f58141966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f581ae36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5814101c50 0x7f581419d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f581ae36640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581419dc90 con 0x7f5814101c50 2024-02-26T04:59:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f581ae36640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581419de00 con 0x7f58141000a0 2024-02-26T04:59:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f581ae36640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581419dfc0 con 0x7f5814101290 2024-02-26T04:59:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f58193d3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5814101c50 0x7f581419d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f58193d3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5814101c50 0x7f581419d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43622/0 (socket says 172.21.15.60:43622) 2024-02-26T04:59:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f58193d3640 1 -- 172.21.15.60:0/3218842152 learned_addr learned my addr 172.21.15.60:0/3218842152 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f5813fff640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5814101290 0x7f58141966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f5818bd2640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58141000a0 0x7f58141961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f58193d3640 1 -- 172.21.15.60:0/3218842152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5814101290 msgr2=0x7f58141966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f58193d3640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5814101290 0x7f58141966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f58193d3640 1 -- 172.21.15.60:0/3218842152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58141000a0 msgr2=0x7f58141961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f58193d3640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58141000a0 0x7f58141961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f58193d3640 1 -- 172.21.15.60:0/3218842152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c009900 con 0x7f5814101c50 2024-02-26T04:59:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.856+0000 7f5813fff640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5814101290 0x7f58141966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:59:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.857+0000 7f58193d3640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5814101c50 0x7f581419d750 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f580400d950 tx=0x7f580400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.857+0000 7f5811ffb640 1 -- 172.21.15.60:0/3218842152 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804009a30 con 0x7f5814101c50 2024-02-26T04:59:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.857+0000 7f5811ffb640 1 -- 172.21.15.60:0/3218842152 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5804009b90 con 0x7f5814101c50 2024-02-26T04:59:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.857+0000 7f581ae36640 1 -- 172.21.15.60:0/3218842152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581419e2a0 con 0x7f5814101c50 2024-02-26T04:59:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.857+0000 7f5811ffb640 1 -- 172.21.15.60:0/3218842152 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804003d60 con 0x7f5814101c50 2024-02-26T04:59:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.857+0000 7f581ae36640 1 -- 172.21.15.60:0/3218842152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581419e7b0 con 0x7f5814101c50 2024-02-26T04:59:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.858+0000 7f581ae36640 1 -- 172.21.15.60:0/3218842152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5814107430 con 0x7f5814101c50 2024-02-26T04:59:36.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.859+0000 7f5811ffb640 1 -- 172.21.15.60:0/3218842152 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5804009d00 con 0x7f5814101c50 2024-02-26T04:59:36.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.859+0000 7f5811ffb640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f57fc0a2140 0x7f57fc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:36.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.860+0000 7f5818bd2640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f57fc0a2140 0x7f57fc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:36.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.860+0000 7f5811ffb640 1 -- 172.21.15.60:0/3218842152 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f58040cdbe0 con 0x7f5814101c50 2024-02-26T04:59:36.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.860+0000 7f5818bd2640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f57fc0a2140 0x7f57fc0a4600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f580c037040 tx=0x7f580c009890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:36.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:36.864+0000 7f5811ffb640 1 -- 172.21.15.60:0/3218842152 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5804096770 con 0x7f5814101c50 2024-02-26T04:59:37.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.043+0000 7f581ae36640 1 -- 172.21.15.60:0/3218842152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f581402ce40 con 0x7f5814101c50 2024-02-26T04:59:37.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.044+0000 7f5811ffb640 1 -- 172.21.15.60:0/3218842152 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5804099e30 con 0x7f5814101c50 2024-02-26T04:59:37.046 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:59:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.046+0000 7f581ae36640 1 -- 172.21.15.60:0/3218842152 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f57fc0a2140 msgr2=0x7f57fc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.046+0000 7f581ae36640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f57fc0a2140 0x7f57fc0a4600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f580c037040 tx=0x7f580c009890 comp rx=0 tx=0).stop 2024-02-26T04:59:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.046+0000 7f581ae36640 1 -- 172.21.15.60:0/3218842152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5814101c50 msgr2=0x7f581419d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.046+0000 7f581ae36640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5814101c50 0x7f581419d750 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f580400d950 tx=0x7f580400de10 comp rx=0 tx=0).stop 2024-02-26T04:59:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.046+0000 7f581ae36640 1 -- 172.21.15.60:0/3218842152 shutdown_connections 2024-02-26T04:59:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.046+0000 7f581ae36640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f57fc0a2140 0x7f57fc0a4600 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.046+0000 7f581ae36640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5814101c50 0x7f581419d750 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.046+0000 7f581ae36640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5814101290 0x7f58141966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.046+0000 7f581ae36640 1 --2- 172.21.15.60:0/3218842152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58141000a0 0x7f58141961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.046+0000 7f581ae36640 1 -- 172.21.15.60:0/3218842152 >> 172.21.15.60:0/3218842152 conn(0x7f58140fb630 msgr2=0x7f5814104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:37.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.046+0000 7f581ae36640 1 -- 172.21.15.60:0/3218842152 shutdown_connections 2024-02-26T04:59:37.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.046+0000 7f581ae36640 1 -- 172.21.15.60:0/3218842152 wait complete. 2024-02-26T04:59:37.059 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T04:59:37.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.162+0000 7f77d4a1c640 1 -- 172.21.15.60:0/4145036167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d0101270 msgr2=0x7f77d01016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.162+0000 7f77d4a1c640 1 --2- 172.21.15.60:0/4145036167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d0101270 0x7f77d01016f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009900 tx=0x7f77b802bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:37.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.162+0000 7f77d4a1c640 1 -- 172.21.15.60:0/4145036167 shutdown_connections 2024-02-26T04:59:37.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.162+0000 7f77d4a1c640 1 --2- 172.21.15.60:0/4145036167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77d0101c30 0x7f77d01084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.162+0000 7f77d4a1c640 1 --2- 172.21.15.60:0/4145036167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d0101270 0x7f77d01016f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.162+0000 7f77d4a1c640 1 --2- 172.21.15.60:0/4145036167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77d0100080 0x7f77d01004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.162+0000 7f77d4a1c640 1 -- 172.21.15.60:0/4145036167 >> 172.21.15.60:0/4145036167 conn(0x7f77d00fb610 msgr2=0x7f77d00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.162+0000 7f77d4a1c640 1 -- 172.21.15.60:0/4145036167 shutdown_connections 2024-02-26T04:59:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.162+0000 7f77d4a1c640 1 -- 172.21.15.60:0/4145036167 wait complete. 2024-02-26T04:59:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.163+0000 7f77d4a1c640 1 Processor -- start 2024-02-26T04:59:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.163+0000 7f77d4a1c640 1 -- start start 2024-02-26T04:59:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.163+0000 7f77d4a1c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77d0100080 0x7f77d010a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.163+0000 7f77d4a1c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d0101270 0x7f77d0108f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.163+0000 7f77d4a1c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77d0101c30 0x7f77d01094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.163+0000 7f77d4a1c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d0109c20 con 0x7f77d0101270 2024-02-26T04:59:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.163+0000 7f77d4a1c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d0109d90 con 0x7f77d0101c30 2024-02-26T04:59:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.163+0000 7f77d4a1c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01a1290 con 0x7f77d0100080 2024-02-26T04:59:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.163+0000 7f77c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d0101270 0x7f77d0108f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.163+0000 7f77c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d0101270 0x7f77d0108f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43648/0 (socket says 172.21.15.60:43648) 2024-02-26T04:59:37.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.163+0000 7f77c7fff640 1 -- 172.21.15.60:0/353136538 learned_addr learned my addr 172.21.15.60:0/353136538 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:37.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.164+0000 7f77ced9d640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77d0101c30 0x7f77d01094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:37.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.164+0000 7f77ce59c640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77d0100080 0x7f77d010a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:37.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.164+0000 7f77c7fff640 1 -- 172.21.15.60:0/353136538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77d0100080 msgr2=0x7f77d010a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.164+0000 7f77c7fff640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77d0100080 0x7f77d010a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.164+0000 7f77c7fff640 1 -- 172.21.15.60:0/353136538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77d0101c30 msgr2=0x7f77d01094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.164+0000 7f77c7fff640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77d0101c30 0x7f77d01094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.164+0000 7f77c7fff640 1 -- 172.21.15.60:0/353136538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b8009560 con 0x7f77d0101270 2024-02-26T04:59:37.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.164+0000 7f77ced9d640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77d0101c30 0x7f77d01094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:59:37.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.164+0000 7f77c7fff640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d0101270 0x7f77d0108f80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f77b8036040 tx=0x7f77b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:37.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.164+0000 7f77c77fe640 1 -- 172.21.15.60:0/353136538 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b80044e0 con 0x7f77d0101270 2024-02-26T04:59:37.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.164+0000 7f77d4a1c640 1 -- 172.21.15.60:0/353136538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d01a14e0 con 0x7f77d0101270 2024-02-26T04:59:37.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.164+0000 7f77c77fe640 1 -- 172.21.15.60:0/353136538 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77b80054e0 con 0x7f77d0101270 2024-02-26T04:59:37.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.164+0000 7f77c77fe640 1 -- 172.21.15.60:0/353136538 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b8003ec0 con 0x7f77d0101270 2024-02-26T04:59:37.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.164+0000 7f77d4a1c640 1 -- 172.21.15.60:0/353136538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d01a19f0 con 0x7f77d0101270 2024-02-26T04:59:37.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.165+0000 7f77d4a1c640 1 -- 172.21.15.60:0/353136538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d01073f0 con 0x7f77d0101270 2024-02-26T04:59:37.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.166+0000 7f77c77fe640 1 -- 172.21.15.60:0/353136538 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f77b80035f0 con 0x7f77d0101270 2024-02-26T04:59:37.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.167+0000 7f77c77fe640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f77bc0a20c0 0x7f77bc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:37.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.167+0000 7f77c77fe640 1 -- 172.21.15.60:0/353136538 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f77b80f0540 con 0x7f77d0101270 2024-02-26T04:59:37.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.167+0000 7f77ce59c640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f77bc0a20c0 0x7f77bc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:37.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.167+0000 7f77ce59c640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f77bc0a20c0 0x7f77bc0a4580 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f77c0005fd0 tx=0x7f77c0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:37.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.171+0000 7f77c77fe640 1 -- 172.21.15.60:0/353136538 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f77b80b90d0 con 0x7f77d0101270 2024-02-26T04:59:37.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.356+0000 7f77d4a1c640 1 -- 172.21.15.60:0/353136538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77d002cdd0 con 0x7f77d0101270 2024-02-26T04:59:37.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.357+0000 7f77c77fe640 1 -- 172.21.15.60:0/353136538 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77b80bc790 con 0x7f77d0101270 2024-02-26T04:59:37.359 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T04:59:37.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.358+0000 7f77d4a1c640 1 -- 172.21.15.60:0/353136538 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f77bc0a20c0 msgr2=0x7f77bc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.358+0000 7f77d4a1c640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f77bc0a20c0 0x7f77bc0a4580 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f77c0005fd0 tx=0x7f77c0005850 comp rx=0 tx=0).stop 2024-02-26T04:59:37.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.359+0000 7f77d4a1c640 1 -- 172.21.15.60:0/353136538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d0101270 msgr2=0x7f77d0108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.359+0000 7f77d4a1c640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d0101270 0x7f77d0108f80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f77b8036040 tx=0x7f77b80042f0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.359+0000 7f77d4a1c640 1 -- 172.21.15.60:0/353136538 shutdown_connections 2024-02-26T04:59:37.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.359+0000 7f77d4a1c640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f77bc0a20c0 0x7f77bc0a4580 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.359+0000 7f77d4a1c640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77d0101c30 0x7f77d01094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.359+0000 7f77d4a1c640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d0101270 0x7f77d0108f80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.359+0000 7f77d4a1c640 1 --2- 172.21.15.60:0/353136538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77d0100080 0x7f77d010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.359+0000 7f77d4a1c640 1 -- 172.21.15.60:0/353136538 >> 172.21.15.60:0/353136538 conn(0x7f77d00fb610 msgr2=0x7f77d01045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:37.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.359+0000 7f77d4a1c640 1 -- 172.21.15.60:0/353136538 shutdown_connections 2024-02-26T04:59:37.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.359+0000 7f77d4a1c640 1 -- 172.21.15.60:0/353136538 wait complete. 2024-02-26T04:59:37.371 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:37 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3916378541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:59:37.371 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:37 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/202612877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:59:37.371 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:37 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3218842152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:59:37.372 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T04:59:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.475+0000 7f123791e640 1 -- 172.21.15.60:0/560126662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1230101270 msgr2=0x7f12301016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.475+0000 7f123791e640 1 --2- 172.21.15.60:0/560126662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1230101270 0x7f12301016f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1220009900 tx=0x7f122002bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.475+0000 7f123791e640 1 -- 172.21.15.60:0/560126662 shutdown_connections 2024-02-26T04:59:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.475+0000 7f123791e640 1 --2- 172.21.15.60:0/560126662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1230101c30 0x7f12301084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.475+0000 7f123791e640 1 --2- 172.21.15.60:0/560126662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1230101270 0x7f12301016f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.475+0000 7f123791e640 1 --2- 172.21.15.60:0/560126662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1230100080 0x7f12301004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.475+0000 7f123791e640 1 -- 172.21.15.60:0/560126662 >> 172.21.15.60:0/560126662 conn(0x7f12300fb610 msgr2=0x7f12300fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.475+0000 7f123791e640 1 -- 172.21.15.60:0/560126662 shutdown_connections 2024-02-26T04:59:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.475+0000 7f123791e640 1 -- 172.21.15.60:0/560126662 wait complete. 2024-02-26T04:59:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.475+0000 7f123791e640 1 Processor -- start 2024-02-26T04:59:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.475+0000 7f123791e640 1 -- start start 2024-02-26T04:59:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f123791e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1230100080 0x7f1230196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f123791e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1230101270 0x7f1230196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f123791e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1230101c30 0x7f123019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f123791e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123019dc20 con 0x7f1230101c30 2024-02-26T04:59:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f123791e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123019dd90 con 0x7f1230100080 2024-02-26T04:59:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f123791e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123019df50 con 0x7f1230101270 2024-02-26T04:59:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f1235ebb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1230101c30 0x7f123019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f1235ebb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1230101c30 0x7f123019d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43676/0 (socket says 172.21.15.60:43676) 2024-02-26T04:59:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f1235ebb640 1 -- 172.21.15.60:0/236417722 learned_addr learned my addr 172.21.15.60:0/236417722 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f1234eb9640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1230101270 0x7f1230196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f12356ba640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1230100080 0x7f1230196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f1235ebb640 1 -- 172.21.15.60:0/236417722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1230101270 msgr2=0x7f1230196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f1235ebb640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1230101270 0x7f1230196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f1235ebb640 1 -- 172.21.15.60:0/236417722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1230100080 msgr2=0x7f1230196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f1235ebb640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1230100080 0x7f1230196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f1235ebb640 1 -- 172.21.15.60:0/236417722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1220009560 con 0x7f1230101c30 2024-02-26T04:59:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f1234eb9640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1230101270 0x7f1230196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T04:59:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f1235ebb640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1230101c30 0x7f123019d6e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f122800eaa0 tx=0x7f122800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f12267fc640 1 -- 172.21.15.60:0/236417722 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122800cdb0 con 0x7f1230101c30 2024-02-26T04:59:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f123791e640 1 -- 172.21.15.60:0/236417722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123019e230 con 0x7f1230101c30 2024-02-26T04:59:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f12267fc640 1 -- 172.21.15.60:0/236417722 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1228005020 con 0x7f1230101c30 2024-02-26T04:59:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f12267fc640 1 -- 172.21.15.60:0/236417722 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12280116b0 con 0x7f1230101c30 2024-02-26T04:59:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.476+0000 7f123791e640 1 -- 172.21.15.60:0/236417722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123019e740 con 0x7f1230101c30 2024-02-26T04:59:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.477+0000 7f123791e640 1 -- 172.21.15.60:0/236417722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12301073f0 con 0x7f1230101c30 2024-02-26T04:59:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.478+0000 7f12267fc640 1 -- 172.21.15.60:0/236417722 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1228005520 con 0x7f1230101c30 2024-02-26T04:59:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.479+0000 7f12267fc640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f121c0a20c0 0x7f121c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.479+0000 7f12356ba640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f121c0a20c0 0x7f121c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.479+0000 7f12267fc640 1 -- 172.21.15.60:0/236417722 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f12280ce370 con 0x7f1230101c30 2024-02-26T04:59:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.480+0000 7f12356ba640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f121c0a20c0 0x7f121c0a4580 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f122c005fd0 tx=0x7f122c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:37.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.483+0000 7f12267fc640 1 -- 172.21.15.60:0/236417722 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1228096f00 con 0x7f1230101c30 2024-02-26T04:59:37.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:37 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3916378541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:59:37.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:37 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/202612877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:59:37.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:37 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3218842152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:59:37.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.664+0000 7f123791e640 1 -- 172.21.15.60:0/236417722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f123010ba90 con 0x7f1230101c30 2024-02-26T04:59:37.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.664+0000 7f12267fc640 1 -- 172.21.15.60:0/236417722 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f123010ba90 con 0x7f1230101c30 2024-02-26T04:59:37.667 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T04:59:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.666+0000 7f123791e640 1 -- 172.21.15.60:0/236417722 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f121c0a20c0 msgr2=0x7f121c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.666+0000 7f123791e640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f121c0a20c0 0x7f121c0a4580 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f122c005fd0 tx=0x7f122c005850 comp rx=0 tx=0).stop 2024-02-26T04:59:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.666+0000 7f123791e640 1 -- 172.21.15.60:0/236417722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1230101c30 msgr2=0x7f123019d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.666+0000 7f123791e640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1230101c30 0x7f123019d6e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f122800eaa0 tx=0x7f122800ef60 comp rx=0 tx=0).stop 2024-02-26T04:59:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.666+0000 7f123791e640 1 -- 172.21.15.60:0/236417722 shutdown_connections 2024-02-26T04:59:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.666+0000 7f123791e640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f121c0a20c0 0x7f121c0a4580 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.666+0000 7f123791e640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1230101c30 0x7f123019d6e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.666+0000 7f123791e640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1230101270 0x7f1230196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.666+0000 7f123791e640 1 --2- 172.21.15.60:0/236417722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1230100080 0x7f1230196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.666+0000 7f123791e640 1 -- 172.21.15.60:0/236417722 >> 172.21.15.60:0/236417722 conn(0x7f12300fb610 msgr2=0x7f12301045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.666+0000 7f123791e640 1 -- 172.21.15.60:0/236417722 shutdown_connections 2024-02-26T04:59:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.666+0000 7f123791e640 1 -- 172.21.15.60:0/236417722 wait complete. 2024-02-26T04:59:37.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:37 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3916378541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T04:59:37.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:37 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/202612877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T04:59:37.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:37 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3218842152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T04:59:37.680 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-02-26T04:59:37.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.779+0000 7fe07f51e640 1 -- 172.21.15.60:0/3085633102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe078101c50 msgr2=0x7fe0781084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.779+0000 7fe07f51e640 1 --2- 172.21.15.60:0/3085633102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe078101c50 0x7fe0781084e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe068009900 tx=0x7fe06802bc30 comp rx=0 tx=0).stop 2024-02-26T04:59:37.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 -- 172.21.15.60:0/3085633102 shutdown_connections 2024-02-26T04:59:37.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 --2- 172.21.15.60:0/3085633102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe078101c50 0x7fe0781084e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 --2- 172.21.15.60:0/3085633102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe078101290 0x7fe078101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 --2- 172.21.15.60:0/3085633102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0781000a0 0x7fe0781004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 -- 172.21.15.60:0/3085633102 >> 172.21.15.60:0/3085633102 conn(0x7fe0780fb610 msgr2=0x7fe0780fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:37.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 -- 172.21.15.60:0/3085633102 shutdown_connections 2024-02-26T04:59:37.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 -- 172.21.15.60:0/3085633102 wait complete. 2024-02-26T04:59:37.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 Processor -- start 2024-02-26T04:59:37.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 -- start start 2024-02-26T04:59:37.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0781000a0 0x7fe0781960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:37.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe078101290 0x7fe078196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:37.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe078101c50 0x7fe07819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:37.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07819dbb0 con 0x7fe078101c50 2024-02-26T04:59:37.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07819dd20 con 0x7fe0781000a0 2024-02-26T04:59:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07f51e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07819dee0 con 0x7fe078101290 2024-02-26T04:59:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07dabb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe078101c50 0x7fe07819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07dabb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe078101c50 0x7fe07819d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43706/0 (socket says 172.21.15.60:43706) 2024-02-26T04:59:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.780+0000 7fe07dabb640 1 -- 172.21.15.60:0/3556912836 learned_addr learned my addr 172.21.15.60:0/3556912836 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T04:59:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.781+0000 7fe07d2ba640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0781000a0 0x7fe0781960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.781+0000 7fe07cab9640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe078101290 0x7fe078196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.781+0000 7fe07dabb640 1 -- 172.21.15.60:0/3556912836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe078101290 msgr2=0x7fe078196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.781+0000 7fe07dabb640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe078101290 0x7fe078196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.781+0000 7fe07dabb640 1 -- 172.21.15.60:0/3556912836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0781000a0 msgr2=0x7fe0781960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.781+0000 7fe07dabb640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0781000a0 0x7fe0781960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.781+0000 7fe07dabb640 1 -- 172.21.15.60:0/3556912836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe068009560 con 0x7fe078101c50 2024-02-26T04:59:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.781+0000 7fe07cab9640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe078101290 0x7fe078196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T04:59:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.781+0000 7fe07dabb640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe078101c50 0x7fe07819d670 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe068009230 tx=0x7fe068004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.781+0000 7fe0767fc640 1 -- 172.21.15.60:0/3556912836 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068004df0 con 0x7fe078101c50 2024-02-26T04:59:37.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.781+0000 7fe0767fc640 1 -- 172.21.15.60:0/3556912836 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe068005430 con 0x7fe078101c50 2024-02-26T04:59:37.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.781+0000 7fe07f51e640 1 -- 172.21.15.60:0/3556912836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07819e160 con 0x7fe078101c50 2024-02-26T04:59:37.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.781+0000 7fe0767fc640 1 -- 172.21.15.60:0/3556912836 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068003de0 con 0x7fe078101c50 2024-02-26T04:59:37.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.781+0000 7fe07f51e640 1 -- 172.21.15.60:0/3556912836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07819e670 con 0x7fe078101c50 2024-02-26T04:59:37.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.782+0000 7fe07f51e640 1 -- 172.21.15.60:0/3556912836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0781073c0 con 0x7fe078101c50 2024-02-26T04:59:37.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.783+0000 7fe0767fc640 1 -- 172.21.15.60:0/3556912836 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe0680055a0 con 0x7fe078101c50 2024-02-26T04:59:37.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.784+0000 7fe0767fc640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe0600a20c0 0x7fe0600a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T04:59:37.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.784+0000 7fe0767fc640 1 -- 172.21.15.60:0/3556912836 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fe0680f0dd0 con 0x7fe078101c50 2024-02-26T04:59:37.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.784+0000 7fe07d2ba640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe0600a20c0 0x7fe0600a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T04:59:37.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.784+0000 7fe07d2ba640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe0600a20c0 0x7fe0600a4580 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe0781010f0 tx=0x7fe070009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T04:59:37.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:37.787+0000 7fe0767fc640 1 -- 172.21.15.60:0/3556912836 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe0680b9a80 con 0x7fe078101c50 2024-02-26T04:59:38.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:38.020+0000 7fe07f51e640 1 -- 172.21.15.60:0/3556912836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fe07802ce40 con 0x7fe078101c50 2024-02-26T04:59:38.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:38.029+0000 7fe0767fc640 1 -- 172.21.15.60:0/3556912836 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+736412 (secure 0 0 0) 0x7fe068043020 con 0x7fe078101c50 2024-02-26T04:59:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T04:59:38.041 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:59:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 18, 2024-02-26T04:59:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: "active_gid": 14152, 2024-02-26T04:59:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: "active_name": "x", 2024-02-26T04:59:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: "active_addrs": { 2024-02-26T04:59:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-02-26T04:59:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-02-26T04:59:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6800", 2024-02-26T04:59:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3458567419 2024-02-26T04:59:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-02-26T04:59:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6801", 2024-02-26T04:59:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3458567419 2024-02-26T04:59:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T04:59:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "active_addr": "172.21.15.60:6801/3458567419", 2024-02-26T04:59:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "active_change": "2024-02-26T04:48:18.224018+0000", 2024-02-26T04:59:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "active_mgr_features": 4540701547738038271, 2024-02-26T04:59:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "available": true, 2024-02-26T04:59:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "standbys": [ 2024-02-26T04:59:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "gid": 24109, 2024-02-26T04:59:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "z", 2024-02-26T04:59:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_features": 4540701547738038271, 2024-02-26T04:59:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "available_modules": [ 2024-02-26T04:59:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "alerts", 2024-02-26T04:59:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How frequently to reexamine health status", 2024-02-26T04:59:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.047 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.048 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.048 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.048 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.048 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.048 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.048 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.048 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.048 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.048 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.048 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.048 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.048 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.048 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_destination": { 2024-02-26T04:59:38.049 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_destination", 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email address to send alerts to", 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_from_name": { 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_from_name", 2024-02-26T04:59:38.050 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "Ceph", 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email From: name", 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_host": { 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_host", 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.051 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.052 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.052 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.052 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.052 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.052 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.052 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP server", 2024-02-26T04:59:38.052 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.052 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.052 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.052 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_password": { 2024-02-26T04:59:38.052 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_password", 2024-02-26T04:59:38.052 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.052 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Password to authenticate with", 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_port": { 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_port", 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.053 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "465", 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP port", 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_sender": { 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_sender", 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.054 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.055 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.055 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.055 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.055 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP envelope sender", 2024-02-26T04:59:38.055 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.055 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.055 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.055 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_ssl": { 2024-02-26T04:59:38.055 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_ssl", 2024-02-26T04:59:38.055 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.055 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.055 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.055 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.056 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.056 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.056 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.056 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-26T04:59:38.056 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.056 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.056 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.056 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_user": { 2024-02-26T04:59:38.056 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_user", 2024-02-26T04:59:38.056 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.056 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.056 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.056 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.057 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.057 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.057 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.057 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "User to authenticate as", 2024-02-26T04:59:38.057 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.057 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.057 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.057 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.057 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.057 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "balancer", 2024-02-26T04:59:38.057 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.057 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.058 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.058 INFO:teuthology.orchestra.run.smithi060.stdout: "active": { 2024-02-26T04:59:38.058 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "active", 2024-02-26T04:59:38.058 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.058 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.058 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.058 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.058 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.058 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.058 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.058 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically balance PGs across cluster", 2024-02-26T04:59:38.058 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.058 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.058 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_time": { 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_time", 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0000", 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "beginning time of day to automatically balance", 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.060 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_weekday": { 2024-02-26T04:59:38.060 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_weekday", 2024-02-26T04:59:38.060 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:38.060 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.060 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.060 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:38.060 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-02-26T04:59:38.060 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-02-26T04:59:38.060 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.060 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-26T04:59:38.060 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T04:59:38.060 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.060 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_max_iterations": { 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_max_iterations", 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "25", 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "250", 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.061 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_metrics": { 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_metrics", 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "pgs,objects,bytes", 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_step": { 2024-02-26T04:59:38.062 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_step", 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.5", 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0.001", 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "0.999", 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "aggressiveness of optimization", 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: "end_time": { 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_time", 2024-02-26T04:59:38.063 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.064 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.064 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.064 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2359", 2024-02-26T04:59:38.064 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.064 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.064 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.064 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ending time of day to automatically balance", 2024-02-26T04:59:38.064 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T04:59:38.064 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.064 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.064 INFO:teuthology.orchestra.run.smithi060.stdout: "end_weekday": { 2024-02-26T04:59:38.064 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_weekday", 2024-02-26T04:59:38.064 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.065 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.066 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.066 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.066 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.066 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.066 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.066 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.066 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.066 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.066 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.066 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.066 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.066 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.066 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.066 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.067 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.067 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.067 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.067 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.067 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.067 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.067 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.067 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.067 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.067 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.067 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.067 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.068 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.068 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.068 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.068 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.068 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.068 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.068 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.068 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.068 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.068 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.068 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.068 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.068 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.069 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.070 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.070 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.070 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.070 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.070 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.070 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.070 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.070 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.070 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.070 INFO:teuthology.orchestra.run.smithi060.stdout: "min_score": { 2024-02-26T04:59:38.070 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_score", 2024-02-26T04:59:38.070 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.070 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.071 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "upmap", 2024-02-26T04:59:38.072 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.072 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.072 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.072 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-compat", 2024-02-26T04:59:38.072 INFO:teuthology.orchestra.run.smithi060.stdout: "none", 2024-02-26T04:59:38.072 INFO:teuthology.orchestra.run.smithi060.stdout: "read", 2024-02-26T04:59:38.072 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap", 2024-02-26T04:59:38.072 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap-read" 2024-02-26T04:59:38.072 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.072 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Balancer mode", 2024-02-26T04:59:38.072 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.072 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.072 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_ids": { 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_ids", 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.074 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:38.074 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:38.074 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.074 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.074 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.074 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:38.074 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.074 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.074 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.074 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-26T04:59:38.074 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.074 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.074 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_deviation": { 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_deviation", 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.075 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_optimizations": { 2024-02-26T04:59:38.076 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_optimizations", 2024-02-26T04:59:38.076 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:38.076 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.076 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.076 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.076 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.076 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.076 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.076 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-26T04:59:38.076 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.076 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.076 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.076 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm", 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_down_multiplier": { 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_down_multiplier", 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.077 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_refresh_rate": { 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_refresh_rate", 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "20", 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.078 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.079 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-26T04:59:38.079 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.079 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.079 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.079 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_starting_port": { 2024-02-26T04:59:38.079 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_starting_port", 2024-02-26T04:59:38.079 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.079 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.079 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.079 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4721", 2024-02-26T04:59:38.079 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.079 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.079 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.080 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-26T04:59:38.080 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.080 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.080 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.080 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_ptrace": { 2024-02-26T04:59:38.080 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_ptrace", 2024-02-26T04:59:38.080 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.080 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.080 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.080 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.080 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.080 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.080 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_interval": { 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_interval", 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_memory_target_ratio": { 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_memory_target_ratio", 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.7", 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-26T04:59:38.082 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm_log_destination": { 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm_log_destination", 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: "file", 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: "file,syslog", 2024-02-26T04:59:38.083 INFO:teuthology.orchestra.run.smithi060.stdout: "syslog" 2024-02-26T04:59:38.084 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.084 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-26T04:59:38.084 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.084 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.084 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.084 INFO:teuthology.orchestra.run.smithi060.stdout: "cgroups_split": { 2024-02-26T04:59:38.084 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cgroups_split", 2024-02-26T04:59:38.084 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.084 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.084 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.084 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.084 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "config_checks_enabled": { 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_checks_enabled", 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.085 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: "config_dashboard": { 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_dashboard", 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.086 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-26T04:59:38.087 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.087 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.087 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.087 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_alertmanager": { 2024-02-26T04:59:38.087 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_alertmanager", 2024-02-26T04:59:38.087 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.087 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.087 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.087 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-26T04:59:38.087 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.087 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.087 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.087 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_base": { 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_base", 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Container image name, without the tag", 2024-02-26T04:59:38.088 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_elasticsearch": { 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_elasticsearch", 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "elasticsearch container image", 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.089 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_grafana": { 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_grafana", 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.090 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.091 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_haproxy": { 2024-02-26T04:59:38.091 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_haproxy", 2024-02-26T04:59:38.091 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.091 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.091 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.091 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-26T04:59:38.091 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.091 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.091 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.091 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "HAproxy container image", 2024-02-26T04:59:38.091 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.091 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.091 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.092 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_agent": { 2024-02-26T04:59:38.092 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_agent", 2024-02-26T04:59:38.092 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.092 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.092 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.092 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-26T04:59:38.092 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.092 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.092 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.092 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger agent container image", 2024-02-26T04:59:38.092 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.092 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_collector": { 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_collector", 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger collector container image", 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.093 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.094 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_query": { 2024-02-26T04:59:38.094 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_query", 2024-02-26T04:59:38.094 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.094 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.094 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.094 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-26T04:59:38.094 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.094 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.094 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.094 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger query container image", 2024-02-26T04:59:38.094 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.094 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.094 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_keepalived": { 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_keepalived", 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Keepalived container image", 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.096 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_loki": { 2024-02-26T04:59:38.096 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_loki", 2024-02-26T04:59:38.096 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.096 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.096 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.096 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-26T04:59:38.096 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.096 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.096 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.096 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Loki container image", 2024-02-26T04:59:38.096 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.096 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.096 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_node_exporter": { 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_node_exporter", 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.097 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nvmeof": { 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nvmeof", 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nvme-of container image", 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_prometheus": { 2024-02-26T04:59:38.098 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_prometheus", 2024-02-26T04:59:38.099 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.099 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.099 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.099 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-26T04:59:38.099 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.099 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.099 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.099 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T04:59:38.099 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.099 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.099 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.099 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_promtail": { 2024-02-26T04:59:38.099 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_promtail", 2024-02-26T04:59:38.100 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.100 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.100 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.100 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-26T04:59:38.100 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.100 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.100 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.100 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Promtail container image", 2024-02-26T04:59:38.100 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.100 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.100 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.100 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_snmp_gateway": { 2024-02-26T04:59:38.100 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_snmp_gateway", 2024-02-26T04:59:38.101 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.101 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.101 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.101 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-26T04:59:38.101 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.101 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.101 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.101 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SNMP Gateway container image", 2024-02-26T04:59:38.101 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.101 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.101 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.101 INFO:teuthology.orchestra.run.smithi060.stdout: "container_init": { 2024-02-26T04:59:38.101 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_init", 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Run podman/docker with `--init`", 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "daemon_cache_timeout": { 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "daemon_cache_timeout", 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.102 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.103 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.103 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:38.103 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.103 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.103 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.103 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-26T04:59:38.103 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.103 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.103 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.103 INFO:teuthology.orchestra.run.smithi060.stdout: "default_cephadm_command_timeout": { 2024-02-26T04:59:38.103 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_cephadm_command_timeout", 2024-02-26T04:59:38.104 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.104 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.104 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.104 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "900", 2024-02-26T04:59:38.104 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.104 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.104 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.104 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-26T04:59:38.104 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.104 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.104 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.104 INFO:teuthology.orchestra.run.smithi060.stdout: "default_registry": { 2024-02-26T04:59:38.104 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_registry", 2024-02-26T04:59:38.105 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.105 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.105 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.105 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io", 2024-02-26T04:59:38.105 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.105 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.105 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.105 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-26T04:59:38.105 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.105 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.105 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.105 INFO:teuthology.orchestra.run.smithi060.stdout: "device_cache_timeout": { 2024-02-26T04:59:38.105 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_cache_timeout", 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1800", 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache device inventory", 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: "device_enhanced_scan": { 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_enhanced_scan", 2024-02-26T04:59:38.106 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: "facts_cache_timeout": { 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "facts_cache_timeout", 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.107 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.108 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.108 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:38.108 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.108 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.108 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.108 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache host facts data", 2024-02-26T04:59:38.108 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.108 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.108 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.108 INFO:teuthology.orchestra.run.smithi060.stdout: "host_check_interval": { 2024-02-26T04:59:38.108 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "host_check_interval", 2024-02-26T04:59:38.108 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.108 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.109 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.109 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:38.109 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.109 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.109 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.109 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to perform a host check", 2024-02-26T04:59:38.109 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.109 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.109 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.109 INFO:teuthology.orchestra.run.smithi060.stdout: "hw_monitoring": { 2024-02-26T04:59:38.109 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hw_monitoring", 2024-02-26T04:59:38.109 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.109 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.110 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.110 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.110 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.110 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.110 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.110 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-26T04:59:38.110 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.110 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.110 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.110 INFO:teuthology.orchestra.run.smithi060.stdout: "inventory_list_all": { 2024-02-26T04:59:38.110 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "inventory_list_all", 2024-02-26T04:59:38.110 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.110 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.111 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: "log_refresh_metadata": { 2024-02-26T04:59:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_refresh_metadata", 2024-02-26T04:59:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-26T04:59:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.114 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.115 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.115 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.115 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.115 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.115 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.115 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.115 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.115 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.115 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.115 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.115 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.115 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.115 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.116 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.116 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.116 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.116 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.116 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.116 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.116 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.116 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.116 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.116 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.116 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.116 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.116 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.117 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.117 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.117 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.117 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.117 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.117 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.117 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-26T04:59:38.117 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-26T04:59:38.117 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.117 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.117 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.117 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.117 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "*", 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.118 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: "max_count_per_host": { 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_count_per_host", 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.119 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of daemons per service per host", 2024-02-26T04:59:38.120 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.120 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.120 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.120 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd_draining_count": { 2024-02-26T04:59:38.120 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_osd_draining_count", 2024-02-26T04:59:38.120 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.120 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.120 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.120 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.120 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.120 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.120 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.120 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-26T04:59:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "migration_current": { 2024-02-26T04:59:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "migration_current", 2024-02-26T04:59:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.121 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "internal - do not modify", 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "root", 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm-package", 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "root" 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "mode for remote execution of cephadm", 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "oob_default_addr": { 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "oob_default_addr", 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "169.254.1.1", 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: "prometheus_alerts_path": { 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus_alerts_path", 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_insecure": { 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_insecure", 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.125 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_password": { 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_password", 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_url": { 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_url", 2024-02-26T04:59:38.126 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_username": { 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_username", 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.127 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.128 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.128 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.128 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.128 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.128 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-26T04:59:38.128 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.128 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.128 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.128 INFO:teuthology.orchestra.run.smithi060.stdout: "secure_monitoring_stack": { 2024-02-26T04:59:38.128 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secure_monitoring_stack", 2024-02-26T04:59:38.128 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.128 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.128 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: "service_discovery_port": { 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "service_discovery_port", 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.129 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8765", 2024-02-26T04:59:38.130 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.130 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.130 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.130 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "cephadm service discovery port", 2024-02-26T04:59:38.130 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.130 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.130 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.130 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_config_file": { 2024-02-26T04:59:38.130 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_config_file", 2024-02-26T04:59:38.130 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.130 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.130 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: "use_agent": { 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_agent", 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.131 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: "use_repo_digest": { 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_repo_digest", 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.132 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_failed_host_check": { 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_failed_host_check", 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.133 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if the host check fails", 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_daemons": { 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_daemons", 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.134 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_hosts": { 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_hosts", 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.135 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-26T04:59:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crash", 2024-02-26T04:59:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.136 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.137 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.138 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.139 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.140 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.140 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.140 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.140 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.140 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.140 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.140 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.140 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.140 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.140 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.140 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.140 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: "retain_interval": { 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retain_interval", 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "31536000", 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.141 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_recent_interval": { 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_recent_interval", 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1209600", 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.142 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dashboard", 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.143 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.144 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.144 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.144 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.144 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.144 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.144 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.144 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.144 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.144 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.144 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.144 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_HOST": { 2024-02-26T04:59:38.144 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-26T04:59:38.144 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.144 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.145 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_ENABLED": { 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_ENABLED", 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.146 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.147 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.147 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.147 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.147 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.147 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.147 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.147 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.147 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-26T04:59:38.147 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-26T04:59:38.147 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.147 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.147 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.147 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: "ENABLE_BROWSABLE_API": { 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.148 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.149 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.150 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.150 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.150 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.150 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.150 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.150 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-26T04:59:38.150 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-26T04:59:38.150 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.150 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.150 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.150 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.150 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.150 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.151 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.151 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.151 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.151 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.151 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.151 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-26T04:59:38.151 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-26T04:59:38.151 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.151 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.151 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.151 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.151 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.151 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.152 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.153 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.153 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.153 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.153 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.153 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-26T04:59:38.153 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-26T04:59:38.153 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.153 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.153 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.153 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.153 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.153 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.153 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.154 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.155 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.156 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.156 INFO:teuthology.orchestra.run.smithi060.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-26T04:59:38.156 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-26T04:59:38.156 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.156 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.156 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.156 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.156 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.156 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.156 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.156 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.156 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_PASSWORD": { 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.157 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_URL": { 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_URL", 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.159 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_USERNAME": { 2024-02-26T04:59:38.160 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-26T04:59:38.160 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.160 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.160 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.160 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T04:59:38.160 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.160 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.160 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.160 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.160 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.160 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.160 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.160 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-26T04:59:38.161 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-26T04:59:38.161 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.161 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.161 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.161 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.161 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.161 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.161 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.161 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.161 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.161 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.161 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.161 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-26T04:59:38.162 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.163 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_HOST": { 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.164 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.165 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-26T04:59:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-26T04:59:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.167 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.168 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.169 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.170 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.170 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.170 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.170 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.170 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-26T04:59:38.170 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-26T04:59:38.170 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.170 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.170 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.170 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.170 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.170 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.170 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.171 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.172 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.172 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.172 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-26T04:59:38.172 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-26T04:59:38.172 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.172 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.172 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.172 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.172 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.172 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.172 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.172 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.172 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_ENABLED": { 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.173 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.174 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.175 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-26T04:59:38.176 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-26T04:59:38.176 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.176 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.176 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.176 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8", 2024-02-26T04:59:38.176 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.176 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.176 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.176 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.176 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.176 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.176 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.177 INFO:teuthology.orchestra.run.smithi060.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-26T04:59:38.177 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-26T04:59:38.177 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.177 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.177 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.177 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "45", 2024-02-26T04:59:38.177 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.177 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.177 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.177 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.177 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.177 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.177 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.178 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ACCESS_KEY": { 2024-02-26T04:59:38.178 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-26T04:59:38.178 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.178 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.178 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.178 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.178 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.178 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.178 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.178 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.178 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.178 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.178 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.179 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SECRET_KEY": { 2024-02-26T04:59:38.180 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-26T04:59:38.180 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.180 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.180 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.180 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.180 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.180 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.180 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.180 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.180 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.180 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.180 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.180 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SSL_VERIFY": { 2024-02-26T04:59:38.181 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-26T04:59:38.181 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.181 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.181 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.181 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.181 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.181 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.181 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.181 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.181 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.181 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.181 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.181 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-26T04:59:38.182 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-26T04:59:38.183 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.183 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.183 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.183 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.183 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.183 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.183 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.183 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.183 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.183 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.183 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.183 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-26T04:59:38.183 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: "cross_origin_url": { 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cross_origin_url", 2024-02-26T04:59:38.184 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: "crt_file": { 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crt_file", 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.185 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: "debug": { 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "debug", 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.186 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.187 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.187 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.187 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.187 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.187 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable/disable debug options", 2024-02-26T04:59:38.187 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.187 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.187 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.187 INFO:teuthology.orchestra.run.smithi060.stdout: "jwt_token_ttl": { 2024-02-26T04:59:38.187 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "jwt_token_ttl", 2024-02-26T04:59:38.187 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.187 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.187 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.188 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "28800", 2024-02-26T04:59:38.188 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.188 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.188 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.188 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.188 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.188 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.188 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.188 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-02-26T04:59:38.188 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-02-26T04:59:38.188 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.188 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.188 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.189 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.189 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.189 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.189 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.189 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.189 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.189 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.189 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.189 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.189 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.189 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.189 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.189 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.190 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.190 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.190 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.190 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.190 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.190 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.190 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.190 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.190 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.190 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.190 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.190 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.190 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.190 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.191 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.191 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.191 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.191 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.191 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.191 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.191 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.191 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.191 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.191 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.191 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.191 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.191 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.192 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.192 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.192 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.192 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.192 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.192 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.192 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.192 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.192 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.192 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.192 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.192 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.192 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.193 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.194 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.194 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.194 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.194 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.194 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.194 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.194 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.194 INFO:teuthology.orchestra.run.smithi060.stdout: "motd": { 2024-02-26T04:59:38.194 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "motd", 2024-02-26T04:59:38.194 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.194 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.194 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.194 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "The message of the day", 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect_resolve_ip_addr": { 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "redirect_resolve_ip_addr", 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.195 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.196 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8080", 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl_server_port": { 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl_server_port", 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8443", 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-02-26T04:59:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-02-26T04:59:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "redirect", 2024-02-26T04:59:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect" 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "url_prefix": { 2024-02-26T04:59:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url_prefix", 2024-02-26T04:59:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_monitoring": { 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_monitoring", 2024-02-26T04:59:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "monitor device health metrics", 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "mark_out_threshold": { 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mark_out_threshold", 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2419200", 2024-02-26T04:59:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-26T04:59:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": { 2024-02-26T04:59:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_name", 2024-02-26T04:59:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "device_health_metrics", 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "retention_period": { 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retention_period", 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15552000", 2024-02-26T04:59:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain device health metrics", 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_frequency": { 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_frequency", 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "self_heal": { 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "self_heal", 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-26T04:59:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and check device health", 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_threshold": { 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_threshold", 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7257600", 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-26T04:59:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "diskprediction_local", 2024-02-26T04:59:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "predict_interval": { 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predict_interval", 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "predictor_model": { 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predictor_model", 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "prophetstor", 2024-02-26T04:59:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "influx", 2024-02-26T04:59:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": false, 2024-02-26T04:59:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "influxdb python module not found", 2024-02-26T04:59:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "batch_size": { 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "batch_size", 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5000", 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "database": { 2024-02-26T04:59:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "database", 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "ceph", 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": { 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hostname", 2024-02-26T04:59:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server hostname", 2024-02-26T04:59:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "30", 2024-02-26T04:59:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "5", 2024-02-26T04:59:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-26T04:59:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: "password": { 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "password", 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.233 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "password of InfluxDB server user", 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: "port": { 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "port", 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8086", 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server port", 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "false", 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T04:59:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: "threads": { 2024-02-26T04:59:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threads", 2024-02-26T04:59:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T04:59:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T04:59:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "32", 2024-02-26T04:59:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: "username": { 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "username", 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "username of InfluxDB server user", 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: "verify_ssl": { 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "verify_ssl", 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "true", 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T04:59:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "insights", 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "iostat", 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.245 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.245 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.245 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.245 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.245 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.245 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.245 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.245 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.245 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.245 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.245 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.245 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.245 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.245 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.246 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.247 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.248 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "k8sevents", 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_event_retention_days": { 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ceph_event_retention_days", 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7", 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.251 INFO:teuthology.orchestra.run.smithi060.stdout: "config_check_secs": { 2024-02-26T04:59:38.251 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_check_secs", 2024-02-26T04:59:38.251 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.251 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.251 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.251 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.251 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "10", 2024-02-26T04:59:38.251 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.251 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.251 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-26T04:59:38.251 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.251 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.251 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.252 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.252 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.252 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.252 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.252 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.252 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.252 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.252 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.252 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.252 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.252 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.252 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.252 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.252 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.253 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.255 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.256 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.256 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.256 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.256 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.256 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.256 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.256 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.256 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.256 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.256 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.256 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.256 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.256 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.257 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "localpool", 2024-02-26T04:59:38.257 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.257 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.257 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.257 INFO:teuthology.orchestra.run.smithi060.stdout: "failure_domain": { 2024-02-26T04:59:38.257 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "failure_domain", 2024-02-26T04:59:38.257 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.257 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.257 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.257 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "host", 2024-02-26T04:59:38.257 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.257 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.257 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.258 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "failure domain for any created local pool", 2024-02-26T04:59:38.258 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-26T04:59:38.258 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.258 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.258 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.258 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.258 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.258 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.258 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.258 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.258 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.258 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.258 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.259 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.260 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.260 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.260 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.260 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.260 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.260 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.260 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.260 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.260 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.260 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.260 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.260 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.260 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.261 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.262 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": { 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_size", 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default min_size for any created local pool", 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.263 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: "num_rep": { 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "num_rep", 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3", 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default replica count for any created local pool", 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": { 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_num", 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "128", 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default pg_num for any created local pool", 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.265 INFO:teuthology.orchestra.run.smithi060.stdout: "prefix": { 2024-02-26T04:59:38.266 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prefix", 2024-02-26T04:59:38.266 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.266 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.266 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.266 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.266 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.266 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.266 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.266 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name prefix for any created local pool", 2024-02-26T04:59:38.266 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.266 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.266 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.266 INFO:teuthology.orchestra.run.smithi060.stdout: "subtree": { 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "subtree", 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "rack", 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.268 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mds_autoscaler", 2024-02-26T04:59:38.268 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.268 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.268 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.268 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.268 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.268 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.268 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.268 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.268 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.268 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.268 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.268 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.269 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.269 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.269 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.269 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.269 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.269 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.269 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.269 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.269 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.269 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.269 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.269 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.269 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.270 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.270 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.270 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.270 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.270 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.270 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.270 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.270 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.270 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.270 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.270 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.270 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.270 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.271 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.272 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirroring", 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.273 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.276 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.277 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "nfs", 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.278 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.279 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.281 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.282 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.283 INFO:teuthology.orchestra.run.smithi060.stdout: "fail_fs": { 2024-02-26T04:59:38.284 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "fail_fs", 2024-02-26T04:59:38.284 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.284 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.284 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.284 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.284 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.284 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.284 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.284 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-26T04:59:38.284 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.284 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.284 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.284 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.285 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.286 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.287 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.288 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator": { 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.289 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm", 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: "rook", 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: "test_orchestrator" 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Orchestrator backend", 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_perf_query", 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.290 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.291 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.292 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.292 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.292 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.292 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.292 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.292 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.292 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.292 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.292 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.292 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.292 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.292 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.292 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.293 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.294 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.295 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.295 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.295 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.295 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.295 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.295 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.295 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.295 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.295 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.295 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.295 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.295 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_support", 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.296 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.297 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.297 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.297 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.297 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.297 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.297 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.297 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.297 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.297 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.297 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.297 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.297 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.297 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.298 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.299 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.299 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.299 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.299 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.299 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.299 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.299 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.299 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.299 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.299 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.299 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.299 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.299 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.299 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.300 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.300 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.300 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.300 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.300 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.300 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.300 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.300 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.300 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.300 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.300 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.300 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.300 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_autoscaler", 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.301 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.302 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.303 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.304 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.304 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.304 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.304 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.304 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.304 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.304 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.304 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.304 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.304 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.304 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.304 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.304 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.304 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.305 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.306 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:38.306 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:38.306 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.306 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.306 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.306 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:38.306 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.306 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.306 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.306 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.306 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.306 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: "threshold": { 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threshold", 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1.0", 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "scaling threshold", 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.307 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "progress", 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_pg_recovery_event": { 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_pg_recovery_event", 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.308 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.309 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.309 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.309 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-26T04:59:38.309 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.309 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.309 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.309 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-02-26T04:59:38.309 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-02-26T04:59:38.309 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.309 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.309 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.309 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.309 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.310 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.311 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.312 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.312 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.312 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.312 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.312 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.312 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.312 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.312 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.312 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.312 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.312 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.312 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.312 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.313 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.314 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: "max_completed_events": { 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_completed_events", 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "50", 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "number of past completed events to remember", 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.315 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.316 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.316 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:38.316 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:38.316 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.316 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.316 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.316 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T04:59:38.316 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.316 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.316 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.316 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long the module is going to sleep", 2024-02-26T04:59:38.316 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.316 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus", 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: "cache": { 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cache", 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.317 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: "exclude_perf_counters": { 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "exclude_perf_counters", 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.318 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.319 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.321 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.322 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.324 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.324 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.324 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.324 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools": { 2024-02-26T04:59:38.324 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools", 2024-02-26T04:59:38.324 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.324 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.324 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.324 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.324 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.324 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.324 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.324 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "300", 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.325 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_interval": { 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_interval", 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15.0", 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.326 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.328 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T04:59:38.328 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T04:59:38.328 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.328 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.328 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.328 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "9283", 2024-02-26T04:59:38.328 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.328 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.328 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.328 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-26T04:59:38.328 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.328 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.328 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "stale_cache_strategy": { 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stale_cache_strategy", 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "log", 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-02-26T04:59:38.329 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-02-26T04:59:38.330 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.330 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.330 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.330 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "default", 2024-02-26T04:59:38.330 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.330 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.330 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.330 INFO:teuthology.orchestra.run.smithi060.stdout: "default", 2024-02-26T04:59:38.330 INFO:teuthology.orchestra.run.smithi060.stdout: "error" 2024-02-26T04:59:38.330 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.330 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.330 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.330 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.331 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.331 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-02-26T04:59:38.331 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-02-26T04:59:38.331 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.331 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.331 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.331 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-02-26T04:59:38.331 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-02-26T04:59:38.331 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-02-26T04:59:38.331 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.331 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.331 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.331 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.332 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.332 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.332 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.332 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-02-26T04:59:38.332 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.332 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.332 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.332 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.332 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.332 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.332 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.332 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.333 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.334 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.334 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.334 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.334 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.334 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.334 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.334 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.334 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.334 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.334 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.334 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.334 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.334 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.335 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_snap_create": { 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_snap_create", 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.337 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.338 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.338 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.338 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.338 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.338 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.338 INFO:teuthology.orchestra.run.smithi060.stdout: "mirror_snapshot_schedule": { 2024-02-26T04:59:38.338 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirror_snapshot_schedule", 2024-02-26T04:59:38.338 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.338 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.338 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.338 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.338 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.338 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.339 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.339 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.339 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.339 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.339 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.339 INFO:teuthology.orchestra.run.smithi060.stdout: "trash_purge_schedule": { 2024-02-26T04:59:38.339 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "trash_purge_schedule", 2024-02-26T04:59:38.339 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.339 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.339 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.339 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.339 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.339 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "restful", 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_auth": { 2024-02-26T04:59:38.340 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_auth", 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.341 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.342 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.343 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.344 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.344 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.344 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.344 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.344 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.344 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.344 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.344 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.344 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.344 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.344 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.345 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.345 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.345 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.345 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.345 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.345 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.345 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.345 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.345 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.345 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.345 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.345 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.346 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.347 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.347 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.347 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.347 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.347 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.347 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.347 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.347 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.347 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.347 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T04:59:38.347 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T04:59:38.347 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.347 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.348 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.348 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.348 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.348 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.348 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.348 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.348 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.348 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.348 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.348 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T04:59:38.348 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T04:59:38.348 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.348 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.349 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.349 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.349 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.349 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.349 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.349 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.349 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.349 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.349 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.349 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rgw", 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.350 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.351 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.352 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.352 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.352 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.352 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.352 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.352 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.352 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.352 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.352 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.352 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.352 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.352 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.352 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.354 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rook", 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: "drive_group_interval": { 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "drive_group_interval", 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "300.0", 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.355 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.356 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-26T04:59:38.356 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.356 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.356 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.356 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.356 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.356 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.356 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.356 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.356 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.356 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.356 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.356 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.357 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.357 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.357 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.357 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.357 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.357 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.357 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.357 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.357 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.357 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.357 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.357 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.357 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.358 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.358 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.358 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.358 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.358 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.358 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.358 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.358 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.358 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.358 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.358 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.358 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.358 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.359 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: "storage_class": { 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "storage_class", 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "local", 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.361 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "selftest", 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.362 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.363 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.364 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.364 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.364 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.364 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.364 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.364 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.364 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.364 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.364 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.364 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.364 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.364 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.364 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.365 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.366 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.367 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.367 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.367 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.367 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.367 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.367 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.367 INFO:teuthology.orchestra.run.smithi060.stdout: "roption1": { 2024-02-26T04:59:38.367 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption1", 2024-02-26T04:59:38.367 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.367 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.367 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.367 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.367 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: "roption2": { 2024-02-26T04:59:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption2", 2024-02-26T04:59:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "xyz", 2024-02-26T04:59:38.368 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption1": { 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption1", 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.369 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.370 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.370 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.370 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.370 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.370 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.370 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption2": { 2024-02-26T04:59:38.370 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption2", 2024-02-26T04:59:38.370 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.370 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.370 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.370 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.370 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.370 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.371 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.371 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.371 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.371 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.371 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.371 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption3": { 2024-02-26T04:59:38.371 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption3", 2024-02-26T04:59:38.371 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.371 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.371 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.371 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.371 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.371 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.372 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.372 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.372 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.372 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.372 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.372 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption4": { 2024-02-26T04:59:38.372 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption4", 2024-02-26T04:59:38.372 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.372 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.372 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.372 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.372 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption5": { 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption5", 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.373 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption6": { 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption6", 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.374 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption7": { 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption7", 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "42", 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.375 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: "testkey": { 2024-02-26T04:59:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testkey", 2024-02-26T04:59:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.377 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.377 INFO:teuthology.orchestra.run.smithi060.stdout: "testlkey": { 2024-02-26T04:59:38.377 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testlkey", 2024-02-26T04:59:38.377 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.377 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.377 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.377 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.377 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.377 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.377 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.377 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.377 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.377 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: "testnewline": { 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testnewline", 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.378 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snap_schedule", 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_m_granularity": { 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_m_granularity", 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.379 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow minute scheduled snapshots", 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "dump_on_update": { 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dump_on_update", 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "dump database to debug log on update", 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.382 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.383 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.384 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.384 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.384 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.384 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.384 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.384 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.384 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.384 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.384 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.384 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.384 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.384 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.384 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.384 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.385 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stats", 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.386 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.387 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.388 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.388 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.388 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.388 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.388 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.388 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.388 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.388 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.388 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.388 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.388 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.388 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.388 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.389 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.390 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.390 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.390 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.390 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.390 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.390 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.390 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.390 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.390 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.390 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.390 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.390 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.390 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.391 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.391 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.391 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.391 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.391 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.391 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.391 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "status", 2024-02-26T04:59:38.391 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.391 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.391 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.391 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.391 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.392 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.392 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.392 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.392 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.392 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.392 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.392 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.392 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.392 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.392 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.392 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.392 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.392 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.392 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.393 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.394 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.395 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telegraf", 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.396 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: "address": { 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "address", 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.397 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:38.398 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:38.398 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.398 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.398 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.398 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15", 2024-02-26T04:59:38.398 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.398 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.398 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.398 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.398 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.398 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.398 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.398 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.399 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.400 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.401 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telemetry", 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.403 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.404 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.404 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_basic": { 2024-02-26T04:59:38.404 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_basic", 2024-02-26T04:59:38.404 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.404 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.404 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.404 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.404 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.404 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.404 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.404 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-26T04:59:38.404 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.404 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.404 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_crash": { 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_crash", 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.405 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_device": { 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_device", 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_ident": { 2024-02-26T04:59:38.406 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_ident", 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_perf": { 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_perf", 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.407 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.408 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.408 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.408 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.408 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.408 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.408 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-26T04:59:38.408 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.408 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.408 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.408 INFO:teuthology.orchestra.run.smithi060.stdout: "contact": { 2024-02-26T04:59:38.408 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "contact", 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: "description": { 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "description", 2024-02-26T04:59:38.409 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.410 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.410 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.410 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.410 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.410 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.410 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.410 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.410 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.410 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.410 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.410 INFO:teuthology.orchestra.run.smithi060.stdout: "device_url": { 2024-02-26T04:59:38.410 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_url", 2024-02-26T04:59:38.410 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.411 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.411 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.411 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-26T04:59:38.411 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.411 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.411 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.411 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.411 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.411 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.411 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.411 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-02-26T04:59:38.411 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-02-26T04:59:38.411 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.412 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.412 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.412 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.412 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.412 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.412 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.412 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.412 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.412 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.412 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.412 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:38.412 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:38.412 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.413 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.413 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.413 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "24", 2024-02-26T04:59:38.413 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "8", 2024-02-26T04:59:38.413 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.413 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.413 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.413 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.413 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.413 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.413 INFO:teuthology.orchestra.run.smithi060.stdout: "last_opt_revision": { 2024-02-26T04:59:38.413 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "last_opt_revision", 2024-02-26T04:59:38.413 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1", 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard": { 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard", 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.414 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard_description": { 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard_description", 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.415 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.416 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.416 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.416 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.416 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.416 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.416 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.416 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.416 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.416 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.416 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.416 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.417 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.418 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.419 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.420 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.420 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.420 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.420 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.420 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.420 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.420 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.420 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.420 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.420 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.420 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.420 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.421 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.421 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.421 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.421 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.421 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.421 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.421 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.421 INFO:teuthology.orchestra.run.smithi060.stdout: "organization": { 2024-02-26T04:59:38.421 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "organization", 2024-02-26T04:59:38.421 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.421 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.421 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.421 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "proxy": { 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "proxy", 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.422 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.423 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.423 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.423 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.423 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.423 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.423 INFO:teuthology.orchestra.run.smithi060.stdout: "url": { 2024-02-26T04:59:38.423 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url", 2024-02-26T04:59:38.423 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.423 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.423 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.423 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-26T04:59:38.423 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.423 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.424 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.424 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.424 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.424 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.424 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.424 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.424 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.424 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "test_orchestrator", 2024-02-26T04:59:38.424 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.424 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.424 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.424 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.425 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.426 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.427 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.427 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.427 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.427 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.427 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.427 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.427 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.427 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.427 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.427 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.427 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.427 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.427 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.428 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.428 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.428 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.428 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.428 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.428 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.428 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.428 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.428 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.428 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.428 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.428 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.428 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.429 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.429 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.429 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.429 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.429 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.429 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.429 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.429 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.429 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.429 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.429 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.429 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-02-26T04:59:38.430 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.430 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.430 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.430 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.430 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.430 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.430 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.430 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.430 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.430 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.430 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.430 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.430 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.430 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.432 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.432 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.432 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.432 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.432 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.432 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.432 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.432 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.432 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.432 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.432 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.432 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.432 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.433 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_clones": { 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_clones", 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4", 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.435 INFO:teuthology.orchestra.run.smithi060.stdout: "periodic_async_work": { 2024-02-26T04:59:38.436 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "periodic_async_work", 2024-02-26T04:59:38.436 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.436 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.436 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.436 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.436 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.436 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.436 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.436 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Periodically check for async work", 2024-02-26T04:59:38.436 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.436 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.436 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.436 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_delay": { 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_delay", 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix", 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: "discovery_interval": { 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "discovery_interval", 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "100", 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: "identifier": { 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "identifier", 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.439 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.440 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_host": { 2024-02-26T04:59:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_host", 2024-02-26T04:59:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_port": { 2024-02-26T04:59:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_port", 2024-02-26T04:59:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10051", 2024-02-26T04:59:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_sender": { 2024-02-26T04:59:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_sender", 2024-02-26T04:59:38.448 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.448 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.448 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.448 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-26T04:59:38.448 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.448 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.448 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.448 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.448 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.448 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.448 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.448 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.448 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.448 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: "gid": 24113, 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "y", 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_features": 4540701547738038271, 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: "available_modules": [ 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "alerts", 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.449 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.450 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.450 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:38.450 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.450 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.450 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.450 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How frequently to reexamine health status", 2024-02-26T04:59:38.450 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.450 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.450 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.450 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.450 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.450 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.450 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.451 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.452 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.452 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.452 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.452 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.452 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.452 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.452 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.452 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.452 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.452 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.452 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.452 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.452 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.453 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.454 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.455 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.455 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.455 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.455 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.455 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.455 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.455 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_destination": { 2024-02-26T04:59:38.455 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_destination", 2024-02-26T04:59:38.455 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.455 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.455 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.455 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email address to send alerts to", 2024-02-26T04:59:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_from_name": { 2024-02-26T04:59:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_from_name", 2024-02-26T04:59:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "Ceph", 2024-02-26T04:59:38.457 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.457 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.457 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.457 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email From: name", 2024-02-26T04:59:38.457 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.457 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.457 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.457 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_host": { 2024-02-26T04:59:38.457 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_host", 2024-02-26T04:59:38.457 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.457 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.457 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.457 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP server", 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_password": { 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_password", 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.458 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Password to authenticate with", 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_port": { 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_port", 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "465", 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.459 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.460 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP port", 2024-02-26T04:59:38.460 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.460 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.460 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.460 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_sender": { 2024-02-26T04:59:38.460 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_sender", 2024-02-26T04:59:38.460 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.460 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.460 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.460 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.460 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.460 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP envelope sender", 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_ssl": { 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_ssl", 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.461 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_user": { 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_user", 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "User to authenticate as", 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.462 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "balancer", 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: "active": { 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "active", 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.463 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.464 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.464 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.464 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically balance PGs across cluster", 2024-02-26T04:59:38.464 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.464 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.464 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.464 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_time": { 2024-02-26T04:59:38.464 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_time", 2024-02-26T04:59:38.464 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.464 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.464 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0000", 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "beginning time of day to automatically balance", 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_weekday": { 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_weekday", 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:38.465 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_max_iterations": { 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_max_iterations", 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "25", 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T04:59:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "250", 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_metrics": { 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_metrics", 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "pgs,objects,bytes", 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.467 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_step": { 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_step", 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.5", 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0.001", 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "0.999", 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "aggressiveness of optimization", 2024-02-26T04:59:38.468 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: "end_time": { 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_time", 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2359", 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ending time of day to automatically balance", 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T04:59:38.469 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: "end_weekday": { 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_weekday", 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.470 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.471 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.472 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.472 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.472 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.472 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.472 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.472 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.472 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.472 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.472 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.472 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.472 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.472 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.472 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.473 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.473 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.473 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.473 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.473 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.473 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.473 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.473 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.473 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.473 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.473 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.473 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.473 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.474 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.474 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.474 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.474 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.474 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.474 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.474 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.474 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.474 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.474 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.474 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.474 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.474 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.476 INFO:teuthology.orchestra.run.smithi060.stdout: "min_score": { 2024-02-26T04:59:38.476 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_score", 2024-02-26T04:59:38.476 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.476 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.476 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.476 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:38.476 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.476 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.476 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.476 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-26T04:59:38.476 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.476 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.476 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.477 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-02-26T04:59:38.477 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-02-26T04:59:38.477 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.477 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.477 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.477 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "upmap", 2024-02-26T04:59:38.477 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.477 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.477 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.477 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-compat", 2024-02-26T04:59:38.477 INFO:teuthology.orchestra.run.smithi060.stdout: "none", 2024-02-26T04:59:38.477 INFO:teuthology.orchestra.run.smithi060.stdout: "read", 2024-02-26T04:59:38.477 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap", 2024-02-26T04:59:38.477 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap-read" 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Balancer mode", 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_ids": { 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_ids", 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.478 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-26T04:59:38.479 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.480 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.480 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.480 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_deviation": { 2024-02-26T04:59:38.480 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_deviation", 2024-02-26T04:59:38.480 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.480 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.480 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.480 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T04:59:38.480 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T04:59:38.480 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.480 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.480 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-26T04:59:38.480 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-26T04:59:38.481 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.481 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.481 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_optimizations": { 2024-02-26T04:59:38.481 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_optimizations", 2024-02-26T04:59:38.481 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:38.481 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.481 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.481 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.481 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.481 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.481 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.481 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-26T04:59:38.481 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.482 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.482 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.482 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.482 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.482 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm", 2024-02-26T04:59:38.482 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.482 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.482 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.482 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_down_multiplier": { 2024-02-26T04:59:38.482 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_down_multiplier", 2024-02-26T04:59:38.482 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.482 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.483 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.483 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-02-26T04:59:38.483 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.483 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.483 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.483 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-26T04:59:38.483 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.483 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.483 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.483 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_refresh_rate": { 2024-02-26T04:59:38.483 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_refresh_rate", 2024-02-26T04:59:38.483 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "20", 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_starting_port": { 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_starting_port", 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.484 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4721", 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_ptrace": { 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_ptrace", 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.485 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.486 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.486 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.486 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.486 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.486 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-26T04:59:38.486 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-26T04:59:38.486 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.486 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.486 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_interval": { 2024-02-26T04:59:38.486 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_interval", 2024-02-26T04:59:38.486 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.486 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.486 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.487 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:38.487 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.487 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.487 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.487 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-26T04:59:38.487 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.487 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.487 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.487 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_memory_target_ratio": { 2024-02-26T04:59:38.487 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_memory_target_ratio", 2024-02-26T04:59:38.487 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.487 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.487 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.488 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.7", 2024-02-26T04:59:38.488 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.488 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.488 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.488 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-26T04:59:38.488 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.488 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.488 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.488 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm_log_destination": { 2024-02-26T04:59:38.488 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm_log_destination", 2024-02-26T04:59:38.488 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.488 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: "file", 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: "file,syslog", 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: "syslog" 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.489 INFO:teuthology.orchestra.run.smithi060.stdout: "cgroups_split": { 2024-02-26T04:59:38.490 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cgroups_split", 2024-02-26T04:59:38.490 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.490 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.490 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.490 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.490 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.490 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.490 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.490 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-26T04:59:38.490 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.490 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.490 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.490 INFO:teuthology.orchestra.run.smithi060.stdout: "config_checks_enabled": { 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_checks_enabled", 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "config_dashboard": { 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_dashboard", 2024-02-26T04:59:38.491 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.492 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.492 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.492 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.492 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.492 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.492 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.492 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-26T04:59:38.492 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.492 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.492 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.492 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_alertmanager": { 2024-02-26T04:59:38.492 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_alertmanager", 2024-02-26T04:59:38.492 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_base": { 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_base", 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.493 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Container image name, without the tag", 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_elasticsearch": { 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_elasticsearch", 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.494 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-26T04:59:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "elasticsearch container image", 2024-02-26T04:59:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_grafana": { 2024-02-26T04:59:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_grafana", 2024-02-26T04:59:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.495 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_haproxy": { 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_haproxy", 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-26T04:59:38.496 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "HAproxy container image", 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_agent": { 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_agent", 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.497 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger agent container image", 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_collector": { 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_collector", 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.498 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger collector container image", 2024-02-26T04:59:38.499 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.499 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.499 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.499 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_query": { 2024-02-26T04:59:38.499 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_query", 2024-02-26T04:59:38.499 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.499 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.499 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.499 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-26T04:59:38.499 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.499 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.499 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.499 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger query container image", 2024-02-26T04:59:38.500 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.500 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.500 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.500 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_keepalived": { 2024-02-26T04:59:38.500 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_keepalived", 2024-02-26T04:59:38.500 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.500 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.500 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.500 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-26T04:59:38.500 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.500 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.500 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.500 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Keepalived container image", 2024-02-26T04:59:38.501 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.501 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.501 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.501 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_loki": { 2024-02-26T04:59:38.501 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_loki", 2024-02-26T04:59:38.501 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.501 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.501 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.501 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-26T04:59:38.501 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.501 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.501 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.501 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Loki container image", 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_node_exporter": { 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_node_exporter", 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T04:59:38.502 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nvmeof": { 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nvmeof", 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nvme-of container image", 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_prometheus": { 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_prometheus", 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.504 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_promtail": { 2024-02-26T04:59:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_promtail", 2024-02-26T04:59:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-26T04:59:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Promtail container image", 2024-02-26T04:59:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_snmp_gateway": { 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_snmp_gateway", 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SNMP Gateway container image", 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "container_init": { 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_init", 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Run podman/docker with `--init`", 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "daemon_cache_timeout": { 2024-02-26T04:59:38.507 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "daemon_cache_timeout", 2024-02-26T04:59:38.508 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.508 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.508 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.508 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:38.508 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.508 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.508 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.508 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-26T04:59:38.508 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.508 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.508 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.508 INFO:teuthology.orchestra.run.smithi060.stdout: "default_cephadm_command_timeout": { 2024-02-26T04:59:38.508 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_cephadm_command_timeout", 2024-02-26T04:59:38.509 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.509 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.509 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.509 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "900", 2024-02-26T04:59:38.509 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.509 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.509 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.509 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-26T04:59:38.509 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.509 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.509 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.509 INFO:teuthology.orchestra.run.smithi060.stdout: "default_registry": { 2024-02-26T04:59:38.509 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_registry", 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io", 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: "device_cache_timeout": { 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_cache_timeout", 2024-02-26T04:59:38.510 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1800", 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache device inventory", 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: "device_enhanced_scan": { 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_enhanced_scan", 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.511 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.512 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.512 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.512 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.512 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.512 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.512 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-26T04:59:38.512 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.512 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.512 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.512 INFO:teuthology.orchestra.run.smithi060.stdout: "facts_cache_timeout": { 2024-02-26T04:59:38.512 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "facts_cache_timeout", 2024-02-26T04:59:38.512 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.512 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache host facts data", 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: "host_check_interval": { 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "host_check_interval", 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.513 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.514 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:38.516 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.517 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.517 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.517 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to perform a host check", 2024-02-26T04:59:38.517 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.517 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.517 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.517 INFO:teuthology.orchestra.run.smithi060.stdout: "hw_monitoring": { 2024-02-26T04:59:38.517 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hw_monitoring", 2024-02-26T04:59:38.517 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.517 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.517 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.517 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.517 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.518 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.518 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.518 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-26T04:59:38.518 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.518 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.518 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.518 INFO:teuthology.orchestra.run.smithi060.stdout: "inventory_list_all": { 2024-02-26T04:59:38.518 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "inventory_list_all", 2024-02-26T04:59:38.518 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.518 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.518 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.518 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.518 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.519 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.519 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.519 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-26T04:59:38.519 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.519 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.519 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.519 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.519 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.519 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.519 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.519 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.519 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.519 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: "log_refresh_metadata": { 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_refresh_metadata", 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.521 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.522 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.522 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.522 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.522 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.522 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.522 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.522 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.522 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.522 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-26T04:59:38.522 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.522 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.522 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.522 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.523 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.524 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.524 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.524 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.524 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.524 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.524 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.524 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.524 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.524 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.524 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.524 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.524 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.524 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.525 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-26T04:59:38.526 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.526 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.526 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.526 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-26T04:59:38.526 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-26T04:59:38.526 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.526 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.526 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.526 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "*", 2024-02-26T04:59:38.526 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.526 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.526 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.527 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-26T04:59:38.527 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.527 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.527 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.527 INFO:teuthology.orchestra.run.smithi060.stdout: "max_count_per_host": { 2024-02-26T04:59:38.527 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_count_per_host", 2024-02-26T04:59:38.527 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.527 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.527 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.527 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.527 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.527 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.527 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.528 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of daemons per service per host", 2024-02-26T04:59:38.528 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.528 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.528 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.528 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd_draining_count": { 2024-02-26T04:59:38.528 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_osd_draining_count", 2024-02-26T04:59:38.528 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.528 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.528 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.528 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.528 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.529 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.529 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.529 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-26T04:59:38.529 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.529 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.529 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.529 INFO:teuthology.orchestra.run.smithi060.stdout: "migration_current": { 2024-02-26T04:59:38.529 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "migration_current", 2024-02-26T04:59:38.529 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.529 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.529 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.529 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.529 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "internal - do not modify", 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "root", 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm-package", 2024-02-26T04:59:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: "root" 2024-02-26T04:59:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "mode for remote execution of cephadm", 2024-02-26T04:59:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: "oob_default_addr": { 2024-02-26T04:59:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "oob_default_addr", 2024-02-26T04:59:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.532 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "169.254.1.1", 2024-02-26T04:59:38.532 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.532 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.532 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.532 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-26T04:59:38.532 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.532 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.532 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.532 INFO:teuthology.orchestra.run.smithi060.stdout: "prometheus_alerts_path": { 2024-02-26T04:59:38.532 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus_alerts_path", 2024-02-26T04:59:38.532 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.532 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.532 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_insecure": { 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_insecure", 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.533 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_password": { 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_password", 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.534 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.535 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.535 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.535 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-26T04:59:38.535 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.535 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.535 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.535 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_url": { 2024-02-26T04:59:38.535 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_url", 2024-02-26T04:59:38.535 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.535 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.535 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.535 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.535 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.536 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.536 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.536 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-26T04:59:38.536 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.536 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.536 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.536 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_username": { 2024-02-26T04:59:38.536 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_username", 2024-02-26T04:59:38.536 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.536 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.536 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.536 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.536 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.537 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.537 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.537 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-26T04:59:38.537 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.537 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.537 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.537 INFO:teuthology.orchestra.run.smithi060.stdout: "secure_monitoring_stack": { 2024-02-26T04:59:38.537 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secure_monitoring_stack", 2024-02-26T04:59:38.537 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.537 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.537 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.537 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.537 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.538 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.538 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.538 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-26T04:59:38.538 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.538 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.538 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.538 INFO:teuthology.orchestra.run.smithi060.stdout: "service_discovery_port": { 2024-02-26T04:59:38.538 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "service_discovery_port", 2024-02-26T04:59:38.538 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.538 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.538 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.538 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8765", 2024-02-26T04:59:38.538 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "cephadm service discovery port", 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_config_file": { 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_config_file", 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.539 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.540 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.540 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-26T04:59:38.540 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.540 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.540 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.540 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.540 INFO:teuthology.orchestra.run.smithi060.stdout: "use_agent": { 2024-02-26T04:59:38.540 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_agent", 2024-02-26T04:59:38.540 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.540 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.540 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.541 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.541 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.541 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.541 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.541 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-26T04:59:38.541 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.541 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.541 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.541 INFO:teuthology.orchestra.run.smithi060.stdout: "use_repo_digest": { 2024-02-26T04:59:38.541 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_repo_digest", 2024-02-26T04:59:38.541 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.541 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.541 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_failed_host_check": { 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_failed_host_check", 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.542 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if the host check fails", 2024-02-26T04:59:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_daemons": { 2024-02-26T04:59:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_daemons", 2024-02-26T04:59:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.543 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.544 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.544 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.544 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.544 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-26T04:59:38.544 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.544 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.544 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.544 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_hosts": { 2024-02-26T04:59:38.544 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_hosts", 2024-02-26T04:59:38.544 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.544 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.544 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crash", 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.545 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.546 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.547 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.548 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.548 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.548 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.548 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.548 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.548 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.548 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.548 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.548 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.548 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.548 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.548 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.548 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.549 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: "retain_interval": { 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retain_interval", 2024-02-26T04:59:38.550 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.551 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.551 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.551 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "31536000", 2024-02-26T04:59:38.551 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.551 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.551 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.551 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-26T04:59:38.551 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.551 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.551 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.551 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_recent_interval": { 2024-02-26T04:59:38.551 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_recent_interval", 2024-02-26T04:59:38.551 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.552 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.552 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.552 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1209600", 2024-02-26T04:59:38.552 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.552 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.552 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.552 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-26T04:59:38.552 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.552 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.552 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.552 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.552 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.552 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dashboard", 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.553 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.554 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.554 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.554 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_HOST": { 2024-02-26T04:59:38.554 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-26T04:59:38.554 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.554 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.554 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.554 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.554 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.554 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.554 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.554 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.554 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.555 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_ENABLED": { 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_ENABLED", 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.556 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.557 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-26T04:59:38.557 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-26T04:59:38.557 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.557 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.557 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.557 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.557 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.557 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.557 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.557 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.557 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.557 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.557 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: "ENABLE_BROWSABLE_API": { 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.558 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-26T04:59:38.559 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-26T04:59:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-26T04:59:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-26T04:59:38.561 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.561 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.561 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.561 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.561 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.561 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.561 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.561 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.561 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.561 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.561 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.561 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-26T04:59:38.561 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.562 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.563 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.563 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.563 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.563 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.563 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.563 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.563 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.563 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.563 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.563 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-26T04:59:38.563 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-26T04:59:38.563 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.564 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.564 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.564 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.564 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.564 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.564 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.564 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.564 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.564 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.564 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.564 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.564 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-26T04:59:38.564 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-26T04:59:38.564 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.565 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.566 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.566 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.566 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.566 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.566 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.566 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.566 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.566 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.566 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.566 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_PASSWORD": { 2024-02-26T04:59:38.566 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-26T04:59:38.566 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.566 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.567 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.567 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T04:59:38.567 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.567 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.567 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.567 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.567 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.567 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.567 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.567 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.567 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-26T04:59:38.567 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-26T04:59:38.567 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.567 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.568 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.568 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.568 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.568 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.568 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.568 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.568 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.568 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.568 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.568 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_URL": { 2024-02-26T04:59:38.568 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_URL", 2024-02-26T04:59:38.568 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_USERNAME": { 2024-02-26T04:59:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-26T04:59:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.570 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.571 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.571 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.571 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.571 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.571 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.571 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.571 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.571 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.571 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.571 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-26T04:59:38.571 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-26T04:59:38.571 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.571 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.572 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.572 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.572 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.572 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.572 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.572 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.572 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.572 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.572 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.572 INFO:teuthology.orchestra.run.smithi060.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-26T04:59:38.572 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-26T04:59:38.572 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.572 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.573 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.574 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.574 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.574 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.574 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.574 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.574 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.574 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.574 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.574 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_HOST": { 2024-02-26T04:59:38.574 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-26T04:59:38.574 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.574 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.574 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.574 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.575 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.576 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.576 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.576 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.576 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.576 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.576 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.576 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-26T04:59:38.576 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-26T04:59:38.576 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.576 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.576 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.576 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.576 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.577 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.578 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.579 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.579 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.579 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.579 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.579 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.579 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-26T04:59:38.579 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-26T04:59:38.579 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.579 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.579 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.579 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.579 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.579 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.580 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.580 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.580 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.580 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.580 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.580 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-26T04:59:38.580 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-26T04:59:38.580 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.580 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.580 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.580 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.580 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.580 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.581 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.581 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.581 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.581 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.581 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.581 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-26T04:59:38.581 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-26T04:59:38.581 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.581 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.581 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.581 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.581 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.581 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.582 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_ENABLED": { 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.583 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.584 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.584 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.584 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.584 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-26T04:59:38.584 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-26T04:59:38.584 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.584 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.584 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.584 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-26T04:59:38.584 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.584 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.584 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.584 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.585 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.586 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.586 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.586 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.586 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-26T04:59:38.586 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-26T04:59:38.586 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.586 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.586 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.586 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8", 2024-02-26T04:59:38.586 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.586 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.586 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.586 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.586 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "45", 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.587 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.588 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.588 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ACCESS_KEY": { 2024-02-26T04:59:38.588 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-26T04:59:38.588 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.588 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.588 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.588 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.588 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.588 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.588 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.588 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.588 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.589 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.589 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.589 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-26T04:59:38.589 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-26T04:59:38.589 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.589 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.589 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.589 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T04:59:38.589 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.589 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.589 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.589 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.589 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.590 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.590 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.590 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SECRET_KEY": { 2024-02-26T04:59:38.590 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-26T04:59:38.590 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.590 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.590 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.590 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.590 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.590 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.590 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.590 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.590 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SSL_VERIFY": { 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.592 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-26T04:59:38.593 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-26T04:59:38.594 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.594 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.594 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.594 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T04:59:38.594 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.594 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.594 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.594 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.594 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.594 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.594 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.594 INFO:teuthology.orchestra.run.smithi060.stdout: "cross_origin_url": { 2024-02-26T04:59:38.594 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cross_origin_url", 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: "crt_file": { 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crt_file", 2024-02-26T04:59:38.595 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.596 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.596 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.596 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.596 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.596 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.596 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.596 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.596 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.596 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.596 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.596 INFO:teuthology.orchestra.run.smithi060.stdout: "debug": { 2024-02-26T04:59:38.596 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "debug", 2024-02-26T04:59:38.596 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.597 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.597 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.597 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.597 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.597 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.597 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.597 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable/disable debug options", 2024-02-26T04:59:38.597 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.597 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.597 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.597 INFO:teuthology.orchestra.run.smithi060.stdout: "jwt_token_ttl": { 2024-02-26T04:59:38.597 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "jwt_token_ttl", 2024-02-26T04:59:38.597 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "28800", 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.599 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.599 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.599 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.599 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.599 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.599 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.599 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.599 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.599 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.599 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.599 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.599 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.599 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.600 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.600 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.600 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.600 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.600 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.600 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.600 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.600 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.600 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.600 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.600 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.600 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.600 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.600 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.601 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.601 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.601 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.601 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.601 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.601 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.601 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.601 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.601 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.601 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.601 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.601 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.601 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.602 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.603 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.603 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.603 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.603 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.603 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.603 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.603 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.603 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.603 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.603 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.603 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.603 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.604 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.604 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.604 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.604 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.604 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.604 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.604 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.604 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.604 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.604 INFO:teuthology.orchestra.run.smithi060.stdout: "motd": { 2024-02-26T04:59:38.604 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "motd", 2024-02-26T04:59:38.604 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.604 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.605 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.605 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.605 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.605 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.605 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.605 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "The message of the day", 2024-02-26T04:59:38.605 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.605 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.605 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.605 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect_resolve_ip_addr": { 2024-02-26T04:59:38.605 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "redirect_resolve_ip_addr", 2024-02-26T04:59:38.605 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.605 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.606 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.607 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8080", 2024-02-26T04:59:38.608 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.608 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.608 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.608 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.608 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.608 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.608 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.608 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-02-26T04:59:38.608 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-02-26T04:59:38.608 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.608 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.608 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.608 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl_server_port": { 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl_server_port", 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8443", 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.610 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.610 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.610 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.610 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.610 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.610 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.610 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-02-26T04:59:38.610 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-02-26T04:59:38.610 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.610 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.610 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.610 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "redirect", 2024-02-26T04:59:38.610 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.610 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect" 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.611 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: "url_prefix": { 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url_prefix", 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.612 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_monitoring": { 2024-02-26T04:59:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_monitoring", 2024-02-26T04:59:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "monitor device health metrics", 2024-02-26T04:59:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.616 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.616 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.616 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.616 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.616 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.616 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.616 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.616 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.616 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.616 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.616 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.616 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.616 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.617 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.617 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.617 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.617 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.617 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.617 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.617 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.617 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.617 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.617 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.617 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.617 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.617 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.618 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: "mark_out_threshold": { 2024-02-26T04:59:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mark_out_threshold", 2024-02-26T04:59:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2419200", 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": { 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_name", 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.620 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.621 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "device_health_metrics", 2024-02-26T04:59:38.621 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.621 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.621 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.621 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-26T04:59:38.621 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.621 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.621 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.621 INFO:teuthology.orchestra.run.smithi060.stdout: "retention_period": { 2024-02-26T04:59:38.621 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retention_period", 2024-02-26T04:59:38.621 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.621 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.621 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15552000", 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain device health metrics", 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_frequency": { 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_frequency", 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.622 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: "self_heal": { 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "self_heal", 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.623 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.624 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.624 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.624 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-26T04:59:38.624 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.624 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.624 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.624 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:38.624 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:38.624 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.624 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.624 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.624 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:38.624 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and check device health", 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_threshold": { 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_threshold", 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7257600", 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.625 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.626 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.626 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-26T04:59:38.626 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.626 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.626 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.626 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.626 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.626 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "diskprediction_local", 2024-02-26T04:59:38.626 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.626 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.626 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.626 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.627 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.628 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.628 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.628 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.628 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.628 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.628 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.628 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.628 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.628 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.628 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.628 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.628 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.628 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.629 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.630 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.630 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.630 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.630 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.630 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.630 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.630 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.630 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.630 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.630 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.630 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.630 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.630 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: "predict_interval": { 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predict_interval", 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.631 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.632 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.632 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-02-26T04:59:38.632 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.632 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.632 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.632 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.632 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.632 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.632 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.632 INFO:teuthology.orchestra.run.smithi060.stdout: "predictor_model": { 2024-02-26T04:59:38.632 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predictor_model", 2024-02-26T04:59:38.632 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.633 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.633 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.633 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "prophetstor", 2024-02-26T04:59:38.633 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.633 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.633 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.633 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.633 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.633 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.633 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.633 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:38.633 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:38.633 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.634 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "influx", 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": false, 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "influxdb python module not found", 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "batch_size": { 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "batch_size", 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5000", 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.635 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.636 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.636 INFO:teuthology.orchestra.run.smithi060.stdout: "database": { 2024-02-26T04:59:38.636 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "database", 2024-02-26T04:59:38.636 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.636 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.636 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.636 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "ceph", 2024-02-26T04:59:38.636 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.636 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.636 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.636 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-26T04:59:38.636 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.636 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": { 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hostname", 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server hostname", 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.637 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.638 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:38.638 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:38.638 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.638 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.638 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.638 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "30", 2024-02-26T04:59:38.638 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "5", 2024-02-26T04:59:38.638 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.638 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.638 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-26T04:59:38.638 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.638 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.638 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.639 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.639 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.639 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.639 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.639 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.639 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.639 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.639 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.639 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.639 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.639 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.639 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.639 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.640 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.641 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.641 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.641 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.641 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.641 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.641 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.641 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.641 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.641 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.641 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.641 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.641 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.643 INFO:teuthology.orchestra.run.smithi060.stdout: "password": { 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "password", 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "password of InfluxDB server user", 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: "port": { 2024-02-26T04:59:38.644 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "port", 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8086", 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server port", 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.645 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "false", 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: "threads": { 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threads", 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.646 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "32", 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "username": { 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "username", 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.647 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.648 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.648 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.648 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "username of InfluxDB server user", 2024-02-26T04:59:38.648 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.648 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.648 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.648 INFO:teuthology.orchestra.run.smithi060.stdout: "verify_ssl": { 2024-02-26T04:59:38.648 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "verify_ssl", 2024-02-26T04:59:38.648 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.648 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.648 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.648 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "true", 2024-02-26T04:59:38.648 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "insights", 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.649 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.650 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.651 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.652 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.653 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "iostat", 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.654 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.655 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.656 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.656 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.656 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.656 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.656 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.656 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.656 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.656 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.656 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.656 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.656 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.656 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.656 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.657 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.657 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.657 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.657 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.657 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.657 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.657 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.657 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.657 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.657 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.657 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.657 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.657 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.658 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.659 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.659 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.659 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.659 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.659 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.659 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.659 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.659 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.659 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.659 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.659 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.659 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "k8sevents", 2024-02-26T04:59:38.660 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.660 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.660 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.660 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_event_retention_days": { 2024-02-26T04:59:38.660 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ceph_event_retention_days", 2024-02-26T04:59:38.660 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.660 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.660 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.660 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7", 2024-02-26T04:59:38.660 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.660 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.660 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.660 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "config_check_secs": { 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_check_secs", 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "10", 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.661 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.662 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.663 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.664 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.665 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.665 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.665 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.665 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.665 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.665 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.665 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.665 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.665 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.665 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.665 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.665 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.665 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.666 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.666 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.666 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.666 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.666 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.666 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.666 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.666 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.666 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.666 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.666 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.666 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.667 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "localpool", 2024-02-26T04:59:38.667 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.667 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.667 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.667 INFO:teuthology.orchestra.run.smithi060.stdout: "failure_domain": { 2024-02-26T04:59:38.667 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "failure_domain", 2024-02-26T04:59:38.667 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.667 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.667 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.667 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "host", 2024-02-26T04:59:38.667 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.667 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.667 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.667 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "failure domain for any created local pool", 2024-02-26T04:59:38.668 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-26T04:59:38.668 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.668 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.668 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.668 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.668 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.668 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.668 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.668 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.668 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.668 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.668 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.668 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.669 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.670 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.670 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.670 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.670 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.670 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.670 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.670 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.670 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.670 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.670 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.670 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.670 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.671 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.671 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.671 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.671 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.671 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.671 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.671 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.671 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.671 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.671 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.671 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.671 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.671 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.672 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.673 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.673 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.673 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.673 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": { 2024-02-26T04:59:38.673 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_size", 2024-02-26T04:59:38.673 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.673 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.673 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.673 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.673 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.673 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.673 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.673 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default min_size for any created local pool", 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: "num_rep": { 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "num_rep", 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3", 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default replica count for any created local pool", 2024-02-26T04:59:38.674 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": { 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_num", 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "128", 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default pg_num for any created local pool", 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.675 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.676 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.676 INFO:teuthology.orchestra.run.smithi060.stdout: "prefix": { 2024-02-26T04:59:38.676 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prefix", 2024-02-26T04:59:38.676 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.676 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.676 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.676 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.676 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.676 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.676 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.676 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name prefix for any created local pool", 2024-02-26T04:59:38.676 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.676 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: "subtree": { 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "subtree", 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "rack", 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.677 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mds_autoscaler", 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.680 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.680 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.680 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.680 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.680 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.680 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.680 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.680 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.680 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.680 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.681 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.681 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.681 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.681 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.681 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.681 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.681 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.681 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.681 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.681 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.681 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.681 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.682 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.683 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.683 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.683 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.683 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.683 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.683 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.683 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.683 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.683 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.683 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.683 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.683 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.684 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirroring", 2024-02-26T04:59:38.684 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.684 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.684 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.684 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.684 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.684 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.684 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.684 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.684 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.684 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.684 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.684 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.685 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.686 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.687 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.687 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.687 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.687 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.687 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.687 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.687 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.687 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.687 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.687 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.687 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.687 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.687 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.687 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.688 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.688 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.688 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.688 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.688 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.688 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.688 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.688 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.688 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.688 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.688 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.688 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.688 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "nfs", 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.689 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.690 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.690 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.690 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.690 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.690 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.690 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.690 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.690 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.690 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.690 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.690 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.690 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.690 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.690 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.691 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.692 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.692 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.692 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.692 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.692 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.692 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.692 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.692 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.692 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.692 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.692 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.692 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.692 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.693 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.694 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.694 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.694 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.694 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.694 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.694 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.694 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.694 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.694 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-02-26T04:59:38.694 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.694 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "fail_fs": { 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "fail_fs", 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.695 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.696 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.696 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.696 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.696 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.696 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.696 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.696 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.696 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.696 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.696 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.696 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.696 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.696 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.696 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.697 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.698 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.698 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.698 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.698 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.698 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.698 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.698 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.698 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.698 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.698 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.698 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.698 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.698 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.699 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.700 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.700 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.700 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.700 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.700 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.700 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.700 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.700 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.700 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.700 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.700 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator": { 2024-02-26T04:59:38.700 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm", 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: "rook", 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: "test_orchestrator" 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Orchestrator backend", 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.701 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_perf_query", 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.702 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.703 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.705 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.706 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.707 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.707 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_support", 2024-02-26T04:59:38.707 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.707 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.707 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.707 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.707 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.707 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.707 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.707 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.707 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.707 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.708 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.708 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.708 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.708 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.708 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.708 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.708 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.708 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.708 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.708 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.708 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.708 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.710 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.710 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.710 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.710 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.710 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.710 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.710 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.710 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.710 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.710 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.710 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.710 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.710 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.710 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.711 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.712 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.712 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.712 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.712 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.712 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.712 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.712 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.712 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_autoscaler", 2024-02-26T04:59:38.712 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.712 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.712 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.712 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.713 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.714 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.714 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.714 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.714 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.714 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.714 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.714 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.714 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.714 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.714 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.714 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.714 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.714 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.715 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.716 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.716 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.716 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.716 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.716 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.716 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.716 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.716 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.716 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.716 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.716 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.716 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.716 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: "threshold": { 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threshold", 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.718 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-02-26T04:59:38.719 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1.0", 2024-02-26T04:59:38.719 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.719 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.719 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "scaling threshold", 2024-02-26T04:59:38.719 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-26T04:59:38.719 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.719 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.719 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.719 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.719 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "progress", 2024-02-26T04:59:38.719 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.719 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.720 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.720 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_pg_recovery_event": { 2024-02-26T04:59:38.720 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_pg_recovery_event", 2024-02-26T04:59:38.720 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.720 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.720 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.720 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.720 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.720 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.720 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.720 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-26T04:59:38.720 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.720 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.721 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.722 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.722 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.722 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.722 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.722 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.722 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.722 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.722 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.722 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.722 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.722 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.722 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.722 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.723 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.723 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.723 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.723 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.723 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.723 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.723 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.723 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.723 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.723 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.723 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.723 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.723 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.724 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.725 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.725 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.725 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.725 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.725 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.725 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.725 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.725 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.725 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.725 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.725 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.725 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.725 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "max_completed_events": { 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_completed_events", 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "50", 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "number of past completed events to remember", 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T04:59:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long the module is going to sleep", 2024-02-26T04:59:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus", 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "cache": { 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cache", 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "exclude_perf_counters": { 2024-02-26T04:59:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "exclude_perf_counters", 2024-02-26T04:59:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.731 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-26T04:59:38.731 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-26T04:59:38.731 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.731 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.731 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.731 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.731 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.731 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.731 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.731 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.731 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.731 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.731 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.732 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.732 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.732 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.732 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.732 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.732 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.732 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.732 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.732 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.732 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.732 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.732 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.732 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.733 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.733 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.733 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.733 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.733 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.733 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.733 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.733 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.733 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.733 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.733 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.733 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.733 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.734 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.735 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.736 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.736 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.736 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools": { 2024-02-26T04:59:38.736 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools", 2024-02-26T04:59:38.736 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.736 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.736 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.737 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.737 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.737 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.737 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.737 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.737 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.737 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.738 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.738 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-26T04:59:38.738 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-26T04:59:38.738 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.738 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.738 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.738 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "300", 2024-02-26T04:59:38.738 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.738 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.738 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.738 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.738 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.739 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.739 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.739 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_interval": { 2024-02-26T04:59:38.739 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_interval", 2024-02-26T04:59:38.739 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.739 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.739 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.739 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15.0", 2024-02-26T04:59:38.739 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.739 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.739 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.739 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.739 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T04:59:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T04:59:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-02-26T04:59:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-26T04:59:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T04:59:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T04:59:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "9283", 2024-02-26T04:59:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-26T04:59:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: "stale_cache_strategy": { 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stale_cache_strategy", 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "log", 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.742 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "default", 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: "default", 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: "error" 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.743 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.744 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.744 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.744 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.744 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-02-26T04:59:38.744 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-02-26T04:59:38.744 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.744 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.744 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.744 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-02-26T04:59:38.744 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-02-26T04:59:38.744 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-02-26T04:59:38.744 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.745 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.747 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_snap_create": { 2024-02-26T04:59:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_snap_create", 2024-02-26T04:59:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mirror_snapshot_schedule": { 2024-02-26T04:59:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirror_snapshot_schedule", 2024-02-26T04:59:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.752 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.752 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.752 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.752 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.752 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.752 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.752 INFO:teuthology.orchestra.run.smithi060.stdout: "trash_purge_schedule": { 2024-02-26T04:59:38.752 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "trash_purge_schedule", 2024-02-26T04:59:38.752 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.752 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.752 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.752 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.752 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "restful", 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.753 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_auth": { 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_auth", 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-02-26T04:59:38.754 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.755 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.756 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.756 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.756 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.756 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.756 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.756 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.756 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.756 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.756 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.756 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.756 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.756 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.756 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.756 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.757 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.758 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.759 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.760 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.760 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.760 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.760 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.760 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.760 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.760 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.760 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T04:59:38.760 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T04:59:38.760 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.760 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.760 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.760 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.761 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.761 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.761 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.761 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.761 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.761 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.761 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.761 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T04:59:38.761 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T04:59:38.761 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.761 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.761 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.761 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rgw", 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.762 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.763 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.763 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.763 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.763 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.763 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.763 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.763 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.763 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.763 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.763 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.763 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.763 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.763 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.763 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.765 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.765 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.765 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.765 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.765 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.765 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.765 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.765 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.765 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.765 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.765 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.765 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.765 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.766 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.767 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.767 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.767 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.767 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.767 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.767 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.767 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.767 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.767 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.767 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.767 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.767 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.767 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.767 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.768 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rook", 2024-02-26T04:59:38.768 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.768 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.768 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.768 INFO:teuthology.orchestra.run.smithi060.stdout: "drive_group_interval": { 2024-02-26T04:59:38.768 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "drive_group_interval", 2024-02-26T04:59:38.768 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.768 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.768 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.768 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "300.0", 2024-02-26T04:59:38.768 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.768 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.769 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.769 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-26T04:59:38.769 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.769 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.769 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.769 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.769 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.769 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.769 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.769 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.769 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.769 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.769 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.770 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.770 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.770 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.770 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.770 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.770 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.770 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.770 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.770 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.770 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.770 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.770 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.770 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.771 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.772 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.772 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.772 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.772 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.772 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.772 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.772 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.772 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.772 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.772 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.772 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.772 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.772 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.773 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.774 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.774 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.774 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.774 INFO:teuthology.orchestra.run.smithi060.stdout: "storage_class": { 2024-02-26T04:59:38.774 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "storage_class", 2024-02-26T04:59:38.774 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.774 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.774 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.774 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "local", 2024-02-26T04:59:38.774 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.774 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.774 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.774 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "selftest", 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.776 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.776 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.776 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.776 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.776 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.776 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.776 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.776 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.776 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.776 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.776 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.776 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.776 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.777 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.778 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.779 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: "roption1": { 2024-02-26T04:59:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption1", 2024-02-26T04:59:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.780 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: "roption2": { 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption2", 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "xyz", 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.781 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption1": { 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption1", 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption2": { 2024-02-26T04:59:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption2", 2024-02-26T04:59:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption3": { 2024-02-26T04:59:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption3", 2024-02-26T04:59:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption4": { 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption4", 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.785 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption5": { 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption5", 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.786 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption6": { 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption6", 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.787 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption7": { 2024-02-26T04:59:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption7", 2024-02-26T04:59:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T04:59:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "42", 2024-02-26T04:59:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "testkey": { 2024-02-26T04:59:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testkey", 2024-02-26T04:59:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "testlkey": { 2024-02-26T04:59:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testlkey", 2024-02-26T04:59:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "testnewline": { 2024-02-26T04:59:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testnewline", 2024-02-26T04:59:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snap_schedule", 2024-02-26T04:59:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_m_granularity": { 2024-02-26T04:59:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_m_granularity", 2024-02-26T04:59:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow minute scheduled snapshots", 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "dump_on_update": { 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dump_on_update", 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "dump database to debug log on update", 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stats", 2024-02-26T04:59:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "status", 2024-02-26T04:59:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telegraf", 2024-02-26T04:59:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "address": { 2024-02-26T04:59:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "address", 2024-02-26T04:59:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-26T04:59:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15", 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telemetry", 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_basic": { 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_basic", 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-26T04:59:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_crash": { 2024-02-26T04:59:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_crash", 2024-02-26T04:59:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_device": { 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_device", 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-26T04:59:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_ident": { 2024-02-26T04:59:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_ident", 2024-02-26T04:59:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-26T04:59:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_perf": { 2024-02-26T04:59:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_perf", 2024-02-26T04:59:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "contact": { 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "contact", 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "description": { 2024-02-26T04:59:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "description", 2024-02-26T04:59:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "device_url": { 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_url", 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "24", 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "8", 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "last_opt_revision": { 2024-02-26T04:59:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "last_opt_revision", 2024-02-26T04:59:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1", 2024-02-26T04:59:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard": { 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard", 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard_description": { 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard_description", 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "organization": { 2024-02-26T04:59:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "organization", 2024-02-26T04:59:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "proxy": { 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "proxy", 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "url": { 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url", 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "test_orchestrator", 2024-02-26T04:59:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_clones": { 2024-02-26T04:59:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_clones", 2024-02-26T04:59:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4", 2024-02-26T04:59:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-26T04:59:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "periodic_async_work": { 2024-02-26T04:59:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "periodic_async_work", 2024-02-26T04:59:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Periodically check for async work", 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_delay": { 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_delay", 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-26T04:59:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix", 2024-02-26T04:59:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "discovery_interval": { 2024-02-26T04:59:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "discovery_interval", 2024-02-26T04:59:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "100", 2024-02-26T04:59:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "identifier": { 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "identifier", 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_host": { 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_host", 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_port": { 2024-02-26T04:59:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_port", 2024-02-26T04:59:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10051", 2024-02-26T04:59:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_sender": { 2024-02-26T04:59:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_sender", 2024-02-26T04:59:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "modules": [ 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm", 2024-02-26T04:59:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "dashboard", 2024-02-26T04:59:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "iostat", 2024-02-26T04:59:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "nfs", 2024-02-26T04:59:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "restful" 2024-02-26T04:59:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "available_modules": [ 2024-02-26T04:59:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "alerts", 2024-02-26T04:59:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How frequently to reexamine health status", 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_destination": { 2024-02-26T04:59:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_destination", 2024-02-26T04:59:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email address to send alerts to", 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_from_name": { 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_from_name", 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "Ceph", 2024-02-26T04:59:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email From: name", 2024-02-26T04:59:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_host": { 2024-02-26T04:59:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_host", 2024-02-26T04:59:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP server", 2024-02-26T04:59:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_password": { 2024-02-26T04:59:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_password", 2024-02-26T04:59:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Password to authenticate with", 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_port": { 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_port", 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "465", 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP port", 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_sender": { 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_sender", 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP envelope sender", 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_ssl": { 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_ssl", 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-26T04:59:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_user": { 2024-02-26T04:59:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_user", 2024-02-26T04:59:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "User to authenticate as", 2024-02-26T04:59:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "balancer", 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "active": { 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "active", 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically balance PGs across cluster", 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_time": { 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_time", 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0000", 2024-02-26T04:59:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "beginning time of day to automatically balance", 2024-02-26T04:59:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T04:59:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_weekday": { 2024-02-26T04:59:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_weekday", 2024-02-26T04:59:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-02-26T04:59:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-02-26T04:59:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-26T04:59:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T04:59:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_max_iterations": { 2024-02-26T04:59:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_max_iterations", 2024-02-26T04:59:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "25", 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "250", 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_metrics": { 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_metrics", 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "pgs,objects,bytes", 2024-02-26T04:59:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-26T04:59:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-26T04:59:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_step": { 2024-02-26T04:59:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_step", 2024-02-26T04:59:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.5", 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0.001", 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "0.999", 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "aggressiveness of optimization", 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "end_time": { 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_time", 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2359", 2024-02-26T04:59:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.884 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.884 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.884 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ending time of day to automatically balance", 2024-02-26T04:59:38.884 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-26T04:59:38.884 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.884 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.884 INFO:teuthology.orchestra.run.smithi060.stdout: "end_weekday": { 2024-02-26T04:59:38.884 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_weekday", 2024-02-26T04:59:38.884 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:38.884 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.884 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.884 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.885 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.886 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.886 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.886 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.886 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.886 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.886 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.886 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.886 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.886 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.886 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.886 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.886 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.886 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.887 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.887 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.887 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.887 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.887 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.887 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.887 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.887 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.887 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.887 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.887 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.887 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.887 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.888 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.888 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.888 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.888 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.888 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.888 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.888 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.888 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.888 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.888 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.888 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.888 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.888 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.888 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.889 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.890 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.890 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.890 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.890 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.890 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.890 INFO:teuthology.orchestra.run.smithi060.stdout: "min_score": { 2024-02-26T04:59:38.890 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_score", 2024-02-26T04:59:38.890 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.890 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.890 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.890 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:38.890 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.890 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.891 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.891 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-26T04:59:38.891 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.891 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.891 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.891 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-02-26T04:59:38.891 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-02-26T04:59:38.891 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.891 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.891 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.891 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "upmap", 2024-02-26T04:59:38.891 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.891 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-compat", 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: "none", 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: "read", 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap", 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap-read" 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Balancer mode", 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_ids": { 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_ids", 2024-02-26T04:59:38.892 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.893 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_deviation": { 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_deviation", 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.894 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_optimizations": { 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_optimizations", 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.895 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.896 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.896 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.896 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.896 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-26T04:59:38.896 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.896 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.896 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.896 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.896 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.896 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm", 2024-02-26T04:59:38.896 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.896 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_down_multiplier": { 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_down_multiplier", 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_refresh_rate": { 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_refresh_rate", 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "20", 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_starting_port": { 2024-02-26T04:59:38.899 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_starting_port", 2024-02-26T04:59:38.899 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.899 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.899 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.899 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4721", 2024-02-26T04:59:38.899 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.899 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.899 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.899 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-26T04:59:38.899 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.899 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.899 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.900 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_ptrace": { 2024-02-26T04:59:38.900 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_ptrace", 2024-02-26T04:59:38.900 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.900 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.900 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.900 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.900 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.900 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.900 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.900 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-26T04:59:38.900 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-26T04:59:38.900 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.900 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_interval": { 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_interval", 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.901 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_memory_target_ratio": { 2024-02-26T04:59:38.902 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_memory_target_ratio", 2024-02-26T04:59:38.902 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:38.902 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.902 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.902 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.7", 2024-02-26T04:59:38.902 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.902 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.902 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.902 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-26T04:59:38.902 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.902 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.902 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.902 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm_log_destination": { 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm_log_destination", 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: "file", 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: "file,syslog", 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: "syslog" 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.904 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.904 INFO:teuthology.orchestra.run.smithi060.stdout: "cgroups_split": { 2024-02-26T04:59:38.904 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cgroups_split", 2024-02-26T04:59:38.904 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.904 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.904 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.904 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.904 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.904 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.904 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.904 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-26T04:59:38.904 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: "config_checks_enabled": { 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_checks_enabled", 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.905 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: "config_dashboard": { 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_dashboard", 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.906 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_alertmanager": { 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_alertmanager", 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.908 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_base": { 2024-02-26T04:59:38.908 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_base", 2024-02-26T04:59:38.908 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.908 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.908 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.908 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-26T04:59:38.908 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.908 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.908 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.908 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Container image name, without the tag", 2024-02-26T04:59:38.908 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.908 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.908 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_elasticsearch": { 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_elasticsearch", 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "elasticsearch container image", 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_grafana": { 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_grafana", 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.910 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_haproxy": { 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_haproxy", 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "HAproxy container image", 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_agent": { 2024-02-26T04:59:38.911 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_agent", 2024-02-26T04:59:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-26T04:59:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger agent container image", 2024-02-26T04:59:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_collector": { 2024-02-26T04:59:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_collector", 2024-02-26T04:59:38.913 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.913 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.913 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.913 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-26T04:59:38.913 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.913 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.913 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.913 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger collector container image", 2024-02-26T04:59:38.913 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.913 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.913 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.913 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_query": { 2024-02-26T04:59:38.913 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_query", 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger query container image", 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_keepalived": { 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_keepalived", 2024-02-26T04:59:38.914 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.915 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.915 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.915 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-26T04:59:38.915 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.915 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.915 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.915 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Keepalived container image", 2024-02-26T04:59:38.915 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.915 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.915 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.915 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_loki": { 2024-02-26T04:59:38.915 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_loki", 2024-02-26T04:59:38.915 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.916 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.916 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.916 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-26T04:59:38.916 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.916 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.916 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.916 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Loki container image", 2024-02-26T04:59:38.916 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.916 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.916 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.916 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_node_exporter": { 2024-02-26T04:59:38.916 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_node_exporter", 2024-02-26T04:59:38.917 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.917 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.917 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.917 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-26T04:59:38.917 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.917 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.917 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.917 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T04:59:38.917 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.917 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.917 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.917 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nvmeof": { 2024-02-26T04:59:38.918 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nvmeof", 2024-02-26T04:59:38.918 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.918 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.918 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.918 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-26T04:59:38.918 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.918 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.918 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.918 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nvme-of container image", 2024-02-26T04:59:38.918 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.919 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.919 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.919 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_prometheus": { 2024-02-26T04:59:38.919 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_prometheus", 2024-02-26T04:59:38.919 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.919 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.920 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.920 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-26T04:59:38.920 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.920 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.920 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.920 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-02-26T04:59:38.920 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.920 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.920 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.920 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_promtail": { 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_promtail", 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Promtail container image", 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_snmp_gateway": { 2024-02-26T04:59:38.921 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_snmp_gateway", 2024-02-26T04:59:38.922 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.922 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.922 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.922 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-26T04:59:38.922 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.922 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.922 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.922 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SNMP Gateway container image", 2024-02-26T04:59:38.922 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.922 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.922 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.922 INFO:teuthology.orchestra.run.smithi060.stdout: "container_init": { 2024-02-26T04:59:38.922 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_init", 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Run podman/docker with `--init`", 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: "daemon_cache_timeout": { 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "daemon_cache_timeout", 2024-02-26T04:59:38.923 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.924 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.924 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.924 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:38.924 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.924 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.924 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.924 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-26T04:59:38.924 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.924 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.924 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.924 INFO:teuthology.orchestra.run.smithi060.stdout: "default_cephadm_command_timeout": { 2024-02-26T04:59:38.924 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_cephadm_command_timeout", 2024-02-26T04:59:38.924 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "900", 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: "default_registry": { 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_registry", 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.925 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io", 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: "device_cache_timeout": { 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_cache_timeout", 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.926 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1800", 2024-02-26T04:59:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache device inventory", 2024-02-26T04:59:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: "device_enhanced_scan": { 2024-02-26T04:59:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_enhanced_scan", 2024-02-26T04:59:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.928 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.928 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.928 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.928 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.928 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-26T04:59:38.928 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.928 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.928 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.928 INFO:teuthology.orchestra.run.smithi060.stdout: "facts_cache_timeout": { 2024-02-26T04:59:38.928 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "facts_cache_timeout", 2024-02-26T04:59:38.928 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.928 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.928 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.929 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:38.929 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.929 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.929 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.929 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache host facts data", 2024-02-26T04:59:38.929 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.929 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.929 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.929 INFO:teuthology.orchestra.run.smithi060.stdout: "host_check_interval": { 2024-02-26T04:59:38.929 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "host_check_interval", 2024-02-26T04:59:38.929 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.929 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.929 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to perform a host check", 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: "hw_monitoring": { 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hw_monitoring", 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.930 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.931 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.931 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.931 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.931 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-26T04:59:38.931 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.931 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.931 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.931 INFO:teuthology.orchestra.run.smithi060.stdout: "inventory_list_all": { 2024-02-26T04:59:38.931 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "inventory_list_all", 2024-02-26T04:59:38.931 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.931 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.931 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.931 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.932 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.932 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.932 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.932 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-26T04:59:38.932 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.932 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.932 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.932 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.932 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.932 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.932 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.932 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.933 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.933 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.933 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.933 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.933 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.933 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.933 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.933 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.933 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.933 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.933 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.933 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.933 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.933 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: "log_refresh_metadata": { 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_refresh_metadata", 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.934 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.935 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.935 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.935 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.935 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.935 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.935 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.935 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.935 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.935 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.935 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-26T04:59:38.935 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.935 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.935 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.936 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.936 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.936 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.936 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.936 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.936 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.936 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.936 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.936 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.936 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.936 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.936 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.937 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.938 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.939 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.939 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.939 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-26T04:59:38.939 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.939 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.939 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.939 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-26T04:59:38.939 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-26T04:59:38.939 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.939 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.939 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.939 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "*", 2024-02-26T04:59:38.939 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.940 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.940 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.940 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-26T04:59:38.940 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.940 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.940 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.940 INFO:teuthology.orchestra.run.smithi060.stdout: "max_count_per_host": { 2024-02-26T04:59:38.940 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_count_per_host", 2024-02-26T04:59:38.940 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.940 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.940 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.940 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.940 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.941 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.941 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.941 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of daemons per service per host", 2024-02-26T04:59:38.941 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.941 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.941 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.941 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd_draining_count": { 2024-02-26T04:59:38.941 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_osd_draining_count", 2024-02-26T04:59:38.941 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.941 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.941 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.941 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.941 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "migration_current": { 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "migration_current", 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.942 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.943 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "internal - do not modify", 2024-02-26T04:59:38.943 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.943 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.943 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.943 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-02-26T04:59:38.943 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-02-26T04:59:38.943 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.943 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.943 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.943 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "root", 2024-02-26T04:59:38.943 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.943 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.943 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.944 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm-package", 2024-02-26T04:59:38.944 INFO:teuthology.orchestra.run.smithi060.stdout: "root" 2024-02-26T04:59:38.944 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.944 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "mode for remote execution of cephadm", 2024-02-26T04:59:38.944 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.944 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.944 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.944 INFO:teuthology.orchestra.run.smithi060.stdout: "oob_default_addr": { 2024-02-26T04:59:38.944 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "oob_default_addr", 2024-02-26T04:59:38.944 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.944 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.944 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.944 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "169.254.1.1", 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: "prometheus_alerts_path": { 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus_alerts_path", 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-26T04:59:38.945 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.946 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.946 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.946 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-26T04:59:38.946 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.946 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.946 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.946 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_insecure": { 2024-02-26T04:59:38.946 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_insecure", 2024-02-26T04:59:38.946 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.946 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.946 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.946 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.946 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_password": { 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_password", 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.947 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.948 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.948 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-26T04:59:38.948 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.948 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.948 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.948 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_url": { 2024-02-26T04:59:38.948 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_url", 2024-02-26T04:59:38.948 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.948 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.948 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.948 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.948 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.948 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.949 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.949 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-26T04:59:38.949 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.949 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.949 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.949 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_username": { 2024-02-26T04:59:38.949 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_username", 2024-02-26T04:59:38.949 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.949 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.949 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.949 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.949 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.949 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: "secure_monitoring_stack": { 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secure_monitoring_stack", 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.950 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.951 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-26T04:59:38.951 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.951 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.951 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.951 INFO:teuthology.orchestra.run.smithi060.stdout: "service_discovery_port": { 2024-02-26T04:59:38.951 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "service_discovery_port", 2024-02-26T04:59:38.951 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.951 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.951 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.951 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8765", 2024-02-26T04:59:38.951 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.951 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.951 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.952 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "cephadm service discovery port", 2024-02-26T04:59:38.952 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.952 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.952 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.952 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_config_file": { 2024-02-26T04:59:38.952 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_config_file", 2024-02-26T04:59:38.952 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.952 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.952 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.952 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.952 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.952 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.952 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: "use_agent": { 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_agent", 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.953 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: "use_repo_digest": { 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_repo_digest", 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-26T04:59:38.954 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_failed_host_check": { 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_failed_host_check", 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if the host check fails", 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.955 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_daemons": { 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_daemons", 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.957 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_hosts": { 2024-02-26T04:59:38.957 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_hosts", 2024-02-26T04:59:38.957 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.957 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.957 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.957 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.957 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.957 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.957 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.957 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-26T04:59:38.957 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.957 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.957 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.957 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.958 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crash", 2024-02-26T04:59:38.958 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.958 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.958 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.958 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:38.958 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:38.958 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.958 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.958 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.958 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.958 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.959 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:38.960 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:38.960 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.960 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.960 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.960 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.960 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.960 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.960 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.960 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.960 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.960 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.960 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:38.961 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:38.962 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:38.962 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.962 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.962 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.962 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.962 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:38.962 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:38.962 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.962 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.962 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.962 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.962 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.962 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: "retain_interval": { 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retain_interval", 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "31536000", 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.963 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-26T04:59:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_recent_interval": { 2024-02-26T04:59:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_recent_interval", 2024-02-26T04:59:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1209600", 2024-02-26T04:59:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dashboard", 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-26T04:59:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_HOST": { 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.966 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.967 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_ENABLED": { 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_ENABLED", 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.968 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.969 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.970 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.970 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.970 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.970 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.970 INFO:teuthology.orchestra.run.smithi060.stdout: "ENABLE_BROWSABLE_API": { 2024-02-26T04:59:38.970 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-26T04:59:38.970 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.970 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.970 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.970 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.970 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.970 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.970 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.971 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.971 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.971 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.971 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.971 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-26T04:59:38.971 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-26T04:59:38.971 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.971 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.971 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.971 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.971 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.971 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.971 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.972 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.973 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.973 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.973 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.973 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-26T04:59:38.973 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-26T04:59:38.973 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.973 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.973 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.973 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.973 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.973 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.973 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.974 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.975 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.976 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.976 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.976 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-26T04:59:38.976 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-26T04:59:38.976 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.976 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.976 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.976 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.976 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.976 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.976 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.976 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.976 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.978 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_PASSWORD": { 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-26T04:59:38.979 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-26T04:59:38.980 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.980 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.980 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.980 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.980 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.980 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.980 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.980 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.980 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.980 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.980 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_URL": { 2024-02-26T04:59:38.980 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_URL", 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_USERNAME": { 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-26T04:59:38.981 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.982 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.983 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.984 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.984 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.984 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.984 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.984 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.984 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.984 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.984 INFO:teuthology.orchestra.run.smithi060.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-26T04:59:38.984 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-26T04:59:38.984 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.984 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.984 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.985 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_HOST": { 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.986 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.987 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.988 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.988 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.988 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.988 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.988 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-26T04:59:38.988 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-26T04:59:38.988 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.988 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.988 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.988 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.988 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.988 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.988 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.989 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.989 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.989 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.989 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.989 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-26T04:59:38.989 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-26T04:59:38.989 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.989 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.989 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.989 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.989 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.989 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.989 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.990 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.990 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.990 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.990 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.990 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-26T04:59:38.990 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-26T04:59:38.990 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.990 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.990 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.990 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.990 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.990 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.990 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.991 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.991 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.991 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.991 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.991 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-26T04:59:38.991 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-26T04:59:38.991 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.991 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.991 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.991 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.991 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.991 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.991 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.992 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.992 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.992 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.992 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.992 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-26T04:59:38.992 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-26T04:59:38.992 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.992 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.992 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.992 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.992 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.992 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.992 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.993 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.994 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_ENABLED": { 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.995 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.996 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.996 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-26T04:59:38.996 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-26T04:59:38.996 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:38.996 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.996 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.996 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-26T04:59:38.996 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.996 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.996 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.996 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.996 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.996 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.997 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8", 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "45", 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:38.999 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ACCESS_KEY": { 2024-02-26T04:59:39.000 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-26T04:59:39.000 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.000 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.000 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.000 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.000 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.000 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.000 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.000 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.000 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.000 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.000 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.000 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SECRET_KEY": { 2024-02-26T04:59:39.001 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SSL_VERIFY": { 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.002 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.003 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.004 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:39.004 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.004 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.004 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.004 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.004 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.004 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.004 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.004 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-26T04:59:39.004 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-26T04:59:39.004 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.004 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.004 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.005 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T04:59:39.006 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.006 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.006 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.006 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.006 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.006 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.006 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.006 INFO:teuthology.orchestra.run.smithi060.stdout: "cross_origin_url": { 2024-02-26T04:59:39.006 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cross_origin_url", 2024-02-26T04:59:39.006 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.006 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.006 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.006 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: "crt_file": { 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crt_file", 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.007 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.008 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.008 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.008 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.008 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.008 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.008 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.008 INFO:teuthology.orchestra.run.smithi060.stdout: "debug": { 2024-02-26T04:59:39.008 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "debug", 2024-02-26T04:59:39.008 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.008 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.008 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.008 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.008 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.009 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.009 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.009 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable/disable debug options", 2024-02-26T04:59:39.009 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.009 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.009 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.009 INFO:teuthology.orchestra.run.smithi060.stdout: "jwt_token_ttl": { 2024-02-26T04:59:39.009 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "jwt_token_ttl", 2024-02-26T04:59:39.009 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.009 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.009 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.009 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "28800", 2024-02-26T04:59:39.009 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.010 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.011 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.012 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.012 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.012 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.012 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.012 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.012 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.012 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.012 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.012 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.012 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.012 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.012 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.012 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.014 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.015 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.015 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.015 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.015 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.015 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.015 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.015 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.015 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.015 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.015 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.015 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.015 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.015 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: "motd": { 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "motd", 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "The message of the day", 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.016 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.017 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect_resolve_ip_addr": { 2024-02-26T04:59:39.017 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "redirect_resolve_ip_addr", 2024-02-26T04:59:39.017 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.017 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.017 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.017 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.017 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.017 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.017 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.017 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.017 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.017 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.018 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.019 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T04:59:39.019 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T04:59:39.019 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.019 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.019 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.019 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8080", 2024-02-26T04:59:39.019 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.019 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.019 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.019 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.019 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.019 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.019 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.020 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-02-26T04:59:39.020 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-02-26T04:59:39.020 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.020 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.020 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.020 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:39.020 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.020 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.020 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.020 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.020 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.020 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.020 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl_server_port": { 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl_server_port", 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8443", 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "redirect", 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect" 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.022 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.023 INFO:teuthology.orchestra.run.smithi060.stdout: "url_prefix": { 2024-02-26T04:59:39.024 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url_prefix", 2024-02-26T04:59:39.024 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.024 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.024 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.024 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.024 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.024 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.024 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.024 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.024 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.024 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.024 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.024 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.024 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.025 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-02-26T04:59:39.025 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.025 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.025 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.025 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_monitoring": { 2024-02-26T04:59:39.025 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_monitoring", 2024-02-26T04:59:39.025 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.025 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.025 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.025 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:39.025 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.025 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "monitor device health metrics", 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.026 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.027 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.028 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.028 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.028 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.028 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.028 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.028 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.028 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.028 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.028 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.028 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.028 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.028 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.029 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "mark_out_threshold": { 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mark_out_threshold", 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2419200", 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": { 2024-02-26T04:59:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_name", 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "device_health_metrics", 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "retention_period": { 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retention_period", 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:39.032 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15552000", 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain device health metrics", 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_frequency": { 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_frequency", 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.033 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: "self_heal": { 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "self_heal", 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:39.034 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.035 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.036 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.036 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and check device health", 2024-02-26T04:59:39.036 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.036 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.036 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.036 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_threshold": { 2024-02-26T04:59:39.036 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_threshold", 2024-02-26T04:59:39.036 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:39.036 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.036 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.036 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7257600", 2024-02-26T04:59:39.036 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.036 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "diskprediction_local", 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.038 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.040 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.040 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.040 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.040 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.040 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.040 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.040 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.040 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.040 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.040 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.040 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.040 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.040 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.041 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: "predict_interval": { 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predict_interval", 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.042 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: "predictor_model": { 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predictor_model", 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.043 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "prophetstor", 2024-02-26T04:59:39.044 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.044 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.044 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.044 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.044 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.044 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.044 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.044 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:39.044 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:39.044 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.044 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.044 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.044 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "influx", 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": false, 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "influxdb python module not found", 2024-02-26T04:59:39.045 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.046 INFO:teuthology.orchestra.run.smithi060.stdout: "batch_size": { 2024-02-26T04:59:39.046 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "batch_size", 2024-02-26T04:59:39.046 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.046 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.046 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.046 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5000", 2024-02-26T04:59:39.046 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.046 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.046 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.046 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-26T04:59:39.046 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.046 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.046 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: "database": { 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "database", 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "ceph", 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.047 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": { 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hostname", 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server hostname", 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:39.048 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "30", 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "5", 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.049 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.050 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.050 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.050 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.050 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.050 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.050 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.050 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.050 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.050 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.050 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.050 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.051 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.052 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.052 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.052 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.052 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.052 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.052 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.052 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.052 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.052 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.052 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.052 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.052 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.052 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.053 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: "password": { 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "password", 2024-02-26T04:59:39.054 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.055 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.055 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.055 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.055 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.055 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.055 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.055 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "password of InfluxDB server user", 2024-02-26T04:59:39.055 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.055 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.055 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.055 INFO:teuthology.orchestra.run.smithi060.stdout: "port": { 2024-02-26T04:59:39.055 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "port", 2024-02-26T04:59:39.055 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8086", 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server port", 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.056 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.057 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.057 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "false", 2024-02-26T04:59:39.057 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.057 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.057 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.057 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T04:59:39.057 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.057 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.057 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.057 INFO:teuthology.orchestra.run.smithi060.stdout: "threads": { 2024-02-26T04:59:39.057 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threads", 2024-02-26T04:59:39.057 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.057 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "32", 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: "username": { 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "username", 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.058 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "username of InfluxDB server user", 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: "verify_ssl": { 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "verify_ssl", 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.059 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "true", 2024-02-26T04:59:39.060 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.060 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.060 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.060 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-26T04:59:39.060 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.060 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.060 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.060 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.060 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.060 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "insights", 2024-02-26T04:59:39.060 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.060 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.061 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.063 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.064 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.064 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.064 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.064 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.064 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.064 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.064 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.064 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.064 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.064 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.064 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.064 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.064 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.066 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "iostat", 2024-02-26T04:59:39.066 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.066 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.066 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.066 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.066 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.066 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.066 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.066 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.066 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.066 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.066 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.067 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.068 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.069 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.070 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.070 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.070 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.070 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.070 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.070 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.070 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.070 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.070 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.070 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.070 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.070 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.070 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "k8sevents", 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_event_retention_days": { 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ceph_event_retention_days", 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.071 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.072 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7", 2024-02-26T04:59:39.072 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.072 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.072 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.072 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-26T04:59:39.072 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.072 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.072 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.072 INFO:teuthology.orchestra.run.smithi060.stdout: "config_check_secs": { 2024-02-26T04:59:39.072 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_check_secs", 2024-02-26T04:59:39.072 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.072 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.072 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.073 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:39.073 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "10", 2024-02-26T04:59:39.073 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.073 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.073 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-26T04:59:39.073 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.073 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.073 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.073 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.073 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.073 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.073 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.073 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.074 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.076 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.076 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.076 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.076 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.076 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.076 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.076 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.076 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.076 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.076 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.076 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.076 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.076 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.076 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.077 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.078 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.078 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.078 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.078 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.078 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.078 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "localpool", 2024-02-26T04:59:39.078 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.078 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.078 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.078 INFO:teuthology.orchestra.run.smithi060.stdout: "failure_domain": { 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "failure_domain", 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "host", 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "failure domain for any created local pool", 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.079 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.080 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.081 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.081 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.081 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.081 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.081 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.081 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.081 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.081 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.081 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.081 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.081 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.081 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.081 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.082 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.082 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.082 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.082 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.082 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.082 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.082 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.082 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.082 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.082 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.082 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.082 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.083 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.083 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.083 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.083 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.083 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.083 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.083 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.083 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.083 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.083 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.083 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.083 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.083 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.084 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.084 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.084 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.084 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.084 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.084 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.084 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.084 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.084 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.084 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.084 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.084 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": { 2024-02-26T04:59:39.084 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_size", 2024-02-26T04:59:39.085 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.085 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.085 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.085 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.085 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.085 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.085 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.085 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default min_size for any created local pool", 2024-02-26T04:59:39.085 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-26T04:59:39.085 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.085 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.085 INFO:teuthology.orchestra.run.smithi060.stdout: "num_rep": { 2024-02-26T04:59:39.085 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "num_rep", 2024-02-26T04:59:39.086 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.086 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.086 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.086 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3", 2024-02-26T04:59:39.086 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.086 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.086 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.086 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default replica count for any created local pool", 2024-02-26T04:59:39.086 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.086 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.086 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.086 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": { 2024-02-26T04:59:39.086 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_num", 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "128", 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default pg_num for any created local pool", 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: "prefix": { 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prefix", 2024-02-26T04:59:39.087 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.088 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.088 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.088 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.088 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.088 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.088 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.088 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name prefix for any created local pool", 2024-02-26T04:59:39.088 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.088 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.088 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.088 INFO:teuthology.orchestra.run.smithi060.stdout: "subtree": { 2024-02-26T04:59:39.088 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "subtree", 2024-02-26T04:59:39.089 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.089 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.089 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.089 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "rack", 2024-02-26T04:59:39.089 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.089 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.089 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.089 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-26T04:59:39.089 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-26T04:59:39.089 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.089 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.089 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.089 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mds_autoscaler", 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.090 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.091 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.091 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.091 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.091 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.091 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.091 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.091 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.091 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.091 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.091 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.091 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.091 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.091 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.092 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.093 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.093 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.093 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.093 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.093 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.093 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.093 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.093 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.093 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.093 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.093 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.093 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.093 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.093 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.094 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.095 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.095 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.095 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirroring", 2024-02-26T04:59:39.095 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.095 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.095 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.095 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.095 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.095 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.095 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.095 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.095 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.096 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.096 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.096 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.096 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.096 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.096 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.096 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.096 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.096 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.096 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.096 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.096 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.096 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.096 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.099 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.099 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.099 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.099 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.099 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.099 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.099 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.099 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.099 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.099 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.099 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.099 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.099 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.100 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.100 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.100 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.100 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.100 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.100 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.100 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.100 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.100 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "nfs", 2024-02-26T04:59:39.100 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.100 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.100 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.101 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.102 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.102 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.102 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.102 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.102 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.102 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.102 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.102 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.102 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.102 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.102 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.102 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.102 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.103 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.104 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.104 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.104 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.104 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.104 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.104 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.104 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.104 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.104 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.104 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.104 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.104 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.104 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.105 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-02-26T04:59:39.106 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.106 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.106 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.106 INFO:teuthology.orchestra.run.smithi060.stdout: "fail_fs": { 2024-02-26T04:59:39.106 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "fail_fs", 2024-02-26T04:59:39.106 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.106 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.106 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.106 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.106 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.106 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.106 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.106 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-26T04:59:39.106 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.107 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.108 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.108 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.108 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.108 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.108 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.108 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.108 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.108 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.108 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.108 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.108 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.108 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.108 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.109 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.110 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.110 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.110 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.110 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.110 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.110 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.110 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.110 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.110 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.110 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.110 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.110 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.110 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.111 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.112 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator": { 2024-02-26T04:59:39.112 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-02-26T04:59:39.112 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.112 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.112 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.112 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.112 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.112 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.112 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.112 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm", 2024-02-26T04:59:39.112 INFO:teuthology.orchestra.run.smithi060.stdout: "rook", 2024-02-26T04:59:39.112 INFO:teuthology.orchestra.run.smithi060.stdout: "test_orchestrator" 2024-02-26T04:59:39.112 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Orchestrator backend", 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_perf_query", 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.113 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.115 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.116 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.116 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.116 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.116 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.116 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.116 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.116 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.116 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.116 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.116 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.116 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.116 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.116 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.116 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.117 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.118 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.118 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.118 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.118 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.118 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.118 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_support", 2024-02-26T04:59:39.118 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.118 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.118 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.118 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.118 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.118 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.119 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.119 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.119 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.119 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.119 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.119 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.119 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.119 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.119 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.119 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.119 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.119 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.119 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.119 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.120 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.120 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.120 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.120 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.120 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.120 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.120 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.120 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.120 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.120 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.120 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.120 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.120 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.121 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.122 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_autoscaler", 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.123 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.124 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.124 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.124 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.124 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.124 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.124 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.124 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.124 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.124 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.124 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.124 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.124 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.124 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.124 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.125 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.125 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.125 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.125 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.125 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.125 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.125 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.125 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.125 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.125 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.125 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.125 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.125 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.126 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.127 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.127 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.127 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.127 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.127 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.127 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.127 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.127 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.127 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.127 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.127 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.127 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.127 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:39.128 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:39.129 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.129 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.129 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:39.129 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.129 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.129 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.129 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.129 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.129 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.129 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.129 INFO:teuthology.orchestra.run.smithi060.stdout: "threshold": { 2024-02-26T04:59:39.129 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threshold", 2024-02-26T04:59:39.129 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1.0", 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "scaling threshold", 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.130 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "progress", 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_pg_recovery_event": { 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_pg_recovery_event", 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.131 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.132 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.132 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-02-26T04:59:39.132 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-02-26T04:59:39.132 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.132 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.132 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.132 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:39.132 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.132 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.132 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.132 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.132 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.132 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.133 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.134 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.134 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.134 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.134 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.134 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.134 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.134 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.134 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.134 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.134 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.134 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.134 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.134 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.135 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.136 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.136 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.136 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.136 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.136 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.136 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.136 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.136 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.136 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.136 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.136 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.136 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.136 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.137 INFO:teuthology.orchestra.run.smithi060.stdout: "max_completed_events": { 2024-02-26T04:59:39.138 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_completed_events", 2024-02-26T04:59:39.138 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.138 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.138 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.138 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "50", 2024-02-26T04:59:39.138 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.138 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.138 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.138 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "number of past completed events to remember", 2024-02-26T04:59:39.138 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.138 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.138 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.138 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long the module is going to sleep", 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus", 2024-02-26T04:59:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: "cache": { 2024-02-26T04:59:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cache", 2024-02-26T04:59:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: "exclude_perf_counters": { 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "exclude_perf_counters", 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.141 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-26T04:59:39.142 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-26T04:59:39.142 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.142 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.142 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.142 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.142 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.142 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.142 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.142 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.142 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.142 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.142 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.142 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.142 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.143 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.144 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.144 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.144 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.144 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.144 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.144 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.144 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.144 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.144 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.144 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.144 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.144 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.144 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.144 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.145 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.146 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools": { 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools", 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "300", 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_interval": { 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_interval", 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15.0", 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T04:59:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T04:59:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-02-26T04:59:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-26T04:59:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T04:59:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "9283", 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: "stale_cache_strategy": { 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stale_cache_strategy", 2024-02-26T04:59:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.152 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.152 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.152 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "log", 2024-02-26T04:59:39.152 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.152 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.152 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.152 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.152 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.152 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.152 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.152 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-02-26T04:59:39.152 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "default", 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: "default", 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: "error" 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.154 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-02-26T04:59:39.154 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-02-26T04:59:39.154 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.154 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.154 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.154 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-02-26T04:59:39.154 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-02-26T04:59:39.154 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-02-26T04:59:39.154 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.154 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.154 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.154 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.154 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.154 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.155 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.155 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-02-26T04:59:39.155 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.155 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.155 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.155 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.155 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.155 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.155 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.155 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.155 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.156 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.156 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.156 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.156 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.156 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.156 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.156 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.156 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.156 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.156 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.156 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.156 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.156 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.156 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.157 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.158 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.159 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.160 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.160 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.160 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_snap_create": { 2024-02-26T04:59:39.160 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_snap_create", 2024-02-26T04:59:39.160 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.160 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.160 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.160 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-02-26T04:59:39.160 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.160 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.160 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.160 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "mirror_snapshot_schedule": { 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirror_snapshot_schedule", 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.161 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: "trash_purge_schedule": { 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "trash_purge_schedule", 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.162 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.163 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.163 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.163 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "restful", 2024-02-26T04:59:39.163 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.163 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.163 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.163 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_auth": { 2024-02-26T04:59:39.163 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_auth", 2024-02-26T04:59:39.163 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.163 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.163 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.163 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.164 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.165 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.166 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.166 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.166 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.166 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.166 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.166 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.166 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.166 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.166 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.166 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.166 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.166 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.166 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.167 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.168 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.168 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.168 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.168 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.168 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.168 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.168 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.168 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.168 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.168 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.168 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.168 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.168 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.169 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.169 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.169 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.169 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.169 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.169 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.169 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.169 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.169 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.169 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.169 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.169 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.169 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.170 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.170 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.170 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.170 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.170 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-02-26T04:59:39.170 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-02-26T04:59:39.170 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.170 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.170 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.170 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.170 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.170 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.170 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.171 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.172 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.172 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.172 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.172 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.172 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.172 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rgw", 2024-02-26T04:59:39.172 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.172 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.172 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.172 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.172 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.173 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.174 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.175 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.175 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.175 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.175 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.175 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.175 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.175 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.175 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.175 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.175 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.175 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.175 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.175 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.176 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rook", 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: "drive_group_interval": { 2024-02-26T04:59:39.177 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "drive_group_interval", 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "300.0", 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.178 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.179 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.180 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.180 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.180 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.180 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.180 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.180 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.180 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.180 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.180 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.180 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.180 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.180 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.181 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.181 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.181 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.181 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.181 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.181 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.181 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.181 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.181 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.181 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.181 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.181 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.181 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.182 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.182 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.182 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.182 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.182 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.182 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.182 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.182 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.182 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.182 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.182 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.182 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.182 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.183 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.183 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.183 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.183 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.183 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.183 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.183 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.183 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.183 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.183 INFO:teuthology.orchestra.run.smithi060.stdout: "storage_class": { 2024-02-26T04:59:39.183 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "storage_class", 2024-02-26T04:59:39.183 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "local", 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.184 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "selftest", 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.185 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.186 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.187 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.189 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.189 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.189 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.189 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.189 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.189 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.189 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.189 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.189 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.189 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.189 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.189 INFO:teuthology.orchestra.run.smithi060.stdout: "roption1": { 2024-02-26T04:59:39.189 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption1", 2024-02-26T04:59:39.190 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.190 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.190 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.190 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.190 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.190 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.190 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.190 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.190 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.190 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.190 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.190 INFO:teuthology.orchestra.run.smithi060.stdout: "roption2": { 2024-02-26T04:59:39.190 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption2", 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "xyz", 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption1": { 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption1", 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.191 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.192 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.192 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.192 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.192 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.192 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.192 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.192 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.192 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.192 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.192 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption2": { 2024-02-26T04:59:39.192 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption2", 2024-02-26T04:59:39.192 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.192 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption3": { 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption3", 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.193 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption4": { 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption4", 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.194 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption5": { 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption5", 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.195 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption6": { 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption6", 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.197 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.197 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.197 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption7": { 2024-02-26T04:59:39.197 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption7", 2024-02-26T04:59:39.197 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.197 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.197 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.197 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.197 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-02-26T04:59:39.197 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "42", 2024-02-26T04:59:39.197 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.197 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.197 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: "testkey": { 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testkey", 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.198 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: "testlkey": { 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testlkey", 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.199 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.200 INFO:teuthology.orchestra.run.smithi060.stdout: "testnewline": { 2024-02-26T04:59:39.200 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testnewline", 2024-02-26T04:59:39.200 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.200 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.200 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.200 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.200 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.200 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.200 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.200 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.200 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.200 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snap_schedule", 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_m_granularity": { 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_m_granularity", 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.201 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow minute scheduled snapshots", 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: "dump_on_update": { 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dump_on_update", 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.202 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "dump database to debug log on update", 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.203 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.204 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.205 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.206 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.207 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.208 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.208 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stats", 2024-02-26T04:59:39.208 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.208 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.208 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.208 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.208 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.208 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.208 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.208 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.208 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.208 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.209 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.211 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.212 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.212 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.212 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.212 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.212 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.212 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.212 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.212 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.212 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.212 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.212 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.212 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "status", 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.216 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.216 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.216 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.216 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.216 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.216 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.216 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.216 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.216 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.216 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.216 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.216 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.216 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.216 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.218 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.218 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.218 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.218 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.218 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.218 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.218 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.218 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telegraf", 2024-02-26T04:59:39.218 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.218 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.218 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.218 INFO:teuthology.orchestra.run.smithi060.stdout: "address": { 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "address", 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:39.219 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:39.220 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:39.220 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.220 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.220 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15", 2024-02-26T04:59:39.220 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.220 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.220 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.220 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.220 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.220 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.220 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.220 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.220 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.221 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.221 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.221 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.221 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.221 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.221 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.221 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.221 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.221 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.221 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.221 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.221 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.221 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.221 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.222 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.223 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.224 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.224 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.224 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.224 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.224 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.224 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.224 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.224 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.224 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.224 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.224 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.224 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telemetry", 2024-02-26T04:59:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_basic": { 2024-02-26T04:59:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_basic", 2024-02-26T04:59:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-26T04:59:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_crash": { 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_crash", 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_device": { 2024-02-26T04:59:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_device", 2024-02-26T04:59:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-02-26T04:59:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-26T04:59:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.229 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.229 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_ident": { 2024-02-26T04:59:39.229 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_ident", 2024-02-26T04:59:39.229 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.229 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.229 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.229 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.229 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.229 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.229 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.229 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-26T04:59:39.229 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.229 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.230 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.230 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_perf": { 2024-02-26T04:59:39.230 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_perf", 2024-02-26T04:59:39.230 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.230 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.230 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.230 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.230 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.230 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.230 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.230 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-26T04:59:39.230 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.230 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: "contact": { 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "contact", 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.231 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.232 INFO:teuthology.orchestra.run.smithi060.stdout: "description": { 2024-02-26T04:59:39.232 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "description", 2024-02-26T04:59:39.232 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.232 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.232 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.232 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.232 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.232 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.232 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.232 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.232 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.232 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.232 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.233 INFO:teuthology.orchestra.run.smithi060.stdout: "device_url": { 2024-02-26T04:59:39.233 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_url", 2024-02-26T04:59:39.233 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.233 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.233 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.233 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-26T04:59:39.233 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.233 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.233 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.233 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.233 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.233 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.233 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.234 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-02-26T04:59:39.234 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-02-26T04:59:39.234 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.234 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.234 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.234 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.234 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.234 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.234 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.234 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.234 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.234 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.234 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "24", 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "8", 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.236 INFO:teuthology.orchestra.run.smithi060.stdout: "last_opt_revision": { 2024-02-26T04:59:39.236 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "last_opt_revision", 2024-02-26T04:59:39.236 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.236 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.236 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.236 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1", 2024-02-26T04:59:39.236 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.236 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.236 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.236 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.236 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.236 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.236 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard": { 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard", 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard_description": { 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard_description", 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.238 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.239 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.240 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.240 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.240 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.240 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.240 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.240 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.240 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.240 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.240 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.240 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.240 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.240 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.241 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.241 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.241 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.241 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.241 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.241 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.241 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.241 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.241 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.241 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.241 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.241 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.241 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.242 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.242 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.242 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.242 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.242 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.242 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.242 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.242 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.242 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.242 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.242 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.242 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.242 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.243 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.243 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.243 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.243 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.243 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.243 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.243 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.243 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.243 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.243 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.243 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.243 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.243 INFO:teuthology.orchestra.run.smithi060.stdout: "organization": { 2024-02-26T04:59:39.244 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "organization", 2024-02-26T04:59:39.244 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.244 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.244 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.244 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.244 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.244 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.244 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.244 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.244 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.244 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.244 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: "proxy": { 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "proxy", 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: "url": { 2024-02-26T04:59:39.246 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url", 2024-02-26T04:59:39.246 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.246 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.246 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.246 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-26T04:59:39.246 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.246 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.246 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.246 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.246 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.246 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.246 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.246 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.246 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "test_orchestrator", 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.247 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.248 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.248 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.248 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.248 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.248 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.248 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.248 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.248 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.248 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.248 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.248 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.248 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.248 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.249 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.250 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.251 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.252 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.252 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.252 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-02-26T04:59:39.252 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.252 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.252 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.252 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.252 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.252 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.252 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.252 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.253 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.254 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.255 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.256 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.256 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.256 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.256 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.256 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.256 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.256 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.256 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.256 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.256 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.256 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.256 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.256 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_clones": { 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_clones", 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4", 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-26T04:59:39.257 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: "periodic_async_work": { 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "periodic_async_work", 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Periodically check for async work", 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.258 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.259 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.259 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_delay": { 2024-02-26T04:59:39.259 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_delay", 2024-02-26T04:59:39.259 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.259 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.259 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.259 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-02-26T04:59:39.259 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.259 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.259 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.259 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-26T04:59:39.259 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.259 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.260 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.260 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.260 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.260 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix", 2024-02-26T04:59:39.260 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-02-26T04:59:39.260 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-02-26T04:59:39.260 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-02-26T04:59:39.260 INFO:teuthology.orchestra.run.smithi060.stdout: "discovery_interval": { 2024-02-26T04:59:39.260 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "discovery_interval", 2024-02-26T04:59:39.260 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-02-26T04:59:39.260 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.260 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.261 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "100", 2024-02-26T04:59:39.261 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.261 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.261 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.261 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.261 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.261 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.261 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.261 INFO:teuthology.orchestra.run.smithi060.stdout: "identifier": { 2024-02-26T04:59:39.261 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "identifier", 2024-02-26T04:59:39.261 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.261 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.261 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.262 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.262 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.262 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.262 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.262 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.262 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.262 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.262 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.262 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-02-26T04:59:39.262 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-02-26T04:59:39.262 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-02-26T04:59:39.262 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.262 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.263 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.264 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.264 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.264 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.264 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.264 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.264 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.264 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.264 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.264 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.264 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.264 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.264 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.264 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.265 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-02-26T04:59:39.266 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.267 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.268 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.268 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.268 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.268 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.268 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.268 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_host": { 2024-02-26T04:59:39.268 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_host", 2024-02-26T04:59:39.268 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.268 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.268 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.268 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-02-26T04:59:39.268 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.268 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.269 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.269 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.269 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.269 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.269 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.269 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_port": { 2024-02-26T04:59:39.269 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_port", 2024-02-26T04:59:39.269 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-02-26T04:59:39.269 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.269 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.269 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10051", 2024-02-26T04:59:39.269 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.269 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_sender": { 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_sender", 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-02-26T04:59:39.270 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: "services": { 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: "dashboard": "https://172.21.15.60:8443/" 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: "always_on_modules": { 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: "octopus": [ 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-02-26T04:59:39.271 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: "pacific": [ 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-02-26T04:59:39.272 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-02-26T04:59:39.273 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-02-26T04:59:39.273 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-02-26T04:59:39.273 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-02-26T04:59:39.273 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-02-26T04:59:39.273 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-02-26T04:59:39.273 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.273 INFO:teuthology.orchestra.run.smithi060.stdout: "quincy": [ 2024-02-26T04:59:39.273 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-02-26T04:59:39.273 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-02-26T04:59:39.273 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-02-26T04:59:39.273 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-02-26T04:59:39.273 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-02-26T04:59:39.273 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-02-26T04:59:39.273 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: "reef": [ 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-02-26T04:59:39.274 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: "squid": [ 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: "last_failure_osd_epoch": 3, 2024-02-26T04:59:39.276 INFO:teuthology.orchestra.run.smithi060.stdout: "active_clients": [ 2024-02-26T04:59:39.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.276 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-02-26T04:59:39.276 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-02-26T04:59:39.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.276 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-02-26T04:59:39.276 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-02-26T04:59:39.276 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 612643565 2024-02-26T04:59:39.276 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.276 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T04:59:39.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.276 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "libcephsqlite", 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2990090132 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-02-26T04:59:39.277 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3636752194 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4212036338 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-02-26T04:59:39.278 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:59:39.280 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:38 smithi156 ceph-mon[35805]: pgmap v422: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 22 op/s 2024-02-26T04:59:39.280 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:38 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/353136538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:59:39.280 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:38 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/236417722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:59:39.280 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:38 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3556912836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-26T04:59:39.280 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:38 smithi191 ceph-mon[36352]: pgmap v422: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 22 op/s 2024-02-26T04:59:39.280 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:38 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/353136538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:59:39.280 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:38 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/236417722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:59:39.281 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:38 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3556912836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-26T04:59:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:38.034+0000 7fe07f51e640 1 -- 172.21.15.60:0/3556912836 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe0600a20c0 msgr2=0x7fe0600a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:38.034+0000 7fe07f51e640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe0600a20c0 0x7fe0600a4580 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe0781010f0 tx=0x7fe070009680 comp rx=0 tx=0).stop 2024-02-26T04:59:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:38.034+0000 7fe07f51e640 1 -- 172.21.15.60:0/3556912836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe078101c50 msgr2=0x7fe07819d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T04:59:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:38.034+0000 7fe07f51e640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe078101c50 0x7fe07819d670 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe068009230 tx=0x7fe068004270 comp rx=0 tx=0).stop 2024-02-26T04:59:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:38.034+0000 7fe07f51e640 1 -- 172.21.15.60:0/3556912836 shutdown_connections 2024-02-26T04:59:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:38.034+0000 7fe07f51e640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe0600a20c0 0x7fe0600a4580 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:38.034+0000 7fe07f51e640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe078101c50 0x7fe07819d670 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:38.034+0000 7fe07f51e640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe078101290 0x7fe078196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:38.034+0000 7fe07f51e640 1 --2- 172.21.15.60:0/3556912836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0781000a0 0x7fe0781960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T04:59:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:38.034+0000 7fe07f51e640 1 -- 172.21.15.60:0/3556912836 >> 172.21.15.60:0/3556912836 conn(0x7fe0780fb610 msgr2=0x7fe078104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T04:59:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:38.034+0000 7fe07f51e640 1 -- 172.21.15.60:0/3556912836 shutdown_connections 2024-02-26T04:59:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T04:59:38.034+0000 7fe07f51e640 1 -- 172.21.15.60:0/3556912836 wait complete. 2024-02-26T04:59:39.287 DEBUG:tasks.check_counter:Getting stats from a 2024-02-26T04:59:39.287 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph daemon mds.a perf dump 2024-02-26T04:59:39.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:38 smithi060 ceph-mon[32382]: pgmap v422: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 22 op/s 2024-02-26T04:59:39.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:38 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/353136538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T04:59:39.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:38 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/236417722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T04:59:39.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:38 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3556912836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-26T04:59:39.544 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:59:40.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:40 smithi060 ceph-mon[32382]: pgmap v423: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T04:59:40.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:40 smithi191 ceph-mon[36352]: pgmap v423: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T04:59:40.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:40 smithi156 ceph-mon[35805]: pgmap v423: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-26T04:59:41.409 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:59:41.409 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T04:59:41.409 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 81, 2024-02-26T04:59:41.409 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 76, 2024-02-26T04:59:41.409 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 437264, 2024-02-26T04:59:41.409 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 43728, 2024-02-26T04:59:41.409 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 0, 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.014452856, 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.004471690, 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.009718423, 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.001275432, 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 76, 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.004449216, 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000058542 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 437264, 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 43728 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 1, 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 56, 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 160, 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 87168, 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 3, 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.004807045, 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.002878163, 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.001329263, 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 54, 2024-02-26T04:59:41.412 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.002162876, 2024-02-26T04:59:41.412 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000040053 2024-02-26T04:59:41.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:41.412 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:41.412 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:41.412 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:41.412 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:41.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:41.412 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-26T04:59:41.412 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 87168 2024-02-26T04:59:41.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:41.412 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T04:59:41.412 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 0, 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2, 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 0, 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 0, 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 2, 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 1, 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.000414878, 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.000363995, 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:41.413 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:41.414 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:41.414 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:41.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:41.414 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T04:59:41.414 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T04:59:41.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:41.414 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-02-26T04:59:41.414 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 0, 2024-02-26T04:59:41.414 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-02-26T04:59:41.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:41.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-02-26T04:59:41.414 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-02-26T04:59:41.414 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-02-26T04:59:41.414 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-02-26T04:59:41.415 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-02-26T04:59:41.416 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T04:59:41.416 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T04:59:41.416 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T04:59:41.416 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T04:59:41.416 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T04:59:41.416 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-02-26T04:59:41.416 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-02-26T04:59:41.416 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-02-26T04:59:41.416 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T04:59:41.416 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T04:59:41.416 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-02-26T04:59:41.416 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-02-26T04:59:41.416 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 148823, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 27, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-02-26T04:59:41.417 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 0, 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 0, 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 928, 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 2, 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-02-26T04:59:41.418 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 78, 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 423416, 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 78, 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 78, 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 423416, 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:41.419 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:59:42.126 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-02-26T04:59:42.126 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-02-26T04:59:42.126 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-02-26T04:59:42.126 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-02-26T04:59:42.126 DEBUG:tasks.check_counter:Getting stats from d 2024-02-26T04:59:42.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph daemon mds.d perf dump 2024-02-26T04:59:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:59:42.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:42 smithi060 ceph-mon[32382]: pgmap v424: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 7 op/s 2024-02-26T04:59:42.480 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:42 smithi156 ceph-mon[35805]: pgmap v424: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 7 op/s 2024-02-26T04:59:42.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:42 smithi191 ceph-mon[36352]: pgmap v424: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 7 op/s 2024-02-26T04:59:44.385 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:44 smithi060 ceph-mon[32382]: pgmap v425: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-02-26T04:59:44.413 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:59:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T04:59:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 0, 2024-02-26T04:59:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2, 2024-02-26T04:59:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 0, 2024-02-26T04:59:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 0, 2024-02-26T04:59:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-02-26T04:59:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 0, 2024-02-26T04:59:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.001239614, 2024-02-26T04:59:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T04:59:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.000341607, 2024-02-26T04:59:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T04:59:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T04:59:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T04:59:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T04:59:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 83, 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 80, 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 435872, 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 47024, 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 2, 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 1, 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.014188303, 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.004897588, 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.010146841, 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.001438533, 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 79, 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.003380685, 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000042793 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 435872, 2024-02-26T04:59:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 47024 2024-02-26T04:59:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T04:59:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 5, 2024-02-26T04:59:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 63, 2024-02-26T04:59:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 3184, 2024-02-26T04:59:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 88496, 2024-02-26T04:59:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 3, 2024-02-26T04:59:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2024-02-26T04:59:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.005427322, 2024-02-26T04:59:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.003450737, 2024-02-26T04:59:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.001383904, 2024-02-26T04:59:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T04:59:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 62, 2024-02-26T04:59:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.003520140, 2024-02-26T04:59:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000056776 2024-02-26T04:59:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-02-26T04:59:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 88496 2024-02-26T04:59:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-02-26T04:59:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 0, 2024-02-26T04:59:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-02-26T04:59:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-02-26T04:59:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-02-26T04:59:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-02-26T04:59:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T04:59:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-02-26T04:59:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T04:59:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-02-26T04:59:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T04:59:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T04:59:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T04:59:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T04:59:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T04:59:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-02-26T04:59:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T04:59:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T04:59:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-02-26T04:59:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-02-26T04:59:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-02-26T04:59:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-02-26T04:59:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T04:59:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T04:59:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T04:59:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T04:59:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 148823, 2024-02-26T04:59:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 27, 2024-02-26T04:59:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2024-02-26T04:59:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2024-02-26T04:59:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-02-26T04:59:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-02-26T04:59:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-02-26T04:59:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-02-26T04:59:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-02-26T04:59:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-02-26T04:59:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 0, 2024-02-26T04:59:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 0, 2024-02-26T04:59:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T04:59:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-02-26T04:59:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-02-26T04:59:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 928, 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 2, 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 84, 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 424196, 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 84, 2024-02-26T04:59:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-02-26T04:59:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-02-26T04:59:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 84, 2024-02-26T04:59:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 424196, 2024-02-26T04:59:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-02-26T04:59:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:44.424 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:59:44.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:44 smithi191 ceph-mon[36352]: pgmap v425: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-02-26T04:59:44.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:44 smithi156 ceph-mon[35805]: pgmap v425: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-02-26T04:59:45.128 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-02-26T04:59:45.128 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-02-26T04:59:45.129 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-02-26T04:59:45.129 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-02-26T04:59:45.129 DEBUG:tasks.check_counter:Getting stats from g 2024-02-26T04:59:45.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph daemon mds.g perf dump 2024-02-26T04:59:45.378 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:59:46.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:46 smithi060 ceph-mon[32382]: pgmap v426: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T04:59:46.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:46 smithi191 ceph-mon[36352]: pgmap v426: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T04:59:46.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:46 smithi156 ceph-mon[35805]: pgmap v426: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T04:59:47.214 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:59:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T04:59:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 83, 2024-02-26T04:59:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 78, 2024-02-26T04:59:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 437808, 2024-02-26T04:59:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 44976, 2024-02-26T04:59:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-02-26T04:59:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 0, 2024-02-26T04:59:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.014021382, 2024-02-26T04:59:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.004353312, 2024-02-26T04:59:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.009942190, 2024-02-26T04:59:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.001309867, 2024-02-26T04:59:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 78, 2024-02-26T04:59:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.003873136, 2024-02-26T04:59:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000049655 2024-02-26T04:59:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 437808, 2024-02-26T04:59:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 44976 2024-02-26T04:59:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T04:59:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 1, 2024-02-26T04:59:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 58, 2024-02-26T04:59:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 160, 2024-02-26T04:59:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 87520, 2024-02-26T04:59:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 3, 2024-02-26T04:59:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2024-02-26T04:59:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.005393978, 2024-02-26T04:59:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.003224477, 2024-02-26T04:59:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.001561899, 2024-02-26T04:59:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T04:59:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 56, 2024-02-26T04:59:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.002353809, 2024-02-26T04:59:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000042032 2024-02-26T04:59:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-26T04:59:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 87520 2024-02-26T04:59:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T04:59:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 0, 2024-02-26T04:59:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2, 2024-02-26T04:59:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 0, 2024-02-26T04:59:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 0, 2024-02-26T04:59:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 2, 2024-02-26T04:59:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 1, 2024-02-26T04:59:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.000334487, 2024-02-26T04:59:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T04:59:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.000278448, 2024-02-26T04:59:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T04:59:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 0, 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-02-26T04:59:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T04:59:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-02-26T04:59:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T04:59:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-02-26T04:59:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T04:59:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T04:59:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T04:59:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T04:59:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T04:59:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-02-26T04:59:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T04:59:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T04:59:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-02-26T04:59:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-02-26T04:59:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-02-26T04:59:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-02-26T04:59:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T04:59:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T04:59:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T04:59:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T04:59:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T04:59:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-02-26T04:59:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-02-26T04:59:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-02-26T04:59:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T04:59:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T04:59:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-02-26T04:59:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-02-26T04:59:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-02-26T04:59:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-02-26T04:59:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T04:59:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-02-26T04:59:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T04:59:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-02-26T04:59:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 148823, 2024-02-26T04:59:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 27, 2024-02-26T04:59:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2024-02-26T04:59:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2024-02-26T04:59:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-02-26T04:59:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-02-26T04:59:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-02-26T04:59:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 0, 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 0, 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 928, 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 2, 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-02-26T04:59:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T04:59:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-02-26T04:59:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-02-26T04:59:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-02-26T04:59:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 80, 2024-02-26T04:59:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 423676, 2024-02-26T04:59:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 80, 2024-02-26T04:59:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-02-26T04:59:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-02-26T04:59:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 80, 2024-02-26T04:59:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 423676, 2024-02-26T04:59:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-02-26T04:59:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:47.225 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:59:47.946 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-02-26T04:59:47.946 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-02-26T04:59:47.947 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-02-26T04:59:47.947 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-02-26T04:59:47.947 DEBUG:tasks.check_counter:Getting stats from j 2024-02-26T04:59:47.947 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph daemon mds.j perf dump 2024-02-26T04:59:48.192 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.a/config 2024-02-26T04:59:48.478 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:48 smithi060 ceph-mon[32382]: pgmap v427: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:59:48.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:48 smithi191 ceph-mon[36352]: pgmap v427: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:59:48.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:48 smithi156 ceph-mon[35805]: pgmap v427: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:59:49.955 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-02-26T04:59:49.955 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T04:59:49.955 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 85, 2024-02-26T04:59:49.955 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 82, 2024-02-26T04:59:49.955 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 436416, 2024-02-26T04:59:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 48272, 2024-02-26T04:59:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2024-02-26T04:59:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 0, 2024-02-26T04:59:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.015848020, 2024-02-26T04:59:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.004804134, 2024-02-26T04:59:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.011693521, 2024-02-26T04:59:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.001483676, 2024-02-26T04:59:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 81, 2024-02-26T04:59:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.003587408, 2024-02-26T04:59:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000044288 2024-02-26T04:59:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:49.956 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 436416, 2024-02-26T04:59:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 48272 2024-02-26T04:59:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T04:59:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 5, 2024-02-26T04:59:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 64, 2024-02-26T04:59:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 3184, 2024-02-26T04:59:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 88672, 2024-02-26T04:59:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 3, 2024-02-26T04:59:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2024-02-26T04:59:49.957 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.006125517, 2024-02-26T04:59:49.958 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.003750160, 2024-02-26T04:59:49.958 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.001750164, 2024-02-26T04:59:49.958 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T04:59:49.958 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:49.958 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 63, 2024-02-26T04:59:49.958 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.003596691, 2024-02-26T04:59:49.958 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000057090 2024-02-26T04:59:49.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:49.958 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-02-26T04:59:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 88672 2024-02-26T04:59:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T04:59:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 0, 2024-02-26T04:59:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2, 2024-02-26T04:59:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 0, 2024-02-26T04:59:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 0, 2024-02-26T04:59:49.959 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 2, 2024-02-26T04:59:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 1, 2024-02-26T04:59:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.000182754, 2024-02-26T04:59:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T04:59:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.000112786, 2024-02-26T04:59:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T04:59:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:49.960 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T04:59:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T04:59:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-02-26T04:59:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 0, 2024-02-26T04:59:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-02-26T04:59:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-02-26T04:59:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-02-26T04:59:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-02-26T04:59:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T04:59:49.961 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-02-26T04:59:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T04:59:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-02-26T04:59:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T04:59:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T04:59:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T04:59:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T04:59:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T04:59:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-02-26T04:59:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T04:59:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T04:59:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-02-26T04:59:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-02-26T04:59:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-02-26T04:59:49.962 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T04:59:49.963 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 148818, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 28, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 0, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 0, 2024-02-26T04:59:49.964 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T04:59:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-02-26T04:59:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-02-26T04:59:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-02-26T04:59:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 928, 2024-02-26T04:59:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 2, 2024-02-26T04:59:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-02-26T04:59:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-02-26T04:59:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-02-26T04:59:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-02-26T04:59:49.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 86, 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 424456, 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 86, 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 86, 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 424456, 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-02-26T04:59:49.966 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-02-26T04:59:49.967 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-02-26T04:59:49.967 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:49.967 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-02-26T04:59:49.967 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-02-26T04:59:50.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:50 smithi060 ceph-mon[32382]: pgmap v428: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T04:59:50.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:50 smithi191 ceph-mon[36352]: pgmap v428: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T04:59:50.662 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-02-26T04:59:50.663 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-02-26T04:59:50.663 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-02-26T04:59:50.663 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-02-26T04:59:50.663 DEBUG:tasks.check_counter:Getting stats from b 2024-02-26T04:59:50.663 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph daemon mds.b perf dump 2024-02-26T04:59:50.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:50 smithi156 ceph-mon[35805]: pgmap v428: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T04:59:50.914 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T04:59:52.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:52 smithi191 ceph-mon[36352]: pgmap v429: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:59:52.585 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:52 smithi156 ceph-mon[35805]: pgmap v429: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:59:52.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:52 smithi060 ceph-mon[32382]: pgmap v429: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:59:53.029 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-02-26T04:59:53.029 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T04:59:53.030 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2379, 2024-02-26T04:59:53.030 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2679, 2024-02-26T04:59:53.030 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 1003364, 2024-02-26T04:59:53.030 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 4741188, 2024-02-26T04:59:53.030 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 32, 2024-02-26T04:59:53.030 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2024-02-26T04:59:53.030 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.304892401, 2024-02-26T04:59:53.030 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.067179228, 2024-02-26T04:59:53.030 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.070634181, 2024-02-26T04:59:53.030 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.157076886, 2024-02-26T04:59:53.030 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:53.030 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2680, 2024-02-26T04:59:53.030 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.324918916, 2024-02-26T04:59:53.030 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000121238 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1276, 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000218345, 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000171 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 478800, 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 56448 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2583, 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2949, 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 767813, 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 4775682, 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 8, 2024-02-26T04:59:53.031 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2024-02-26T04:59:53.032 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.187533779, 2024-02-26T04:59:53.032 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.059314362, 2024-02-26T04:59:53.032 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.059803947, 2024-02-26T04:59:53.032 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.055234992, 2024-02-26T04:59:53.032 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:53.032 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2950, 2024-02-26T04:59:53.032 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.335081784, 2024-02-26T04:59:53.032 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000113587 2024-02-26T04:59:53.032 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.032 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:53.032 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1240, 2024-02-26T04:59:53.032 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000209369, 2024-02-26T04:59:53.032 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000168 2024-02-26T04:59:53.032 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 182912 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 7787, 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 13049, 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 2265895, 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 9249641, 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 18, 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 8, 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.466460600, 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.202453718, 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.334456685, 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.071329373, 2024-02-26T04:59:53.033 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 13043, 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.785361109, 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000060213 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 8472, 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.002189800, 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000258 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 55264, 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 30352 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2024-02-26T04:59:53.034 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2633, 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1.807055850, 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000686310 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 4699, 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.243689868, 2024-02-26T04:59:53.035 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000051859 2024-02-26T04:59:53.036 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:53.036 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.036 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-02-26T04:59:53.036 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 2482, 2024-02-26T04:59:53.036 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 2482, 2024-02-26T04:59:53.036 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2024-02-26T04:59:53.036 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2482, 2024-02-26T04:59:53.036 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2.770565970, 2024-02-26T04:59:53.036 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001116263 2024-02-26T04:59:53.036 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.036 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2024-02-26T04:59:53.036 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2024-02-26T04:59:53.036 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 12, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 23, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 0, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 23, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 0, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 23, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 0, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 2, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 2, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 2, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 2518, 2024-02-26T04:59:53.037 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 2506, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 6, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 38, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2024-02-26T04:59:53.038 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 4, 2024-02-26T04:59:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 234, 2024-02-26T04:59:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2024-02-26T04:59:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 38, 2024-02-26T04:59:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2024-02-26T04:59:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T04:59:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2024-02-26T04:59:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 25, 2024-02-26T04:59:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 3, 2024-02-26T04:59:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 22, 2024-02-26T04:59:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 13, 2024-02-26T04:59:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2024-02-26T04:59:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 28, 2024-02-26T04:59:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 4448, 2024-02-26T04:59:53.039 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T04:59:53.040 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T04:59:53.040 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T04:59:53.040 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 2446, 2024-02-26T04:59:53.040 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 1212, 2024-02-26T04:59:53.040 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 213, 2024-02-26T04:59:53.040 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 1243 2024-02-26T04:59:53.040 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.040 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2024-02-26T04:59:53.040 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 11, 2024-02-26T04:59:53.040 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2024-02-26T04:59:53.040 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2024-02-26T04:59:53.040 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2024-02-26T04:59:53.040 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 19, 2024-02-26T04:59:53.040 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 1212, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 1212, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 3, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2024-02-26T04:59:53.041 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2024-02-26T04:59:53.042 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2024-02-26T04:59:53.042 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2024-02-26T04:59:53.042 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.042 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2024-02-26T04:59:53.042 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2024-02-26T04:59:53.042 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 6114, 2024-02-26T04:59:53.042 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2024-02-26T04:59:53.042 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2024-02-26T04:59:53.042 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 6114, 2024-02-26T04:59:53.042 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2024-02-26T04:59:53.042 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2024-02-26T04:59:53.042 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 10, 2024-02-26T04:59:53.042 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2024-02-26T04:59:53.042 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2024-02-26T04:59:53.043 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 10, 2024-02-26T04:59:53.043 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 2, 2024-02-26T04:59:53.043 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2024-02-26T04:59:53.043 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2024-02-26T04:59:53.043 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2024-02-26T04:59:53.043 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 16810742, 2024-02-26T04:59:53.043 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 0, 2024-02-26T04:59:53.043 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2024-02-26T04:59:53.043 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1272, 2024-02-26T04:59:53.043 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2.133971590, 2024-02-26T04:59:53.043 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001677650 2024-02-26T04:59:53.043 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.043 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 0 2024-02-26T04:59:53.043 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.044 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2024-02-26T04:59:53.044 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 23, 2024-02-26T04:59:53.044 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 1238, 2024-02-26T04:59:53.044 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 1215, 2024-02-26T04:59:53.044 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 18, 2024-02-26T04:59:53.044 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 24, 2024-02-26T04:59:53.044 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 6, 2024-02-26T04:59:53.044 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 23, 2024-02-26T04:59:53.044 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 1242, 2024-02-26T04:59:53.044 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 1219, 2024-02-26T04:59:53.044 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 2, 2024-02-26T04:59:53.044 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 1226, 2024-02-26T04:59:53.044 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 1224, 2024-02-26T04:59:53.044 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 61956, 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 215420 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 2783, 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 2482, 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 13, 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1210, 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.942144667, 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000778631 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 28, 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.007617510, 2024-02-26T04:59:53.045 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000272053 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000511223, 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000511223 2024-02-26T04:59:53.046 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000206123, 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000206123 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.047 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 12, 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.017013875, 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001417822 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.049 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.050 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2024-02-26T04:59:53.050 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.050 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.050 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.050 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.050 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2024-02-26T04:59:53.050 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6, 2024-02-26T04:59:53.050 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.013620650, 2024-02-26T04:59:53.050 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002270108 2024-02-26T04:59:53.050 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.050 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2024-02-26T04:59:53.050 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2, 2024-02-26T04:59:53.050 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.002474263, 2024-02-26T04:59:53.050 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001237131 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6, 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.035308910, 2024-02-26T04:59:53.051 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.005884818 2024-02-26T04:59:53.052 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.052 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2024-02-26T04:59:53.052 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.052 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.052 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.052 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.052 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2024-02-26T04:59:53.052 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.052 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.052 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.052 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.052 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2024-02-26T04:59:53.052 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2, 2024-02-26T04:59:53.052 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.001577987, 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000788993 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.053 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.054 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.054 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.054 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2024-02-26T04:59:53.054 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 8, 2024-02-26T04:59:53.054 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.022318214, 2024-02-26T04:59:53.054 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002789776 2024-02-26T04:59:53.054 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.054 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2024-02-26T04:59:53.054 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.054 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.054 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.054 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.054 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2024-02-26T04:59:53.054 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1206, 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1.727772548, 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001432647 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 1, 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 3, 2024-02-26T04:59:53.055 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 2, 2024-02-26T04:59:53.056 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 1, 2024-02-26T04:59:53.056 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2024-02-26T04:59:53.056 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 2164, 2024-02-26T04:59:53.056 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 2164, 2024-02-26T04:59:53.056 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 120, 2024-02-26T04:59:53.056 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2024-02-26T04:59:53.056 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.056 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-02-26T04:59:53.056 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-02-26T04:59:53.056 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-02-26T04:59:53.056 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T04:59:53.056 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-02-26T04:59:53.056 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T04:59:53.057 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T04:59:53.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T04:59:53.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T04:59:53.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T04:59:53.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-02-26T04:59:53.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-02-26T04:59:53.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-02-26T04:59:53.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T04:59:53.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T04:59:53.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-02-26T04:59:53.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-02-26T04:59:53.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-02-26T04:59:53.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-02-26T04:59:53.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T04:59:53.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 291188, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 119, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 528, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 6, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 9464, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 182, 2024-02-26T04:59:53.059 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T04:59:53.060 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-02-26T04:59:53.060 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-02-26T04:59:53.060 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-02-26T04:59:53.060 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 94122, 2024-02-26T04:59:53.060 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 691, 2024-02-26T04:59:53.060 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-02-26T04:59:53.060 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-02-26T04:59:53.060 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-02-26T04:59:53.060 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-02-26T04:59:53.060 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.060 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2024-02-26T04:59:53.060 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2024-02-26T04:59:53.060 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2024-02-26T04:59:53.060 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 6256, 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 12865996, 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 37, 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 6256, 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6256, 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 38.937409710, 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.006224010 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6256, 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 6326 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 6256, 2024-02-26T04:59:53.061 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 62, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 6194, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 0, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 11, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 21, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 1290, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 25, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 4815, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2024-02-26T04:59:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 41, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 22, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2024-02-26T04:59:53.063 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2024-02-26T04:59:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2024-02-26T04:59:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2024-02-26T04:59:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2024-02-26T04:59:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2024-02-26T04:59:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2024-02-26T04:59:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2024-02-26T04:59:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2024-02-26T04:59:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2024-02-26T04:59:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2024-02-26T04:59:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2024-02-26T04:59:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2024-02-26T04:59:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2024-02-26T04:59:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2024-02-26T04:59:53.064 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 93, 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 9, 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 12, 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 12, 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 87, 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 0, 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 14 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 1, 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2024-02-26T04:59:53.065 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 19, 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 7, 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed_ops": 4300, 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 1212, 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:53.066 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-02-26T04:59:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 12745, 2024-02-26T04:59:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 2957222, 2024-02-26T04:59:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 12745, 2024-02-26T04:59:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-26T04:59:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-26T04:59:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 12745, 2024-02-26T04:59:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 2957222, 2024-02-26T04:59:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.067 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:53.068 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.068 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2024-02-26T04:59:53.068 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:53.068 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-02-26T04:59:53.068 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:53.068 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-26T04:59:53.068 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-26T04:59:53.068 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:53.068 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-26T04:59:53.068 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 6256, 2024-02-26T04:59:53.068 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 12998056, 2024-02-26T04:59:53.069 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 6256, 2024-02-26T04:59:53.069 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 12998056, 2024-02-26T04:59:53.069 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:53.069 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.069 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.069 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.069 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:53.069 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.069 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2024-02-26T04:59:53.069 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 6256, 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 6256, 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 6256, 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 6256, 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.070 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 1212, 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 132060, 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 1212, 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 18, 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 132060, 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:53.071 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.072 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.072 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.072 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:53.072 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:53.072 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x55ddb75746e0": { 2024-02-26T04:59:53.072 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:53.072 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-02-26T04:59:53.072 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:53.072 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 6114, 2024-02-26T04:59:53.072 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 12616438, 2024-02-26T04:59:53.072 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:53.072 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 6114, 2024-02-26T04:59:53.072 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-26T04:59:53.072 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-26T04:59:53.073 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 1272, 2024-02-26T04:59:53.073 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 12616438, 2024-02-26T04:59:53.073 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:53.073 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:53.073 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:53.073 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:53.073 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:53.073 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:53.073 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-02-26T04:59:53.625 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-02-26T04:59:53.625 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-02-26T04:59:53.625 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=11 2024-02-26T04:59:53.625 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-02-26T04:59:53.625 DEBUG:tasks.check_counter:Getting stats from e 2024-02-26T04:59:53.626 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph daemon mds.e perf dump 2024-02-26T04:59:53.938 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T04:59:54.319 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:54 smithi156 ceph-mon[35805]: pgmap v430: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-02-26T04:59:54.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:54 smithi191 ceph-mon[36352]: pgmap v430: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-02-26T04:59:54.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:54 smithi060 ceph-mon[32382]: pgmap v430: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-02-26T04:59:55.670 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-02-26T04:59:55.670 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T04:59:55.670 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 338, 2024-02-26T04:59:55.670 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 334, 2024-02-26T04:59:55.671 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 538016, 2024-02-26T04:59:55.671 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 125230, 2024-02-26T04:59:55.671 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 5, 2024-02-26T04:59:55.671 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 1, 2024-02-26T04:59:55.671 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.044075369, 2024-02-26T04:59:55.671 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.016522491, 2024-02-26T04:59:55.671 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.017233474, 2024-02-26T04:59:55.671 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.009167465, 2024-02-26T04:59:55.671 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:55.671 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 333, 2024-02-26T04:59:55.671 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.019128974, 2024-02-26T04:59:55.671 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000057444 2024-02-26T04:59:55.671 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.671 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:55.672 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.672 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.672 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.672 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.672 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 456704, 2024-02-26T04:59:55.672 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 49968 2024-02-26T04:59:55.672 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.672 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T04:59:55.672 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 3, 2024-02-26T04:59:55.672 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 64, 2024-02-26T04:59:55.672 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 716, 2024-02-26T04:59:55.672 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 174812, 2024-02-26T04:59:55.672 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 4, 2024-02-26T04:59:55.673 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 3, 2024-02-26T04:59:55.673 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.004598324, 2024-02-26T04:59:55.673 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.002529177, 2024-02-26T04:59:55.673 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.001299742, 2024-02-26T04:59:55.673 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000054919, 2024-02-26T04:59:55.673 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:55.673 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 62, 2024-02-26T04:59:55.673 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.003627425, 2024-02-26T04:59:55.673 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000058506 2024-02-26T04:59:55.673 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.673 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:55.673 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.673 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.673 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.674 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.674 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-26T04:59:55.674 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 174128 2024-02-26T04:59:55.674 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.674 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T04:59:55.674 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 250, 2024-02-26T04:59:55.674 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 250, 2024-02-26T04:59:55.674 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 68645, 2024-02-26T04:59:55.674 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 76961, 2024-02-26T04:59:55.674 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 4, 2024-02-26T04:59:55.674 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 3, 2024-02-26T04:59:55.674 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.019695248, 2024-02-26T04:59:55.674 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.005443004, 2024-02-26T04:59:55.674 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.006407528, 2024-02-26T04:59:55.675 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.006125513, 2024-02-26T04:59:55.675 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:55.675 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 249, 2024-02-26T04:59:55.675 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.009543335, 2024-02-26T04:59:55.675 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000038326 2024-02-26T04:59:55.675 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.675 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:55.675 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.675 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.675 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.675 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.675 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-26T04:59:55.675 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 448 2024-02-26T04:59:55.675 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.676 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-02-26T04:59:55.676 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2024-02-26T04:59:55.676 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-02-26T04:59:55.676 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.676 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2024-02-26T04:59:55.676 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-02-26T04:59:55.676 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-02-26T04:59:55.676 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 973, 2024-02-26T04:59:55.676 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.025530252, 2024-02-26T04:59:55.676 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000026238 2024-02-26T04:59:55.676 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:55.676 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.676 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2024-02-26T04:59:55.676 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-02-26T04:59:55.677 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-02-26T04:59:55.677 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.677 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.677 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.677 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:55.677 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.677 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-02-26T04:59:55.677 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2024-02-26T04:59:55.677 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2024-02-26T04:59:55.677 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2024-02-26T04:59:55.677 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.677 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.677 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.677 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.678 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2024-02-26T04:59:55.678 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2024-02-26T04:59:55.678 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2024-02-26T04:59:55.678 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2024-02-26T04:59:55.678 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2024-02-26T04:59:55.678 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2024-02-26T04:59:55.678 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2024-02-26T04:59:55.678 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 0, 2024-02-26T04:59:55.678 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 0, 2024-02-26T04:59:55.678 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 0, 2024-02-26T04:59:55.678 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 0, 2024-02-26T04:59:55.678 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 0, 2024-02-26T04:59:55.678 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 0, 2024-02-26T04:59:55.679 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2024-02-26T04:59:55.679 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2024-02-26T04:59:55.679 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 1, 2024-02-26T04:59:55.679 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2024-02-26T04:59:55.679 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2024-02-26T04:59:55.679 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2024-02-26T04:59:55.679 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2024-02-26T04:59:55.679 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2024-02-26T04:59:55.679 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2024-02-26T04:59:55.679 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2024-02-26T04:59:55.679 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2024-02-26T04:59:55.679 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2024-02-26T04:59:55.679 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2024-02-26T04:59:55.679 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2024-02-26T04:59:55.680 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2024-02-26T04:59:55.680 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2024-02-26T04:59:55.680 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2024-02-26T04:59:55.680 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2024-02-26T04:59:55.680 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2024-02-26T04:59:55.680 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 0, 2024-02-26T04:59:55.680 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 0, 2024-02-26T04:59:55.680 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2024-02-26T04:59:55.680 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T04:59:55.680 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2024-02-26T04:59:55.680 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T04:59:55.681 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2024-02-26T04:59:55.681 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2024-02-26T04:59:55.681 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T04:59:55.681 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T04:59:55.681 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2024-02-26T04:59:55.681 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2024-02-26T04:59:55.681 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T04:59:55.681 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2024-02-26T04:59:55.681 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T04:59:55.681 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T04:59:55.681 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T04:59:55.681 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2024-02-26T04:59:55.681 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2024-02-26T04:59:55.681 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2024-02-26T04:59:55.682 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2024-02-26T04:59:55.683 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2024-02-26T04:59:55.683 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2024-02-26T04:59:55.683 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2024-02-26T04:59:55.683 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2024-02-26T04:59:55.683 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2024-02-26T04:59:55.683 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T04:59:55.683 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2024-02-26T04:59:55.683 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2024-02-26T04:59:55.683 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2024-02-26T04:59:55.683 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2024-02-26T04:59:55.683 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2024-02-26T04:59:55.683 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.683 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2024-02-26T04:59:55.683 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2024-02-26T04:59:55.684 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2024-02-26T04:59:55.684 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2024-02-26T04:59:55.684 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2024-02-26T04:59:55.684 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 2, 2024-02-26T04:59:55.684 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2024-02-26T04:59:55.684 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2024-02-26T04:59:55.684 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2024-02-26T04:59:55.684 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2024-02-26T04:59:55.684 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2024-02-26T04:59:55.684 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 2, 2024-02-26T04:59:55.684 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 2, 2024-02-26T04:59:55.684 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2024-02-26T04:59:55.684 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2024-02-26T04:59:55.684 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2024-02-26T04:59:55.685 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 4194892, 2024-02-26T04:59:55.685 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 4194352, 2024-02-26T04:59:55.685 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2024-02-26T04:59:55.685 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.685 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.685 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.685 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.685 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 2 2024-02-26T04:59:55.685 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.685 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2024-02-26T04:59:55.685 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 3, 2024-02-26T04:59:55.685 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 3, 2024-02-26T04:59:55.685 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 0, 2024-02-26T04:59:55.685 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 1, 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 1, 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 0, 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 0, 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 0, 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 0, 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 47468, 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 215420 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2024-02-26T04:59:55.686 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2024-02-26T04:59:55.687 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2024-02-26T04:59:55.687 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2024-02-26T04:59:55.687 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2024-02-26T04:59:55.687 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.687 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.687 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.687 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.687 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2024-02-26T04:59:55.687 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.687 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.687 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.687 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.688 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2024-02-26T04:59:55.688 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.688 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.688 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.688 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.688 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2024-02-26T04:59:55.688 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.688 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.688 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.688 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.688 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2024-02-26T04:59:55.688 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.688 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.689 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.690 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.691 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.691 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.691 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2024-02-26T04:59:55.691 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.691 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.691 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.691 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.692 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2024-02-26T04:59:55.692 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.692 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.692 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.692 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.692 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2024-02-26T04:59:55.692 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.692 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.692 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.692 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.692 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2024-02-26T04:59:55.693 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.693 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.693 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.693 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.693 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2024-02-26T04:59:55.693 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.693 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.693 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.693 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.693 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2024-02-26T04:59:55.693 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.693 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.693 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.693 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.694 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.695 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2024-02-26T04:59:55.695 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.695 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.695 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.695 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.695 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2024-02-26T04:59:55.695 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.695 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.695 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.695 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.695 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2024-02-26T04:59:55.695 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.695 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.695 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.696 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.696 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2024-02-26T04:59:55.696 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.696 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.696 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.696 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.696 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2024-02-26T04:59:55.696 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.696 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.696 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.696 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.696 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2024-02-26T04:59:55.696 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.696 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.697 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 0, 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 0, 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 0, 2024-02-26T04:59:55.698 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 0, 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 0, 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T04:59:55.699 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T04:59:55.700 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T04:59:55.700 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T04:59:55.700 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T04:59:55.700 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-02-26T04:59:55.700 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T04:59:55.700 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T04:59:55.700 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-02-26T04:59:55.700 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-02-26T04:59:55.700 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-02-26T04:59:55.700 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-02-26T04:59:55.700 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T04:59:55.700 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T04:59:55.700 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T04:59:55.701 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T04:59:55.701 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T04:59:55.701 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-02-26T04:59:55.701 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-02-26T04:59:55.701 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-02-26T04:59:55.701 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T04:59:55.701 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T04:59:55.701 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-02-26T04:59:55.701 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-02-26T04:59:55.701 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-02-26T04:59:55.701 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-02-26T04:59:55.701 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T04:59:55.701 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-02-26T04:59:55.701 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 181159, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 44, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 9464, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 182, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-02-26T04:59:55.702 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-02-26T04:59:55.703 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-02-26T04:59:55.703 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 8320, 2024-02-26T04:59:55.703 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 13, 2024-02-26T04:59:55.703 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-02-26T04:59:55.703 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-02-26T04:59:55.703 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-02-26T04:59:55.703 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-02-26T04:59:55.703 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.703 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2024-02-26T04:59:55.703 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2024-02-26T04:59:55.703 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2024-02-26T04:59:55.703 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 490, 2024-02-26T04:59:55.703 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 21, 2024-02-26T04:59:55.703 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 3, 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 490, 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 490, 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.222996626, 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000455095 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 490, 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 491 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 490, 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 490, 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2024-02-26T04:59:55.704 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2024-02-26T04:59:55.705 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2024-02-26T04:59:55.705 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 242, 2024-02-26T04:59:55.705 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2024-02-26T04:59:55.705 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 246, 2024-02-26T04:59:55.705 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2024-02-26T04:59:55.705 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2024-02-26T04:59:55.705 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2024-02-26T04:59:55.705 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2024-02-26T04:59:55.705 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2024-02-26T04:59:55.705 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2024-02-26T04:59:55.705 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2024-02-26T04:59:55.705 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2024-02-26T04:59:55.705 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2024-02-26T04:59:55.705 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 1, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2024-02-26T04:59:55.706 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 93, 2024-02-26T04:59:55.707 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2024-02-26T04:59:55.708 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 4, 2024-02-26T04:59:55.708 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 4, 2024-02-26T04:59:55.708 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 4, 2024-02-26T04:59:55.708 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2024-02-26T04:59:55.708 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2024-02-26T04:59:55.708 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2024-02-26T04:59:55.708 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2024-02-26T04:59:55.708 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2024-02-26T04:59:55.708 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.708 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2024-02-26T04:59:55.708 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2024-02-26T04:59:55.708 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2024-02-26T04:59:55.708 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2024-02-26T04:59:55.708 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed_ops": 0, 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T04:59:55.709 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 587, 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 546617, 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 587, 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 587, 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 546617, 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.710 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:55.711 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.711 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2024-02-26T04:59:55.711 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:55.711 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-02-26T04:59:55.711 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:55.711 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-26T04:59:55.711 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-26T04:59:55.711 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:55.711 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-26T04:59:55.711 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 490, 2024-02-26T04:59:55.711 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 593, 2024-02-26T04:59:55.711 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 490, 2024-02-26T04:59:55.711 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 593, 2024-02-26T04:59:55.711 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:55.712 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.712 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.712 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.712 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:55.712 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.712 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2024-02-26T04:59:55.712 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:55.712 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2024-02-26T04:59:55.712 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:55.712 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-26T04:59:55.712 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-26T04:59:55.712 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:55.712 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-26T04:59:55.712 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 490, 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 490, 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 490, 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 490, 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-26T04:59:55.713 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x55abed826960": { 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:55.714 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:55.715 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:55.716 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-02-26T04:59:56.427 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:56 smithi156 ceph-mon[35805]: pgmap v431: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T04:59:56.486 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-02-26T04:59:56.486 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-02-26T04:59:56.486 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-02-26T04:59:56.486 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-02-26T04:59:56.486 DEBUG:tasks.check_counter:Getting stats from h 2024-02-26T04:59:56.486 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph daemon mds.h perf dump 2024-02-26T04:59:56.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:56 smithi191 ceph-mon[36352]: pgmap v431: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T04:59:56.727 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T04:59:56.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:56 smithi060 ceph-mon[32382]: pgmap v431: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T04:59:58.506 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 04:59:58 smithi156 ceph-mon[35805]: pgmap v432: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:59:58.507 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-02-26T04:59:58.507 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T04:59:58.507 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 938, 2024-02-26T04:59:58.507 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 617, 2024-02-26T04:59:58.507 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 707476, 2024-02-26T04:59:58.507 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 144574, 2024-02-26T04:59:58.507 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 7, 2024-02-26T04:59:58.507 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 3, 2024-02-26T04:59:58.507 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.069339792, 2024-02-26T04:59:58.507 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.027781288, 2024-02-26T04:59:58.507 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.049412765, 2024-02-26T04:59:58.507 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.003392844, 2024-02-26T04:59:58.507 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:58.508 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 617, 2024-02-26T04:59:58.508 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.027919148, 2024-02-26T04:59:58.508 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000045249 2024-02-26T04:59:58.508 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.508 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:58.508 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1263, 2024-02-26T04:59:58.508 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000182102, 2024-02-26T04:59:58.508 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000144 2024-02-26T04:59:58.508 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.508 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 470448, 2024-02-26T04:59:58.508 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 54592 2024-02-26T04:59:58.508 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.508 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 21, 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 83, 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 4395, 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 277512, 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 18, 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 8, 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.009781018, 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.002726492, 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.005805124, 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000216352, 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 79, 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.015621568, 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000197741 2024-02-26T04:59:58.509 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.510 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:58.510 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 10, 2024-02-26T04:59:58.510 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000948, 2024-02-26T04:59:58.510 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000094 2024-02-26T04:59:58.510 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.510 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-26T04:59:58.510 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 175552 2024-02-26T04:59:58.510 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.510 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T04:59:58.510 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 24, 2024-02-26T04:59:58.510 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 24, 2024-02-26T04:59:58.510 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 10566, 2024-02-26T04:59:58.510 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 18497, 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 7, 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.005189913, 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.000513410, 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.004352918, 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000267944, 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 24, 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.019997806, 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000833241 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 20, 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000003380, 2024-02-26T04:59:58.511 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000169 2024-02-26T04:59:58.512 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.512 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T04:59:58.512 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T04:59:58.512 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.512 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-02-26T04:59:58.512 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2024-02-26T04:59:58.512 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-02-26T04:59:58.512 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.512 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2024-02-26T04:59:58.512 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-02-26T04:59:58.512 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-02-26T04:59:58.512 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 36, 2024-02-26T04:59:58.512 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.001472613, 2024-02-26T04:59:58.513 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000040905 2024-02-26T04:59:58.513 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:58.513 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.513 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2024-02-26T04:59:58.513 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-02-26T04:59:58.513 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-02-26T04:59:58.513 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2, 2024-02-26T04:59:58.513 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000013757, 2024-02-26T04:59:58.513 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000006878 2024-02-26T04:59:58.513 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:58.513 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.513 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-02-26T04:59:58.513 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2024-02-26T04:59:58.514 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2024-02-26T04:59:58.514 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2024-02-26T04:59:58.514 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.514 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.514 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.514 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.514 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2024-02-26T04:59:58.514 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2024-02-26T04:59:58.514 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2024-02-26T04:59:58.514 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2024-02-26T04:59:58.514 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 11, 2024-02-26T04:59:58.514 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2024-02-26T04:59:58.514 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2024-02-26T04:59:58.514 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 17, 2024-02-26T04:59:58.515 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 1, 2024-02-26T04:59:58.515 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 16, 2024-02-26T04:59:58.515 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 0, 2024-02-26T04:59:58.515 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 13, 2024-02-26T04:59:58.515 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 8, 2024-02-26T04:59:58.515 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2024-02-26T04:59:58.515 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2024-02-26T04:59:58.515 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 3, 2024-02-26T04:59:58.515 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 9, 2024-02-26T04:59:58.515 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 6, 2024-02-26T04:59:58.515 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2024-02-26T04:59:58.515 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 3, 2024-02-26T04:59:58.515 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2024-02-26T04:59:58.515 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2024-02-26T04:59:58.516 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2024-02-26T04:59:58.516 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2024-02-26T04:59:58.516 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2024-02-26T04:59:58.516 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2024-02-26T04:59:58.516 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2024-02-26T04:59:58.516 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2024-02-26T04:59:58.516 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2024-02-26T04:59:58.516 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2024-02-26T04:59:58.516 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2024-02-26T04:59:58.516 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2024-02-26T04:59:58.516 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 4, 2024-02-26T04:59:58.516 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 234, 2024-02-26T04:59:58.516 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2024-02-26T04:59:58.517 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T04:59:58.517 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2024-02-26T04:59:58.517 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T04:59:58.517 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2024-02-26T04:59:58.517 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2024-02-26T04:59:58.517 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T04:59:58.517 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T04:59:58.517 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2024-02-26T04:59:58.517 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2024-02-26T04:59:58.517 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T04:59:58.517 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2024-02-26T04:59:58.517 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T04:59:58.518 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T04:59:58.518 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T04:59:58.518 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2024-02-26T04:59:58.518 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2024-02-26T04:59:58.518 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2024-02-26T04:59:58.518 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2024-02-26T04:59:58.518 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.518 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2024-02-26T04:59:58.518 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2024-02-26T04:59:58.518 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 15, 2024-02-26T04:59:58.518 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 9, 2024-02-26T04:59:58.518 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 10, 2024-02-26T04:59:58.518 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2024-02-26T04:59:58.519 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2024-02-26T04:59:58.520 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2024-02-26T04:59:58.520 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2024-02-26T04:59:58.520 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.520 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2024-02-26T04:59:58.520 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2024-02-26T04:59:58.520 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 2, 2024-02-26T04:59:58.520 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2024-02-26T04:59:58.520 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2024-02-26T04:59:58.520 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 2, 2024-02-26T04:59:58.520 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2024-02-26T04:59:58.520 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2024-02-26T04:59:58.520 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 2, 2024-02-26T04:59:58.520 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2024-02-26T04:59:58.520 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2024-02-26T04:59:58.521 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 2, 2024-02-26T04:59:58.521 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 2, 2024-02-26T04:59:58.521 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2024-02-26T04:59:58.521 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2024-02-26T04:59:58.521 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2024-02-26T04:59:58.521 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 4194892, 2024-02-26T04:59:58.521 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 0, 2024-02-26T04:59:58.521 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2024-02-26T04:59:58.521 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2024-02-26T04:59:58.521 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.001335561, 2024-02-26T04:59:58.521 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001335561 2024-02-26T04:59:58.521 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.521 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 0 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 21, 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 27, 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 6, 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 20, 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 26, 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 6, 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 17, 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 25, 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 8, 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2024-02-26T04:59:58.522 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 48904, 2024-02-26T04:59:58.523 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207228 2024-02-26T04:59:58.523 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.523 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2024-02-26T04:59:58.523 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2024-02-26T04:59:58.523 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2024-02-26T04:59:58.523 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2024-02-26T04:59:58.523 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2024-02-26T04:59:58.523 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2024-02-26T04:59:58.523 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2024-02-26T04:59:58.523 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.523 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.523 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.524 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.524 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2024-02-26T04:59:58.524 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.524 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.524 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.524 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.524 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2024-02-26T04:59:58.524 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.524 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.524 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.524 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.524 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2024-02-26T04:59:58.524 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.524 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.525 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.525 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.525 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2024-02-26T04:59:58.525 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.525 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.525 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.525 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.525 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2024-02-26T04:59:58.525 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.525 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.525 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.525 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.525 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2024-02-26T04:59:58.525 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.526 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.526 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.526 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.526 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2024-02-26T04:59:58.526 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.526 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.526 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.526 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.526 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2024-02-26T04:59:58.526 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.526 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.526 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.527 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.528 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 04:59:58 smithi191 ceph-mon[36352]: pgmap v432: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:59:58.528 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2024-02-26T04:59:58.529 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.529 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.529 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.529 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.529 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2024-02-26T04:59:58.529 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.529 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.529 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.529 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.529 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2024-02-26T04:59:58.529 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.530 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.530 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.530 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.530 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2024-02-26T04:59:58.530 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.530 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.530 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.530 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.530 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2024-02-26T04:59:58.530 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.530 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.530 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.530 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.530 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.531 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2024-02-26T04:59:58.532 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.532 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.532 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.532 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.532 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2024-02-26T04:59:58.532 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.532 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.532 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.532 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.532 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2024-02-26T04:59:58.532 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.532 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.532 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.533 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.534 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.534 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2024-02-26T04:59:58.534 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.534 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.534 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.534 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.534 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2024-02-26T04:59:58.534 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.534 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.534 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.534 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.534 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2024-02-26T04:59:58.534 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.534 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.535 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.535 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.535 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2024-02-26T04:59:58.535 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2024-02-26T04:59:58.535 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2024-02-26T04:59:58.535 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.535 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.535 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.535 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:58.535 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.535 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2024-02-26T04:59:58.535 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 0, 2024-02-26T04:59:58.535 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 0, 2024-02-26T04:59:58.535 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 0, 2024-02-26T04:59:58.536 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 0, 2024-02-26T04:59:58.536 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2024-02-26T04:59:58.536 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2024-02-26T04:59:58.536 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2024-02-26T04:59:58.536 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 0, 2024-02-26T04:59:58.536 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2024-02-26T04:59:58.536 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.536 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-02-26T04:59:58.536 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-02-26T04:59:58.536 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-02-26T04:59:58.536 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T04:59:58.536 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-02-26T04:59:58.537 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T04:59:58.537 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-02-26T04:59:58.537 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T04:59:58.537 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T04:59:58.537 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T04:59:58.537 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T04:59:58.537 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T04:59:58.537 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-02-26T04:59:58.537 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T04:59:58.537 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T04:59:58.537 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-02-26T04:59:58.537 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-02-26T04:59:58.537 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-02-26T04:59:58.537 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T04:59:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 238215, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 78, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 9464, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 182, 2024-02-26T04:59:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T04:59:58.540 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-02-26T04:59:58.540 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-02-26T04:59:58.540 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-02-26T04:59:58.540 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 83741, 2024-02-26T04:59:58.540 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 331, 2024-02-26T04:59:58.540 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-02-26T04:59:58.540 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-02-26T04:59:58.540 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-02-26T04:59:58.540 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-02-26T04:59:58.540 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.540 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2024-02-26T04:59:58.540 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2024-02-26T04:59:58.541 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2024-02-26T04:59:58.541 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 35, 2024-02-26T04:59:58.541 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 10710, 2024-02-26T04:59:58.541 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 33, 2024-02-26T04:59:58.541 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 35, 2024-02-26T04:59:58.541 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2024-02-26T04:59:58.541 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 35, 2024-02-26T04:59:58.541 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.208996864, 2024-02-26T04:59:58.541 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.005971338 2024-02-26T04:59:58.541 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.541 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2024-02-26T04:59:58.541 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2024-02-26T04:59:58.541 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 35, 2024-02-26T04:59:58.541 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 57 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 35, 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 0, 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 35, 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 0, 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 10, 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 0, 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 1, 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 5, 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2024-02-26T04:59:58.542 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2024-02-26T04:59:58.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 6, 2024-02-26T04:59:58.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2024-02-26T04:59:58.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2024-02-26T04:59:58.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2024-02-26T04:59:58.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 0, 2024-02-26T04:59:58.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 20, 2024-02-26T04:59:58.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2024-02-26T04:59:58.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2024-02-26T04:59:58.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2024-02-26T04:59:58.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2024-02-26T04:59:58.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2024-02-26T04:59:58.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2024-02-26T04:59:58.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T04:59:58.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2024-02-26T04:59:58.544 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 93, 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 4, 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 9, 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 9, 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 14, 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 0, 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 1 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.545 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 1, 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed_ops": 0, 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2024-02-26T04:59:58.546 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2024-02-26T04:59:58.547 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.547 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T04:59:58.547 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:58.547 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-02-26T04:59:58.547 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:58.547 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 979, 2024-02-26T04:59:58.547 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 624951, 2024-02-26T04:59:58.547 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:58.547 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 979, 2024-02-26T04:59:58.547 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-26T04:59:58.547 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-26T04:59:58.547 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 979, 2024-02-26T04:59:58.547 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 624951, 2024-02-26T04:59:58.547 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:58.548 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.548 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.548 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.548 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:58.548 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.548 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2024-02-26T04:59:58.548 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:58.548 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-02-26T04:59:58.548 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:58.548 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-26T04:59:58.548 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-26T04:59:58.548 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:58.548 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-26T04:59:58.548 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 35, 2024-02-26T04:59:58.549 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 10710, 2024-02-26T04:59:58.549 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 35, 2024-02-26T04:59:58.549 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 10710, 2024-02-26T04:59:58.549 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:58.549 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.549 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.549 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.551 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:58.551 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.552 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2024-02-26T04:59:58.552 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:58.552 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2024-02-26T04:59:58.552 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:58.552 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-26T04:59:58.552 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-26T04:59:58.552 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:58.552 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-26T04:59:58.552 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 35, 2024-02-26T04:59:58.552 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 35, 2024-02-26T04:59:58.552 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 35, 2024-02-26T04:59:58.552 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 35, 2024-02-26T04:59:58.552 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:58.552 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2024-02-26T04:59:58.553 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x5591f2fae6e0": { 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 2, 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 588, 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 2, 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-26T04:59:58.554 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-26T04:59:58.555 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 1, 2024-02-26T04:59:58.555 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 588, 2024-02-26T04:59:58.555 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T04:59:58.555 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T04:59:58.555 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T04:59:58.555 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T04:59:58.555 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:58.555 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T04:59:58.555 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-02-26T04:59:58.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 04:59:58 smithi060 ceph-mon[32382]: pgmap v432: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T04:59:59.328 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-02-26T04:59:59.328 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-02-26T04:59:59.328 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-02-26T04:59:59.328 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=15 2024-02-26T04:59:59.328 DEBUG:tasks.check_counter:Getting stats from k 2024-02-26T04:59:59.329 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph daemon mds.k perf dump 2024-02-26T04:59:59.568 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.b/config 2024-02-26T05:00:00.306 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:00 smithi156 ceph-mon[35805]: pgmap v433: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T05:00:00.307 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:00 smithi156 ceph-mon[35805]: overall HEALTH_OK 2024-02-26T05:00:00.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:00 smithi191 ceph-mon[36352]: pgmap v433: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T05:00:00.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:00 smithi191 ceph-mon[36352]: overall HEALTH_OK 2024-02-26T05:00:00.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:00 smithi060 ceph-mon[32382]: pgmap v433: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T05:00:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:00 smithi060 ceph-mon[32382]: overall HEALTH_OK 2024-02-26T05:00:01.355 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-02-26T05:00:01.355 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T05:00:01.355 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 0, 2024-02-26T05:00:01.355 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2, 2024-02-26T05:00:01.355 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 0, 2024-02-26T05:00:01.355 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 0, 2024-02-26T05:00:01.355 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 4, 2024-02-26T05:00:01.356 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 0, 2024-02-26T05:00:01.356 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.001214930, 2024-02-26T05:00:01.356 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T05:00:01.356 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.000358133, 2024-02-26T05:00:01.356 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T05:00:01.356 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:01.356 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T05:00:01.356 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T05:00:01.356 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T05:00:01.356 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T05:00:01.356 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:01.356 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T05:00:01.356 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T05:00:01.357 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T05:00:01.357 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T05:00:01.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T05:00:01.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T05:00:01.357 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T05:00:01.357 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T05:00:01.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 94, 2024-02-26T05:00:01.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 89, 2024-02-26T05:00:01.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 440864, 2024-02-26T05:00:01.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 51376, 2024-02-26T05:00:01.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 2, 2024-02-26T05:00:01.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 1, 2024-02-26T05:00:01.357 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.017565058, 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.005357849, 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.012731543, 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.002006163, 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 89, 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.005526789, 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000062098 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 440864, 2024-02-26T05:00:01.358 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 51376 2024-02-26T05:00:01.359 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T05:00:01.359 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T05:00:01.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1, 2024-02-26T05:00:01.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 65, 2024-02-26T05:00:01.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 160, 2024-02-26T05:00:01.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 88752, 2024-02-26T05:00:01.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 3, 2024-02-26T05:00:01.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 2, 2024-02-26T05:00:01.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.004720186, 2024-02-26T05:00:01.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.002866925, 2024-02-26T05:00:01.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.001002803, 2024-02-26T05:00:01.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T05:00:01.359 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:01.359 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 63, 2024-02-26T05:00:01.360 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.004144955, 2024-02-26T05:00:01.360 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000065792 2024-02-26T05:00:01.360 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T05:00:01.360 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:01.360 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T05:00:01.360 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T05:00:01.360 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T05:00:01.360 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T05:00:01.360 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-26T05:00:01.360 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 88752 2024-02-26T05:00:01.360 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T05:00:01.360 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-02-26T05:00:01.360 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 0, 2024-02-26T05:00:01.360 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-02-26T05:00:01.361 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T05:00:01.362 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T05:00:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-02-26T05:00:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-02-26T05:00:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-02-26T05:00:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-02-26T05:00:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T05:00:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-02-26T05:00:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T05:00:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-02-26T05:00:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 148823, 2024-02-26T05:00:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 27, 2024-02-26T05:00:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2024-02-26T05:00:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2024-02-26T05:00:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-02-26T05:00:01.363 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-02-26T05:00:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-02-26T05:00:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-02-26T05:00:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-02-26T05:00:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-02-26T05:00:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 0, 2024-02-26T05:00:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 0, 2024-02-26T05:00:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T05:00:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-02-26T05:00:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-02-26T05:00:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-02-26T05:00:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 928, 2024-02-26T05:00:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 2, 2024-02-26T05:00:01.364 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 91, 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 425170, 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 91, 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-02-26T05:00:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 91, 2024-02-26T05:00:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 425170, 2024-02-26T05:00:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-02-26T05:00:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-02-26T05:00:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-02-26T05:00:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-02-26T05:00:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T05:00:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-02-26T05:00:01.366 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-02-26T05:00:02.031 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-02-26T05:00:02.031 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-02-26T05:00:02.032 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-02-26T05:00:02.032 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-02-26T05:00:02.032 DEBUG:tasks.check_counter:Getting stats from c 2024-02-26T05:00:02.032 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph daemon mds.c perf dump 2024-02-26T05:00:02.304 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T05:00:02.427 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:02 smithi191 ceph-mon[36352]: pgmap v434: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T05:00:02.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:02 smithi156 ceph-mon[35805]: pgmap v434: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T05:00:02.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:02 smithi060 ceph-mon[32382]: pgmap v434: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T05:00:04.304 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-02-26T05:00:04.304 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T05:00:04.304 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 473, 2024-02-26T05:00:04.304 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 473, 2024-02-26T05:00:04.304 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 4338895, 2024-02-26T05:00:04.304 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 146317, 2024-02-26T05:00:04.304 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 6, 2024-02-26T05:00:04.304 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 2, 2024-02-26T05:00:04.304 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.048813333, 2024-02-26T05:00:04.304 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.014428109, 2024-02-26T05:00:04.304 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.015171636, 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.014120984, 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 472, 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.019904221, 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000042169 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 448 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.305 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 120, 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 182, 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 4259630, 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 214579, 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 5, 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 4, 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.014717369, 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.005468111, 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.005404704, 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.002524261, 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 180, 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.012306660, 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000068370 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.306 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 177504 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 314, 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 310, 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 4704967, 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 117929, 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 3, 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 2, 2024-02-26T05:00:04.307 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.037727281, 2024-02-26T05:00:04.308 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.012589904, 2024-02-26T05:00:04.308 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.016629513, 2024-02-26T05:00:04.308 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.008632086, 2024-02-26T05:00:04.308 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:04.308 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 309, 2024-02-26T05:00:04.308 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.014084194, 2024-02-26T05:00:04.308 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000045579 2024-02-26T05:00:04.308 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.308 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:04.308 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.308 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.308 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.308 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.308 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 457136, 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 50752 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1321, 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.035329444, 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000026744 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.309 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 0, 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 0, 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.310 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 0, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 0, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 1231, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 0, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 1231, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 1231, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 0, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 0, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 0, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 2, 2024-02-26T05:00:04.311 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 0, 2024-02-26T05:00:04.312 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 4, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 234, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 0, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 0, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 0, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T05:00:04.313 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 0, 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 0, 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 0, 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 0 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 0, 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 0, 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-02-26T05:00:04.314 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 0, 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 0, 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-02-26T05:00:04.315 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 0, 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 6114, 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 0, 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 10, 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 2, 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-02-26T05:00:04.316 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 16810742, 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 16809604, 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 6114 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 19, 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 1231, 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 1212, 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 14, 2024-02-26T05:00:04.317 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 20, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 6, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 1231, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 1238, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 7, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 0, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 0, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 0, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 49472, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 215420 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 0, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 0, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 0, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 0, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 0, 2024-02-26T05:00:04.318 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-02-26T05:00:04.319 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-02-26T05:00:04.320 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.321 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.322 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.322 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-02-26T05:00:04.322 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.322 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.322 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.322 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.322 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-02-26T05:00:04.322 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.322 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.322 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.323 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.324 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-02-26T05:00:04.325 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.326 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.326 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.326 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.326 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-02-26T05:00:04.326 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.326 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.326 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.326 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.326 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-02-26T05:00:04.326 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.326 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.326 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.326 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.326 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.327 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-02-26T05:00:04.328 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.328 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.328 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.328 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.328 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-02-26T05:00:04.328 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.328 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.328 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.328 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.329 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-02-26T05:00:04.329 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-02-26T05:00:04.329 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-02-26T05:00:04.329 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.329 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.329 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.329 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:04.329 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.329 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-02-26T05:00:04.329 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 1, 2024-02-26T05:00:04.329 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 3, 2024-02-26T05:00:04.329 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 2, 2024-02-26T05:00:04.329 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 1, 2024-02-26T05:00:04.329 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-02-26T05:00:04.330 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 0, 2024-02-26T05:00:04.330 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 0, 2024-02-26T05:00:04.330 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 1720, 2024-02-26T05:00:04.330 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-02-26T05:00:04.330 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.330 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-02-26T05:00:04.330 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-02-26T05:00:04.330 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-02-26T05:00:04.330 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T05:00:04.330 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-02-26T05:00:04.330 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T05:00:04.330 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-02-26T05:00:04.330 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T05:00:04.330 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T05:00:04.331 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T05:00:04.331 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T05:00:04.331 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T05:00:04.331 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-02-26T05:00:04.331 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T05:00:04.331 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T05:00:04.331 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-02-26T05:00:04.331 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-02-26T05:00:04.331 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-02-26T05:00:04.331 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-02-26T05:00:04.331 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T05:00:04.331 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T05:00:04.331 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T05:00:04.331 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T05:00:04.332 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T05:00:04.332 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-02-26T05:00:04.332 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-02-26T05:00:04.332 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-02-26T05:00:04.332 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T05:00:04.332 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T05:00:04.332 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-02-26T05:00:04.332 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-02-26T05:00:04.332 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-02-26T05:00:04.332 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-02-26T05:00:04.332 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T05:00:04.332 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-02-26T05:00:04.332 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T05:00:04.332 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 189217, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 48, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 9464, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 182, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-02-26T05:00:04.333 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-02-26T05:00:04.334 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-02-26T05:00:04.334 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 925946, 2024-02-26T05:00:04.334 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 5173, 2024-02-26T05:00:04.334 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-02-26T05:00:04.334 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-02-26T05:00:04.334 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-02-26T05:00:04.334 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-02-26T05:00:04.334 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.334 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-02-26T05:00:04.334 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-02-26T05:00:04.334 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-02-26T05:00:04.334 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 804, 2024-02-26T05:00:04.334 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 26, 2024-02-26T05:00:04.334 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 4, 2024-02-26T05:00:04.335 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 804, 2024-02-26T05:00:04.335 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-02-26T05:00:04.335 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 804, 2024-02-26T05:00:04.335 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.436992336, 2024-02-26T05:00:04.335 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000543522 2024-02-26T05:00:04.335 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.335 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-02-26T05:00:04.335 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-02-26T05:00:04.335 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 804, 2024-02-26T05:00:04.335 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 805 2024-02-26T05:00:04.335 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.335 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 804, 2024-02-26T05:00:04.336 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 804, 2024-02-26T05:00:04.336 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 0, 2024-02-26T05:00:04.336 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-02-26T05:00:04.336 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-02-26T05:00:04.336 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 501, 2024-02-26T05:00:04.336 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 0, 2024-02-26T05:00:04.336 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 300, 2024-02-26T05:00:04.336 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 0, 2024-02-26T05:00:04.336 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 0, 2024-02-26T05:00:04.336 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-02-26T05:00:04.336 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-02-26T05:00:04.337 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-02-26T05:00:04.337 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-02-26T05:00:04.337 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 0, 2024-02-26T05:00:04.337 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-02-26T05:00:04.337 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-02-26T05:00:04.337 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-02-26T05:00:04.337 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 2, 2024-02-26T05:00:04.337 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 0, 2024-02-26T05:00:04.337 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-02-26T05:00:04.337 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-02-26T05:00:04.337 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-02-26T05:00:04.337 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-02-26T05:00:04.337 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-02-26T05:00:04.337 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-02-26T05:00:04.338 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T05:00:04.338 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-02-26T05:00:04.338 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-02-26T05:00:04.338 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-02-26T05:00:04.338 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-02-26T05:00:04.338 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-02-26T05:00:04.338 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-02-26T05:00:04.338 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-02-26T05:00:04.338 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-02-26T05:00:04.338 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-02-26T05:00:04.338 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-02-26T05:00:04.338 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-02-26T05:00:04.339 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-02-26T05:00:04.339 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-02-26T05:00:04.339 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-02-26T05:00:04.339 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-02-26T05:00:04.339 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-02-26T05:00:04.339 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-02-26T05:00:04.339 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-02-26T05:00:04.339 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-02-26T05:00:04.339 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 93, 2024-02-26T05:00:04.339 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-02-26T05:00:04.340 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 4, 2024-02-26T05:00:04.340 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 5, 2024-02-26T05:00:04.340 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 5, 2024-02-26T05:00:04.340 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-02-26T05:00:04.340 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-02-26T05:00:04.340 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 0, 2024-02-26T05:00:04.340 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 2, 2024-02-26T05:00:04.340 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 0 2024-02-26T05:00:04.340 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.340 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-02-26T05:00:04.340 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 0, 2024-02-26T05:00:04.340 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-02-26T05:00:04.340 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 0 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 0, 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 0, 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 0, 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.341 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T05:00:04.342 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:04.342 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-26T05:00:04.342 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:04.342 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 903, 2024-02-26T05:00:04.342 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 13214194, 2024-02-26T05:00:04.342 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:04.342 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 903, 2024-02-26T05:00:04.342 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-26T05:00:04.342 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-26T05:00:04.342 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 903, 2024-02-26T05:00:04.342 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 13214194, 2024-02-26T05:00:04.342 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:04.342 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.342 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.343 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.343 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:04.343 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.343 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-02-26T05:00:04.343 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:04.343 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-26T05:00:04.343 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:04.343 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-26T05:00:04.343 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-26T05:00:04.343 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:04.343 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-26T05:00:04.343 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 804, 2024-02-26T05:00:04.343 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 12616558, 2024-02-26T05:00:04.343 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 804, 2024-02-26T05:00:04.344 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 12616558, 2024-02-26T05:00:04.344 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:04.344 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.344 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.344 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.344 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:04.344 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.344 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-02-26T05:00:04.344 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:04.344 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 804, 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 804, 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 804, 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 804, 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:04.345 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.346 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x55d3445ea960": { 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-02-26T05:00:04.347 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-02-26T05:00:04.348 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:04.348 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:04.348 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:04.348 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:04.348 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:04.348 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:04.348 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-02-26T05:00:04.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:04 smithi191 ceph-mon[36352]: pgmap v435: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-02-26T05:00:04.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:04 smithi156 ceph-mon[35805]: pgmap v435: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-02-26T05:00:04.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:04 smithi060 ceph-mon[32382]: pgmap v435: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-02-26T05:00:05.059 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-02-26T05:00:05.059 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-02-26T05:00:05.059 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-02-26T05:00:05.059 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-02-26T05:00:05.059 DEBUG:tasks.check_counter:Getting stats from f 2024-02-26T05:00:05.060 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph daemon mds.f perf dump 2024-02-26T05:00:05.307 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T05:00:06.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:06 smithi191 ceph-mon[36352]: pgmap v436: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T05:00:06.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:06 smithi156 ceph-mon[35805]: pgmap v436: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T05:00:06.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:06 smithi060 ceph-mon[32382]: pgmap v436: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T05:00:07.280 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-02-26T05:00:07.281 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T05:00:07.281 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 99, 2024-02-26T05:00:07.281 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 95, 2024-02-26T05:00:07.281 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 458082, 2024-02-26T05:00:07.281 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 51533, 2024-02-26T05:00:07.281 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 5, 2024-02-26T05:00:07.281 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 1, 2024-02-26T05:00:07.281 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.018868374, 2024-02-26T05:00:07.281 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.006309451, 2024-02-26T05:00:07.281 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.012192699, 2024-02-26T05:00:07.281 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.001867684, 2024-02-26T05:00:07.282 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:07.282 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 94, 2024-02-26T05:00:07.282 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.005923461, 2024-02-26T05:00:07.282 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000063015 2024-02-26T05:00:07.282 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.282 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:07.282 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.282 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.282 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.282 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.282 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 457248, 2024-02-26T05:00:07.282 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 51216 2024-02-26T05:00:07.282 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.283 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T05:00:07.283 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 142, 2024-02-26T05:00:07.283 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 203, 2024-02-26T05:00:07.283 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 36908, 2024-02-26T05:00:07.283 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 222340, 2024-02-26T05:00:07.283 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 5, 2024-02-26T05:00:07.283 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 4, 2024-02-26T05:00:07.283 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.017925156, 2024-02-26T05:00:07.283 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.006245938, 2024-02-26T05:00:07.283 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.004876198, 2024-02-26T05:00:07.283 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.004943014, 2024-02-26T05:00:07.283 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:07.283 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 202, 2024-02-26T05:00:07.284 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.011063408, 2024-02-26T05:00:07.284 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000054769 2024-02-26T05:00:07.284 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.284 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:07.284 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.284 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.284 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.284 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.284 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-02-26T05:00:07.284 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 179728 2024-02-26T05:00:07.284 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.284 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T05:00:07.284 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 510, 2024-02-26T05:00:07.284 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 512, 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 153273, 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 158668, 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 4, 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 3, 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.048313604, 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.015209621, 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.013116957, 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.015466222, 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 510, 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.021595906, 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000042344 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:07.285 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.286 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.286 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.286 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.286 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T05:00:07.286 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T05:00:07.286 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.286 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-02-26T05:00:07.286 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-02-26T05:00:07.286 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-02-26T05:00:07.286 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.286 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-02-26T05:00:07.286 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-02-26T05:00:07.286 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-02-26T05:00:07.287 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1130, 2024-02-26T05:00:07.287 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.032910270, 2024-02-26T05:00:07.287 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000029124 2024-02-26T05:00:07.287 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:07.287 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.287 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-02-26T05:00:07.287 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-02-26T05:00:07.287 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-02-26T05:00:07.287 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.287 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.287 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.287 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:07.287 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.288 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-02-26T05:00:07.288 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 0, 2024-02-26T05:00:07.288 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 0, 2024-02-26T05:00:07.288 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-02-26T05:00:07.288 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.288 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.288 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.288 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.288 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-02-26T05:00:07.288 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 0, 2024-02-26T05:00:07.288 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-02-26T05:00:07.288 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-02-26T05:00:07.288 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 0, 2024-02-26T05:00:07.288 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 0, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 0, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 0, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 0, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 0, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 0, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 0, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 1, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 0, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 0, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 0, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 0, 2024-02-26T05:00:07.289 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-02-26T05:00:07.290 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-02-26T05:00:07.290 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-02-26T05:00:07.290 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-02-26T05:00:07.290 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-02-26T05:00:07.290 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-02-26T05:00:07.290 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-02-26T05:00:07.290 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 0, 2024-02-26T05:00:07.290 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 0, 2024-02-26T05:00:07.290 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-02-26T05:00:07.290 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 0, 2024-02-26T05:00:07.291 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-02-26T05:00:07.291 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 0, 2024-02-26T05:00:07.291 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 0, 2024-02-26T05:00:07.291 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 0, 2024-02-26T05:00:07.291 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T05:00:07.291 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-02-26T05:00:07.291 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T05:00:07.291 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-02-26T05:00:07.291 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-02-26T05:00:07.291 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T05:00:07.291 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T05:00:07.292 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 0, 2024-02-26T05:00:07.292 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-02-26T05:00:07.292 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T05:00:07.292 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 0, 2024-02-26T05:00:07.292 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T05:00:07.292 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T05:00:07.292 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T05:00:07.292 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 0, 2024-02-26T05:00:07.292 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 0, 2024-02-26T05:00:07.292 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 0, 2024-02-26T05:00:07.292 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 0 2024-02-26T05:00:07.292 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.292 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-02-26T05:00:07.292 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-02-26T05:00:07.293 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-02-26T05:00:07.293 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-02-26T05:00:07.293 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 0, 2024-02-26T05:00:07.293 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 0, 2024-02-26T05:00:07.293 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-02-26T05:00:07.293 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-02-26T05:00:07.293 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-02-26T05:00:07.293 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 0, 2024-02-26T05:00:07.293 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 0, 2024-02-26T05:00:07.293 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-02-26T05:00:07.293 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-02-26T05:00:07.293 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-02-26T05:00:07.293 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-02-26T05:00:07.294 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-02-26T05:00:07.294 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-02-26T05:00:07.294 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-02-26T05:00:07.294 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T05:00:07.294 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-02-26T05:00:07.294 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-02-26T05:00:07.294 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-02-26T05:00:07.294 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-02-26T05:00:07.294 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-02-26T05:00:07.294 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.294 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-02-26T05:00:07.294 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-02-26T05:00:07.294 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 0, 2024-02-26T05:00:07.295 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-02-26T05:00:07.295 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-02-26T05:00:07.295 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 3, 2024-02-26T05:00:07.295 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-02-26T05:00:07.295 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-02-26T05:00:07.295 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 0, 2024-02-26T05:00:07.295 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-02-26T05:00:07.295 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-02-26T05:00:07.295 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 2, 2024-02-26T05:00:07.295 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 2, 2024-02-26T05:00:07.295 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-02-26T05:00:07.295 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-02-26T05:00:07.295 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-02-26T05:00:07.295 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 4195200, 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 4194892, 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 3 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 3, 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 3, 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 0, 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 1, 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 1, 2024-02-26T05:00:07.296 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 0, 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 0, 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 0, 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 0, 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 0, 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 0, 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 0, 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 48512, 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 215420 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 0, 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 0, 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 0, 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 0, 2024-02-26T05:00:07.297 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 0, 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.298 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.299 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-02-26T05:00:07.300 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.300 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.300 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.300 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.300 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-02-26T05:00:07.300 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.300 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.300 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.300 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.300 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-02-26T05:00:07.300 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.300 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.300 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.300 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.301 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.302 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-02-26T05:00:07.302 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.302 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.302 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.302 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.303 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.304 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-02-26T05:00:07.304 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.304 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.304 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.304 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.304 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-02-26T05:00:07.304 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.304 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.304 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.304 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.304 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-02-26T05:00:07.304 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.304 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.304 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.305 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.305 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-02-26T05:00:07.305 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.305 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.305 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.305 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.305 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-02-26T05:00:07.305 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.305 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.305 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.305 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.305 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-02-26T05:00:07.305 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.305 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.306 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.308 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.308 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-02-26T05:00:07.308 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.308 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.308 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.308 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.308 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-02-26T05:00:07.308 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-02-26T05:00:07.308 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-02-26T05:00:07.308 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.308 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.308 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.308 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:07.308 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 1, 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 1, 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 0, 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 1, 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 0, 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 0, 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 1722, 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T05:00:07.309 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-02-26T05:00:07.310 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T05:00:07.310 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-02-26T05:00:07.310 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T05:00:07.310 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T05:00:07.310 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T05:00:07.310 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T05:00:07.310 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T05:00:07.310 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-02-26T05:00:07.310 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T05:00:07.310 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T05:00:07.310 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-02-26T05:00:07.310 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-02-26T05:00:07.311 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-02-26T05:00:07.312 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-02-26T05:00:07.312 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T05:00:07.312 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-02-26T05:00:07.312 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T05:00:07.312 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-02-26T05:00:07.312 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 189279, 2024-02-26T05:00:07.312 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 49, 2024-02-26T05:00:07.312 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-02-26T05:00:07.312 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-02-26T05:00:07.312 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-02-26T05:00:07.312 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-02-26T05:00:07.312 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-02-26T05:00:07.312 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-02-26T05:00:07.312 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-02-26T05:00:07.313 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-02-26T05:00:07.313 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 9464, 2024-02-26T05:00:07.313 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 182, 2024-02-26T05:00:07.313 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T05:00:07.313 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-02-26T05:00:07.313 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-02-26T05:00:07.313 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-02-26T05:00:07.313 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 8320, 2024-02-26T05:00:07.313 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 13, 2024-02-26T05:00:07.313 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-02-26T05:00:07.313 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-02-26T05:00:07.313 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-02-26T05:00:07.313 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-02-26T05:00:07.314 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.314 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-02-26T05:00:07.314 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-02-26T05:00:07.314 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-02-26T05:00:07.314 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 648, 2024-02-26T05:00:07.314 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 21, 2024-02-26T05:00:07.314 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 3, 2024-02-26T05:00:07.314 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 648, 2024-02-26T05:00:07.314 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-02-26T05:00:07.314 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 648, 2024-02-26T05:00:07.314 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.320994376, 2024-02-26T05:00:07.314 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000495361 2024-02-26T05:00:07.314 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 648, 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 649 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 648, 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 648, 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 0, 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 388, 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 0, 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 258, 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 0, 2024-02-26T05:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 0, 2024-02-26T05:00:07.316 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-02-26T05:00:07.316 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-02-26T05:00:07.316 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-02-26T05:00:07.316 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-02-26T05:00:07.316 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 0, 2024-02-26T05:00:07.316 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-02-26T05:00:07.316 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-02-26T05:00:07.316 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-02-26T05:00:07.316 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 1, 2024-02-26T05:00:07.316 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 0, 2024-02-26T05:00:07.316 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-02-26T05:00:07.316 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-02-26T05:00:07.316 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-02-26T05:00:07.316 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-02-26T05:00:07.317 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-02-26T05:00:07.317 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-02-26T05:00:07.317 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T05:00:07.317 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-02-26T05:00:07.317 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-02-26T05:00:07.317 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-02-26T05:00:07.317 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-02-26T05:00:07.317 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-02-26T05:00:07.317 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-02-26T05:00:07.317 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-02-26T05:00:07.317 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-02-26T05:00:07.317 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-02-26T05:00:07.317 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-02-26T05:00:07.317 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-02-26T05:00:07.318 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-02-26T05:00:07.318 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-02-26T05:00:07.318 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-02-26T05:00:07.318 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-02-26T05:00:07.318 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-02-26T05:00:07.318 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-02-26T05:00:07.318 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-02-26T05:00:07.318 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-02-26T05:00:07.318 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 93, 2024-02-26T05:00:07.318 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-02-26T05:00:07.318 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 4, 2024-02-26T05:00:07.318 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 5, 2024-02-26T05:00:07.318 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 5, 2024-02-26T05:00:07.318 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 0, 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 2, 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 0 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 0, 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 0 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.319 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-02-26T05:00:07.320 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-02-26T05:00:07.320 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T05:00:07.320 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-02-26T05:00:07.320 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 0, 2024-02-26T05:00:07.320 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 0, 2024-02-26T05:00:07.320 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 0, 2024-02-26T05:00:07.320 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-02-26T05:00:07.320 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.320 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T05:00:07.320 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:07.320 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-26T05:00:07.320 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:07.320 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 747, 2024-02-26T05:00:07.320 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 572999, 2024-02-26T05:00:07.321 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:07.321 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 747, 2024-02-26T05:00:07.321 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-26T05:00:07.321 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-26T05:00:07.321 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 747, 2024-02-26T05:00:07.321 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 572999, 2024-02-26T05:00:07.321 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:07.321 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.321 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.321 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.321 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:07.321 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.321 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 648, 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 901, 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 648, 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 901, 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.322 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.323 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:07.323 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.323 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-02-26T05:00:07.323 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:07.323 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-02-26T05:00:07.323 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:07.323 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-26T05:00:07.323 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-26T05:00:07.323 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:07.323 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-26T05:00:07.323 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 648, 2024-02-26T05:00:07.323 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 648, 2024-02-26T05:00:07.323 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 648, 2024-02-26T05:00:07.324 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 648, 2024-02-26T05:00:07.324 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:07.324 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.324 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.324 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.324 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:07.324 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.324 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-02-26T05:00:07.324 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:07.324 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-02-26T05:00:07.324 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:07.324 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-26T05:00:07.324 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x557476cf8960": { 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-02-26T05:00:07.325 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:07.326 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-02-26T05:00:08.060 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-02-26T05:00:08.060 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-02-26T05:00:08.061 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-02-26T05:00:08.061 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-02-26T05:00:08.061 DEBUG:tasks.check_counter:Getting stats from i 2024-02-26T05:00:08.061 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph daemon mds.i perf dump 2024-02-26T05:00:08.314 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T05:00:08.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:08 smithi191 ceph-mon[36352]: pgmap v437: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T05:00:08.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:08 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T05:00:08.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:08 smithi156 ceph-mon[35805]: pgmap v437: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T05:00:08.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:08 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T05:00:08.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:08 smithi060 ceph-mon[32382]: pgmap v437: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T05:00:08.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:08 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-26T05:00:09.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:09 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T05:00:09.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:09 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T05:00:09.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:09 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T05:00:09.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:09 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T05:00:09.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:09 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T05:00:09.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:09 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T05:00:09.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:09 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-26T05:00:09.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:09 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-26T05:00:09.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:09 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' 2024-02-26T05:00:10.176 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-02-26T05:00:10.176 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T05:00:10.176 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 26, 2024-02-26T05:00:10.176 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 28, 2024-02-26T05:00:10.176 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 6301, 2024-02-26T05:00:10.177 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 49095, 2024-02-26T05:00:10.177 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 13, 2024-02-26T05:00:10.177 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 2, 2024-02-26T05:00:10.177 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.007162049, 2024-02-26T05:00:10.177 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.000971257, 2024-02-26T05:00:10.177 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.003987594, 2024-02-26T05:00:10.177 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.000437452, 2024-02-26T05:00:10.177 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:10.177 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 26, 2024-02-26T05:00:10.177 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.017153726, 2024-02-26T05:00:10.177 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000659758 2024-02-26T05:00:10.177 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.177 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:10.177 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 8, 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000003605, 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000450 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 22, 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 84, 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 5109, 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 251090, 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 13, 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 7, 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.008256592, 2024-02-26T05:00:10.178 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.002932415, 2024-02-26T05:00:10.179 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.003994138, 2024-02-26T05:00:10.179 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.000416573, 2024-02-26T05:00:10.179 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:10.179 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 82, 2024-02-26T05:00:10.179 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.013374900, 2024-02-26T05:00:10.179 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000163108 2024-02-26T05:00:10.179 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.179 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:10.179 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 6, 2024-02-26T05:00:10.179 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000625, 2024-02-26T05:00:10.179 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000104 2024-02-26T05:00:10.179 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.179 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-26T05:00:10.179 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 180704 2024-02-26T05:00:10.180 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.180 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T05:00:10.180 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 937, 2024-02-26T05:00:10.180 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 622, 2024-02-26T05:00:10.180 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 706746, 2024-02-26T05:00:10.180 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 170855, 2024-02-26T05:00:10.180 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 6, 2024-02-26T05:00:10.180 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 5, 2024-02-26T05:00:10.180 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.067596246, 2024-02-26T05:00:10.180 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.024805864, 2024-02-26T05:00:10.180 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.051347429, 2024-02-26T05:00:10.180 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.003452720, 2024-02-26T05:00:10.180 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:10.180 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 622, 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.024728508, 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000039756 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1290, 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000214795, 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000166 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 466720, 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 55664 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-02-26T05:00:10.181 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 48, 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.001342873, 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000027976 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2, 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000008418, 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000004209 2024-02-26T05:00:10.182 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 2, 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 1, 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1, 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000256781, 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000256781 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 1, 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 11, 2024-02-26T05:00:10.183 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-02-26T05:00:10.184 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-02-26T05:00:10.184 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 17, 2024-02-26T05:00:10.184 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 1, 2024-02-26T05:00:10.184 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 16, 2024-02-26T05:00:10.184 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-02-26T05:00:10.184 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 13, 2024-02-26T05:00:10.184 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 8, 2024-02-26T05:00:10.184 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 1, 2024-02-26T05:00:10.184 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 1, 2024-02-26T05:00:10.184 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 3, 2024-02-26T05:00:10.184 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 9, 2024-02-26T05:00:10.184 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 5, 2024-02-26T05:00:10.184 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 1, 2024-02-26T05:00:10.184 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 3, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 1000, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 1, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 0, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 0, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 0, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 4, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 234, 2024-02-26T05:00:10.185 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-26T05:00:10.186 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 0, 2024-02-26T05:00:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 0, 2024-02-26T05:00:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 0, 2024-02-26T05:00:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 0 2024-02-26T05:00:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-02-26T05:00:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-02-26T05:00:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 13, 2024-02-26T05:00:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 8, 2024-02-26T05:00:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 9, 2024-02-26T05:00:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 0, 2024-02-26T05:00:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-02-26T05:00:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-02-26T05:00:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-02-26T05:00:10.187 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 0, 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 0, 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-02-26T05:00:10.188 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 3, 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 3, 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 2, 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 2, 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 2, 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-02-26T05:00:10.189 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-02-26T05:00:10.190 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-02-26T05:00:10.190 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 4195200, 2024-02-26T05:00:10.190 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 0, 2024-02-26T05:00:10.190 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-02-26T05:00:10.190 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2, 2024-02-26T05:00:10.190 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.003350385, 2024-02-26T05:00:10.190 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.001675192 2024-02-26T05:00:10.190 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.190 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 0 2024-02-26T05:00:10.190 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.190 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-02-26T05:00:10.190 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 21, 2024-02-26T05:00:10.190 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 27, 2024-02-26T05:00:10.190 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 6, 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 20, 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 25, 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 5, 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 17, 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 25, 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 8, 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 1, 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 1, 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 0, 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 51360, 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 207228 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 2, 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 0, 2024-02-26T05:00:10.191 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 2, 2024-02-26T05:00:10.192 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 2, 2024-02-26T05:00:10.192 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 0, 2024-02-26T05:00:10.192 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-02-26T05:00:10.192 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.192 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.192 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.192 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.192 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-02-26T05:00:10.192 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.192 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.192 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.192 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.192 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-02-26T05:00:10.192 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.193 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.193 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.193 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.193 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-02-26T05:00:10.193 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.193 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.193 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.193 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.193 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-02-26T05:00:10.193 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.193 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1, 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000256781, 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000256781 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.194 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.195 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.195 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.195 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-02-26T05:00:10.195 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.195 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.195 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.195 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.195 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-02-26T05:00:10.195 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.195 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.195 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.195 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.195 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-02-26T05:00:10.195 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.196 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.196 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.196 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.196 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-02-26T05:00:10.196 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.196 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.196 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.196 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.197 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-02-26T05:00:10.197 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.197 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.197 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.197 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.197 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-02-26T05:00:10.197 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.197 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.197 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.197 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.197 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-02-26T05:00:10.197 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.198 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.198 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.198 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.198 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-02-26T05:00:10.198 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.198 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.198 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.198 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.198 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-02-26T05:00:10.198 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.198 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.198 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.198 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.198 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-02-26T05:00:10.199 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.199 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.199 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.199 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.199 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-02-26T05:00:10.199 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.199 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.199 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.199 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.199 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-02-26T05:00:10.199 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.199 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.199 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.199 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.200 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-02-26T05:00:10.200 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.200 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.200 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.200 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.200 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-02-26T05:00:10.200 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.200 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.200 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.200 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.200 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-02-26T05:00:10.200 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.200 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.200 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.201 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.202 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 1, 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 1, 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 0, 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 1, 2024-02-26T05:00:10.203 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-02-26T05:00:10.204 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 1, 2024-02-26T05:00:10.204 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 1, 2024-02-26T05:00:10.204 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 137, 2024-02-26T05:00:10.204 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-02-26T05:00:10.204 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.204 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-02-26T05:00:10.204 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-02-26T05:00:10.204 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-02-26T05:00:10.204 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T05:00:10.204 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-02-26T05:00:10.204 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T05:00:10.204 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-02-26T05:00:10.204 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T05:00:10.204 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T05:00:10.205 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 254857, 2024-02-26T05:00:10.206 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 86, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 9464, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 182, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-02-26T05:00:10.207 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 84141, 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 337, 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 38, 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 11319, 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 28, 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 38, 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-02-26T05:00:10.208 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 38, 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.222996115, 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.005868318 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 38, 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 61 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 38, 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 0, 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 38, 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 0, 2024-02-26T05:00:10.209 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 10, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 0, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 2, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 6, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 6, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 0, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 20, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-02-26T05:00:10.210 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-02-26T05:00:10.211 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-02-26T05:00:10.212 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-02-26T05:00:10.212 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-02-26T05:00:10.212 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-02-26T05:00:10.212 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-02-26T05:00:10.212 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-02-26T05:00:10.212 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-02-26T05:00:10.212 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-02-26T05:00:10.212 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-02-26T05:00:10.212 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-02-26T05:00:10.212 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 93, 2024-02-26T05:00:10.212 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-02-26T05:00:10.212 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 4, 2024-02-26T05:00:10.212 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 8, 2024-02-26T05:00:10.212 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 8, 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 16, 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 0, 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 1 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 1, 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 0 2024-02-26T05:00:10.213 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 0, 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 0, 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 0, 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 0, 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 981, 2024-02-26T05:00:10.214 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 620706, 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 981, 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 981, 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 620706, 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-26T05:00:10.215 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 38, 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 11319, 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 38, 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 11319, 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.216 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 38, 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 38, 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 38, 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 38, 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.217 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-02-26T05:00:10.218 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x55c908afe6e0": { 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 3, 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 896, 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 3, 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-26T05:00:10.219 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-26T05:00:10.220 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2, 2024-02-26T05:00:10.220 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 896, 2024-02-26T05:00:10.220 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:10.220 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:10.220 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:10.220 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:10.220 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:10.220 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:10.220 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-02-26T05:00:10.475 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:10 smithi191 ceph-mon[36352]: pgmap v438: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T05:00:10.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:10 smithi060 ceph-mon[32382]: pgmap v438: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T05:00:10.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:10 smithi156 ceph-mon[35805]: pgmap v438: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T05:00:10.967 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-02-26T05:00:10.967 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-02-26T05:00:10.967 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-02-26T05:00:10.967 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=13 2024-02-26T05:00:10.967 DEBUG:tasks.check_counter:Getting stats from l 2024-02-26T05:00:10.968 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33 shell --fsid f12e2206-d461-11ee-95c3-87774f69a715 -- ceph daemon mds.l perf dump 2024-02-26T05:00:11.209 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/mon.c/config 2024-02-26T05:00:12.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:12 smithi191 ceph-mon[36352]: pgmap v439: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T05:00:12.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:12 smithi156 ceph-mon[35805]: pgmap v439: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T05:00:12.729 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:12 smithi060 ceph-mon[32382]: pgmap v439: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T05:00:13.215 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-02-26T05:00:13.216 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-02-26T05:00:13.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 97, 2024-02-26T05:00:13.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 92, 2024-02-26T05:00:13.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 441680, 2024-02-26T05:00:13.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 53248, 2024-02-26T05:00:13.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 4, 2024-02-26T05:00:13.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 0, 2024-02-26T05:00:13.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.018016514, 2024-02-26T05:00:13.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.005777684, 2024-02-26T05:00:13.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.010973884, 2024-02-26T05:00:13.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.002303630, 2024-02-26T05:00:13.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 92, 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.005307379, 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000057688 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 441680, 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 53248 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 1, 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 68, 2024-02-26T05:00:13.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 160, 2024-02-26T05:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 89280, 2024-02-26T05:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 3, 2024-02-26T05:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 2, 2024-02-26T05:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.004384750, 2024-02-26T05:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.002905835, 2024-02-26T05:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.000665916, 2024-02-26T05:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T05:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 66, 2024-02-26T05:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.004393856, 2024-02-26T05:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000066573 2024-02-26T05:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 89280 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 0, 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 2, 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 0, 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 0, 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 2, 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 1, 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.000356853, 2024-02-26T05:00:13.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.000000000, 2024-02-26T05:00:13.220 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.000303442, 2024-02-26T05:00:13.220 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-26T05:00:13.220 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-02-26T05:00:13.220 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:13.220 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:13.220 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:13.220 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:13.220 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-02-26T05:00:13.220 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:13.220 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:13.220 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:13.220 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:13.220 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-26T05:00:13.221 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 0 2024-02-26T05:00:13.221 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:13.221 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-02-26T05:00:13.221 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 0, 2024-02-26T05:00:13.221 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-02-26T05:00:13.221 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:13.221 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-02-26T05:00:13.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-02-26T05:00:13.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-02-26T05:00:13.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-02-26T05:00:13.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-02-26T05:00:13.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-02-26T05:00:13.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-02-26T05:00:13.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-26T05:00:13.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-02-26T05:00:13.223 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 148880, 2024-02-26T05:00:13.224 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 29, 2024-02-26T05:00:13.224 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-02-26T05:00:13.224 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-02-26T05:00:13.224 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-02-26T05:00:13.224 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-02-26T05:00:13.224 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-02-26T05:00:13.224 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-02-26T05:00:13.224 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-02-26T05:00:13.224 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-02-26T05:00:13.224 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 0, 2024-02-26T05:00:13.224 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 0, 2024-02-26T05:00:13.224 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-02-26T05:00:13.224 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-02-26T05:00:13.224 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 928, 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 2, 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 94, 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 425560, 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-02-26T05:00:13.225 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 94, 2024-02-26T05:00:13.226 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-02-26T05:00:13.226 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-02-26T05:00:13.226 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 94, 2024-02-26T05:00:13.226 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 425560, 2024-02-26T05:00:13.226 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-02-26T05:00:13.226 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-02-26T05:00:13.226 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-02-26T05:00:13.226 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-02-26T05:00:13.226 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:13.226 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-02-26T05:00:13.226 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-02-26T05:00:13.978 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-02-26T05:00:13.978 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-02-26T05:00:13.978 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-02-26T05:00:13.978 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-02-26T05:00:13.979 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-26T05:00:13.989 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-26T05:00:13.989 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T05:00:13.989 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T05:00:14.011 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-02-26T05:00:14.011 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T05:00:14.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T05:00:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.224+0000 7f405fade640 1 -- 172.21.15.60:0/3777884602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4058100d90 msgr2=0x7f4058101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.224+0000 7f405fade640 1 --2- 172.21.15.60:0/3777884602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4058100d90 0x7f4058101210 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4050009ca0 tx=0x7f405002c0f0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.224+0000 7f405fade640 1 -- 172.21.15.60:0/3777884602 shutdown_connections 2024-02-26T05:00:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.224+0000 7f405fade640 1 --2- 172.21.15.60:0/3777884602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4058100d90 0x7f4058101210 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.224+0000 7f405fade640 1 --2- 172.21.15.60:0/3777884602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40581003f0 0x7f4058100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.224+0000 7f405fade640 1 --2- 172.21.15.60:0/3777884602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40580ffa90 0x7f40580ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.224+0000 7f405fade640 1 -- 172.21.15.60:0/3777884602 >> 172.21.15.60:0/3777884602 conn(0x7f40580fb6b0 msgr2=0x7f40580fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.224+0000 7f405fade640 1 -- 172.21.15.60:0/3777884602 shutdown_connections 2024-02-26T05:00:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.224+0000 7f405fade640 1 -- 172.21.15.60:0/3777884602 wait complete. 2024-02-26T05:00:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.224+0000 7f405fade640 1 Processor -- start 2024-02-26T05:00:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.224+0000 7f405fade640 1 -- start start 2024-02-26T05:00:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405fade640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40580ffa90 0x7f4058199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405fade640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40581003f0 0x7f4058199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405fade640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4058100d90 0x7f40581a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405fade640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40581a0eb0 con 0x7f40580ffa90 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405fade640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40581a1020 con 0x7f4058100d90 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405fade640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40581a11e0 con 0x7f40581003f0 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405d87a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40580ffa90 0x7f4058199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405d87a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40580ffa90 0x7f4058199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58834/0 (socket says 172.21.15.60:58834) 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405d87a640 1 -- 172.21.15.60:0/3432121961 learned_addr learned my addr 172.21.15.60:0/3432121961 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405d079640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40581003f0 0x7f4058199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405e07b640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4058100d90 0x7f40581a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405d87a640 1 -- 172.21.15.60:0/3432121961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40581003f0 msgr2=0x7f4058199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405d87a640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40581003f0 0x7f4058199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405d87a640 1 -- 172.21.15.60:0/3432121961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4058100d90 msgr2=0x7f40581a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405d87a640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4058100d90 0x7f40581a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405d87a640 1 -- 172.21.15.60:0/3432121961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4050009900 con 0x7f40580ffa90 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405e07b640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4058100d90 0x7f40581a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405d87a640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40580ffa90 0x7f4058199410 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f405400e9f0 tx=0x7f405400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:14.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f404effd640 1 -- 172.21.15.60:0/3432121961 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054009700 con 0x7f40580ffa90 2024-02-26T05:00:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f404effd640 1 -- 172.21.15.60:0/3432121961 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f405400cd80 con 0x7f40580ffa90 2024-02-26T05:00:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f404effd640 1 -- 172.21.15.60:0/3432121961 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054018920 con 0x7f40580ffa90 2024-02-26T05:00:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405fade640 1 -- 172.21.15.60:0/3432121961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40581a14c0 con 0x7f40580ffa90 2024-02-26T05:00:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.225+0000 7f405fade640 1 -- 172.21.15.60:0/3432121961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40581a19d0 con 0x7f40580ffa90 2024-02-26T05:00:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.227+0000 7f405fade640 1 -- 172.21.15.60:0/3432121961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405810a7e0 con 0x7f40580ffa90 2024-02-26T05:00:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.228+0000 7f404effd640 1 -- 172.21.15.60:0/3432121961 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f4054018a80 con 0x7f40580ffa90 2024-02-26T05:00:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.228+0000 7f404effd640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f40440a20c0 0x7f40440a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.229+0000 7f405d079640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f40440a20c0 0x7f40440a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.229+0000 7f404effd640 1 -- 172.21.15.60:0/3432121961 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f4054014070 con 0x7f40580ffa90 2024-02-26T05:00:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.229+0000 7f405d079640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f40440a20c0 0x7f40440a4580 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f405819a8d0 tx=0x7f4048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:14.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.232+0000 7f404effd640 1 -- 172.21.15.60:0/3432121961 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f40540975b0 con 0x7f40580ffa90 2024-02-26T05:00:14.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.415+0000 7f405fade640 1 -- 172.21.15.60:0/3432121961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f405810edf0 con 0x7f40580ffa90 2024-02-26T05:00:14.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.417+0000 7f404effd640 1 -- 172.21.15.60:0/3432121961 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24972 (secure 0 0 0) 0x7f405409ac70 con 0x7f40580ffa90 2024-02-26T05:00:14.420 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T05:00:14.420 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:57:21.168776+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T05:00:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.419+0000 7f405fade640 1 -- 172.21.15.60:0/3432121961 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f40440a20c0 msgr2=0x7f40440a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.419+0000 7f405fade640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f40440a20c0 0x7f40440a4580 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f405819a8d0 tx=0x7f4048005f90 comp rx=0 tx=0).stop 2024-02-26T05:00:14.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.419+0000 7f405fade640 1 -- 172.21.15.60:0/3432121961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40580ffa90 msgr2=0x7f4058199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:14.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.419+0000 7f405fade640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40580ffa90 0x7f4058199410 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f405400e9f0 tx=0x7f405400eeb0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.419+0000 7f405fade640 1 -- 172.21.15.60:0/3432121961 shutdown_connections 2024-02-26T05:00:14.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.419+0000 7f405fade640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f40440a20c0 0x7f40440a4580 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.419+0000 7f405fade640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4058100d90 0x7f40581a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.419+0000 7f405fade640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40581003f0 0x7f4058199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.419+0000 7f405fade640 1 --2- 172.21.15.60:0/3432121961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40580ffa90 0x7f4058199410 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.419+0000 7f405fade640 1 -- 172.21.15.60:0/3432121961 >> 172.21.15.60:0/3432121961 conn(0x7f40580fb6b0 msgr2=0x7f40580fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:14.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.419+0000 7f405fade640 1 -- 172.21.15.60:0/3432121961 shutdown_connections 2024-02-26T05:00:14.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.419+0000 7f405fade640 1 -- 172.21.15.60:0/3432121961 wait complete. 2024-02-26T05:00:14.431 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:14 smithi060 ceph-mon[32382]: pgmap v440: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-02-26T05:00:14.433 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T04:50:44.572921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T04:55:25.530720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T04:55:26.741604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-26T04:55:30.453569+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-26T05:00:14.433 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T05:00:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.536+0000 7f7ecd2de640 1 -- 172.21.15.60:0/2755602176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec8106f10 msgr2=0x7f7ec8109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.536+0000 7f7ecd2de640 1 --2- 172.21.15.60:0/2755602176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec8106f10 0x7f7ec8109300 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009ca0 tx=0x7f7ec002c0f0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.536+0000 7f7ecd2de640 1 -- 172.21.15.60:0/2755602176 shutdown_connections 2024-02-26T05:00:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.536+0000 7f7ecd2de640 1 --2- 172.21.15.60:0/2755602176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec8106f10 0x7f7ec8109300 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.536+0000 7f7ecd2de640 1 --2- 172.21.15.60:0/2755602176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec81045e0 0x7f7ec81069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.536+0000 7f7ecd2de640 1 --2- 172.21.15.60:0/2755602176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ec8101cb0 0x7f7ec81040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.536+0000 7f7ecd2de640 1 -- 172.21.15.60:0/2755602176 >> 172.21.15.60:0/2755602176 conn(0x7f7ec80fb620 msgr2=0x7f7ec80fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.536+0000 7f7ecd2de640 1 -- 172.21.15.60:0/2755602176 shutdown_connections 2024-02-26T05:00:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.536+0000 7f7ecd2de640 1 -- 172.21.15.60:0/2755602176 wait complete. 2024-02-26T05:00:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ecd2de640 1 Processor -- start 2024-02-26T05:00:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ecd2de640 1 -- start start 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ecd2de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec8101cb0 0x7f7ec8101620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ecd2de640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ec81045e0 0x7f7ec80ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ecd2de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec8106f10 0x7f7ec81001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ecd2de640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec8100900 con 0x7f7ec8101cb0 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ecd2de640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec8100a70 con 0x7f7ec81045e0 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ecd2de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec81a1260 con 0x7f7ec8106f10 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ec6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec8101cb0 0x7f7ec8101620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ec6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec8101cb0 0x7f7ec8101620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58844/0 (socket says 172.21.15.60:58844) 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ec6ffd640 1 -- 172.21.15.60:0/1511259032 learned_addr learned my addr 172.21.15.60:0/1511259032 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ebffff640 1 --2- 172.21.15.60:0/1511259032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ec81045e0 0x7f7ec80ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ec77fe640 1 --2- 172.21.15.60:0/1511259032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec8106f10 0x7f7ec81001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ec6ffd640 1 -- 172.21.15.60:0/1511259032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec8106f10 msgr2=0x7f7ec81001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ec6ffd640 1 --2- 172.21.15.60:0/1511259032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec8106f10 0x7f7ec81001a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ec6ffd640 1 -- 172.21.15.60:0/1511259032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ec81045e0 msgr2=0x7f7ec80ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:14.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ec6ffd640 1 --2- 172.21.15.60:0/1511259032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ec81045e0 0x7f7ec80ffc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.537+0000 7f7ec6ffd640 1 -- 172.21.15.60:0/1511259032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec0009900 con 0x7f7ec8101cb0 2024-02-26T05:00:14.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.538+0000 7f7ec6ffd640 1 --2- 172.21.15.60:0/1511259032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec8101cb0 0x7f7ec8101620 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800c8c0 tx=0x7f7eb800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:14.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.538+0000 7f7ec4ff9640 1 -- 172.21.15.60:0/1511259032 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8004220 con 0x7f7ec8101cb0 2024-02-26T05:00:14.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.538+0000 7f7ec4ff9640 1 -- 172.21.15.60:0/1511259032 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7eb8007e90 con 0x7f7ec8101cb0 2024-02-26T05:00:14.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.538+0000 7f7ecd2de640 1 -- 172.21.15.60:0/1511259032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec81a1510 con 0x7f7ec8101cb0 2024-02-26T05:00:14.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.538+0000 7f7ec4ff9640 1 -- 172.21.15.60:0/1511259032 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8004a00 con 0x7f7ec8101cb0 2024-02-26T05:00:14.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.538+0000 7f7ecd2de640 1 -- 172.21.15.60:0/1511259032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec81a1a70 con 0x7f7ec8101cb0 2024-02-26T05:00:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.539+0000 7f7ecd2de640 1 -- 172.21.15.60:0/1511259032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec80fd110 con 0x7f7ec8101cb0 2024-02-26T05:00:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.540+0000 7f7ec4ff9640 1 -- 172.21.15.60:0/1511259032 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f7eb80079b0 con 0x7f7ec8101cb0 2024-02-26T05:00:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.540+0000 7f7ec4ff9640 1 --2- 172.21.15.60:0/1511259032 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7eb40a2140 0x7f7eb40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.541+0000 7f7ebffff640 1 --2- 172.21.15.60:0/1511259032 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7eb40a2140 0x7f7eb40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.541+0000 7f7ec4ff9640 1 -- 172.21.15.60:0/1511259032 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f7eb80cdd50 con 0x7f7ec8101cb0 2024-02-26T05:00:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.541+0000 7f7ebffff640 1 --2- 172.21.15.60:0/1511259032 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7eb40a2140 0x7f7eb40a4600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8101390 tx=0x7f7eb000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:14.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.545+0000 7f7ec4ff9640 1 -- 172.21.15.60:0/1511259032 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7eb80d1030 con 0x7f7ec8101cb0 2024-02-26T05:00:14.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:14 smithi156 ceph-mon[35805]: pgmap v440: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-02-26T05:00:14.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.726+0000 7f7ecd2de640 1 -- 172.21.15.60:0/1511259032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ec810cbc0 con 0x7f7ec8101cb0 2024-02-26T05:00:14.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.726+0000 7f7ec4ff9640 1 -- 172.21.15.60:0/1511259032 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7eb80968e0 con 0x7f7ec8101cb0 2024-02-26T05:00:14.729 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-02-26T05:00:14.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.728+0000 7f7ecd2de640 1 -- 172.21.15.60:0/1511259032 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7eb40a2140 msgr2=0x7f7eb40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.728+0000 7f7ecd2de640 1 --2- 172.21.15.60:0/1511259032 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7eb40a2140 0x7f7eb40a4600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8101390 tx=0x7f7eb000b410 comp rx=0 tx=0).stop 2024-02-26T05:00:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.728+0000 7f7ecd2de640 1 -- 172.21.15.60:0/1511259032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec8101cb0 msgr2=0x7f7ec8101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.728+0000 7f7ecd2de640 1 --2- 172.21.15.60:0/1511259032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec8101cb0 0x7f7ec8101620 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800c8c0 tx=0x7f7eb800cd80 comp rx=0 tx=0).stop 2024-02-26T05:00:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.728+0000 7f7ecd2de640 1 -- 172.21.15.60:0/1511259032 shutdown_connections 2024-02-26T05:00:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.728+0000 7f7ecd2de640 1 --2- 172.21.15.60:0/1511259032 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f7eb40a2140 0x7f7eb40a4600 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.728+0000 7f7ecd2de640 1 --2- 172.21.15.60:0/1511259032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ec8106f10 0x7f7ec81001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.728+0000 7f7ecd2de640 1 --2- 172.21.15.60:0/1511259032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ec81045e0 0x7f7ec80ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.728+0000 7f7ecd2de640 1 --2- 172.21.15.60:0/1511259032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ec8101cb0 0x7f7ec8101620 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.728+0000 7f7ecd2de640 1 -- 172.21.15.60:0/1511259032 >> 172.21.15.60:0/1511259032 conn(0x7f7ec80fb620 msgr2=0x7f7ec81026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.728+0000 7f7ecd2de640 1 -- 172.21.15.60:0/1511259032 shutdown_connections 2024-02-26T05:00:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.728+0000 7f7ecd2de640 1 -- 172.21.15.60:0/1511259032 wait complete. 2024-02-26T05:00:14.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T05:00:14.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:14 smithi191 ceph-mon[36352]: pgmap v440: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-02-26T05:00:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.844+0000 7f566d0a2640 1 -- 172.21.15.60:0/1400451829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56681000a0 msgr2=0x7f56681004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.844+0000 7f566d0a2640 1 --2- 172.21.15.60:0/1400451829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56681000a0 0x7f56681004c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5658009900 tx=0x7f565802bc30 comp rx=0 tx=0).stop 2024-02-26T05:00:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.844+0000 7f566d0a2640 1 -- 172.21.15.60:0/1400451829 shutdown_connections 2024-02-26T05:00:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.844+0000 7f566d0a2640 1 --2- 172.21.15.60:0/1400451829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5668101c50 0x7f56681084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.844+0000 7f566d0a2640 1 --2- 172.21.15.60:0/1400451829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5668101290 0x7f5668101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.844+0000 7f566d0a2640 1 --2- 172.21.15.60:0/1400451829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56681000a0 0x7f56681004c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.844+0000 7f566d0a2640 1 -- 172.21.15.60:0/1400451829 >> 172.21.15.60:0/1400451829 conn(0x7f56680fb630 msgr2=0x7f56680fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.844+0000 7f566d0a2640 1 -- 172.21.15.60:0/1400451829 shutdown_connections 2024-02-26T05:00:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.844+0000 7f566d0a2640 1 -- 172.21.15.60:0/1400451829 wait complete. 2024-02-26T05:00:14.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.845+0000 7f566d0a2640 1 Processor -- start 2024-02-26T05:00:14.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.845+0000 7f566d0a2640 1 -- start start 2024-02-26T05:00:14.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.845+0000 7f566d0a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56681000a0 0x7f5668104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:14.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.845+0000 7f566d0a2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5668101290 0x7f5668102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:14.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.845+0000 7f566d0a2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5668101c50 0x7f5668102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:14.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.845+0000 7f566d0a2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5668103310 con 0x7f56681000a0 2024-02-26T05:00:14.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.845+0000 7f566d0a2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5668103480 con 0x7f5668101c50 2024-02-26T05:00:14.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.845+0000 7f566d0a2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56681a1310 con 0x7f5668101290 2024-02-26T05:00:14.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.845+0000 7f5666d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56681000a0 0x7f5668104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:14.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.845+0000 7f5666d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56681000a0 0x7f5668104030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58858/0 (socket says 172.21.15.60:58858) 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.845+0000 7f5666d9d640 1 -- 172.21.15.60:0/2487881367 learned_addr learned my addr 172.21.15.60:0/2487881367 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.846+0000 7f566759e640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5668101c50 0x7f5668102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.846+0000 7f566659c640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5668101290 0x7f5668102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.846+0000 7f5666d9d640 1 -- 172.21.15.60:0/2487881367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5668101290 msgr2=0x7f5668102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.846+0000 7f5666d9d640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5668101290 0x7f5668102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.846+0000 7f5666d9d640 1 -- 172.21.15.60:0/2487881367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5668101c50 msgr2=0x7f5668102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.846+0000 7f5666d9d640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5668101c50 0x7f5668102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.846+0000 7f5666d9d640 1 -- 172.21.15.60:0/2487881367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5658009560 con 0x7f56681000a0 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.846+0000 7f566659c640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5668101290 0x7f5668102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.846+0000 7f5666d9d640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56681000a0 0x7f5668104030 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5658036040 tx=0x7f5658004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.846+0000 7f564ffff640 1 -- 172.21.15.60:0/2487881367 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658004df0 con 0x7f56681000a0 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.846+0000 7f566d0a2640 1 -- 172.21.15.60:0/2487881367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56681a1560 con 0x7f56681000a0 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.846+0000 7f564ffff640 1 -- 172.21.15.60:0/2487881367 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5658005430 con 0x7f56681000a0 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.846+0000 7f564ffff640 1 -- 172.21.15.60:0/2487881367 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565803e450 con 0x7f56681000a0 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.846+0000 7f566d0a2640 1 -- 172.21.15.60:0/2487881367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56681a1ac0 con 0x7f56681000a0 2024-02-26T05:00:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.847+0000 7f566d0a2640 1 -- 172.21.15.60:0/2487881367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f566810bab0 con 0x7f56681000a0 2024-02-26T05:00:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.848+0000 7f564ffff640 1 -- 172.21.15.60:0/2487881367 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5658003680 con 0x7f56681000a0 2024-02-26T05:00:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.849+0000 7f564ffff640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f56500a20c0 0x7f56500a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.849+0000 7f566659c640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f56500a20c0 0x7f56500a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:14.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.849+0000 7f564ffff640 1 -- 172.21.15.60:0/2487881367 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f56580f06c0 con 0x7f56681000a0 2024-02-26T05:00:14.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.849+0000 7f566659c640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f56500a20c0 0x7f56500a4580 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5668103da0 tx=0x7f565c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:14.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:14.853+0000 7f564ffff640 1 -- 172.21.15.60:0/2487881367 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56580b9250 con 0x7f56681000a0 2024-02-26T05:00:15.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.035+0000 7f566d0a2640 1 -- 172.21.15.60:0/2487881367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56681a1e60 con 0x7f56681000a0 2024-02-26T05:00:15.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.035+0000 7f564ffff640 1 -- 172.21.15.60:0/2487881367 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f56580bc910 con 0x7f56681000a0 2024-02-26T05:00:15.038 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T05:00:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.037+0000 7f566d0a2640 1 -- 172.21.15.60:0/2487881367 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f56500a20c0 msgr2=0x7f56500a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.037+0000 7f566d0a2640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f56500a20c0 0x7f56500a4580 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5668103da0 tx=0x7f565c005e50 comp rx=0 tx=0).stop 2024-02-26T05:00:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.037+0000 7f566d0a2640 1 -- 172.21.15.60:0/2487881367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56681000a0 msgr2=0x7f5668104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.037+0000 7f566d0a2640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56681000a0 0x7f5668104030 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5658036040 tx=0x7f5658004270 comp rx=0 tx=0).stop 2024-02-26T05:00:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.037+0000 7f566d0a2640 1 -- 172.21.15.60:0/2487881367 shutdown_connections 2024-02-26T05:00:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.037+0000 7f566d0a2640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f56500a20c0 0x7f56500a4580 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.037+0000 7f566d0a2640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5668101c50 0x7f5668102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.037+0000 7f566d0a2640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5668101290 0x7f5668102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.037+0000 7f566d0a2640 1 --2- 172.21.15.60:0/2487881367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56681000a0 0x7f5668104030 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.037+0000 7f566d0a2640 1 -- 172.21.15.60:0/2487881367 >> 172.21.15.60:0/2487881367 conn(0x7f56680fb630 msgr2=0x7f56680fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.037+0000 7f566d0a2640 1 -- 172.21.15.60:0/2487881367 shutdown_connections 2024-02-26T05:00:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.037+0000 7f566d0a2640 1 -- 172.21.15.60:0/2487881367 wait complete. 2024-02-26T05:00:15.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T05:00:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.153+0000 7fd3aab99640 1 -- 172.21.15.60:0/3036834098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a4106ec0 msgr2=0x7fd3a41092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.153+0000 7fd3aab99640 1 --2- 172.21.15.60:0/3036834098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a4106ec0 0x7fd3a41092b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd394009900 tx=0x7fd39402bc30 comp rx=0 tx=0).stop 2024-02-26T05:00:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.153+0000 7fd3aab99640 1 -- 172.21.15.60:0/3036834098 shutdown_connections 2024-02-26T05:00:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.153+0000 7fd3aab99640 1 --2- 172.21.15.60:0/3036834098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a4106ec0 0x7fd3a41092b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.153+0000 7fd3aab99640 1 --2- 172.21.15.60:0/3036834098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3a4104590 0x7fd3a4106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.153+0000 7fd3aab99640 1 --2- 172.21.15.60:0/3036834098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3a4101c60 0x7fd3a4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.153+0000 7fd3aab99640 1 -- 172.21.15.60:0/3036834098 >> 172.21.15.60:0/3036834098 conn(0x7fd3a40fb610 msgr2=0x7fd3a40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.153+0000 7fd3aab99640 1 -- 172.21.15.60:0/3036834098 shutdown_connections 2024-02-26T05:00:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.153+0000 7fd3aab99640 1 -- 172.21.15.60:0/3036834098 wait complete. 2024-02-26T05:00:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.154+0000 7fd3aab99640 1 Processor -- start 2024-02-26T05:00:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.154+0000 7fd3aab99640 1 -- start start 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.154+0000 7fd3aab99640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3a4101c60 0x7fd3a41015d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.154+0000 7fd3aab99640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a4104590 0x7fd3a40ffc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.154+0000 7fd3aab99640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3a4106ec0 0x7fd3a4100150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.154+0000 7fd3aab99640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a41008b0 con 0x7fd3a4104590 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.154+0000 7fd3aab99640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a4100a20 con 0x7fd3a4106ec0 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.154+0000 7fd3aab99640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a41a1230 con 0x7fd3a4101c60 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.154+0000 7fd3a3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a4104590 0x7fd3a40ffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.154+0000 7fd3a3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a4104590 0x7fd3a40ffc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58874/0 (socket says 172.21.15.60:58874) 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.154+0000 7fd3a3fff640 1 -- 172.21.15.60:0/3352098422 learned_addr learned my addr 172.21.15.60:0/3352098422 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.154+0000 7fd3a8935640 1 --2- 172.21.15.60:0/3352098422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3a4101c60 0x7fd3a41015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.155+0000 7fd3a9136640 1 --2- 172.21.15.60:0/3352098422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3a4106ec0 0x7fd3a4100150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.155+0000 7fd3a3fff640 1 -- 172.21.15.60:0/3352098422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3a4101c60 msgr2=0x7fd3a41015d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.155+0000 7fd3a3fff640 1 --2- 172.21.15.60:0/3352098422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3a4101c60 0x7fd3a41015d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.155+0000 7fd3a3fff640 1 -- 172.21.15.60:0/3352098422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3a4106ec0 msgr2=0x7fd3a4100150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.155+0000 7fd3a3fff640 1 --2- 172.21.15.60:0/3352098422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3a4106ec0 0x7fd3a4100150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.155+0000 7fd3a3fff640 1 -- 172.21.15.60:0/3352098422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd394009560 con 0x7fd3a4104590 2024-02-26T05:00:15.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.155+0000 7fd3a3fff640 1 --2- 172.21.15.60:0/3352098422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a4104590 0x7fd3a40ffc10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd39800c9a0 tx=0x7fd39800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:15.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.155+0000 7fd3a1ffb640 1 -- 172.21.15.60:0/3352098422 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd398007bb0 con 0x7fd3a4104590 2024-02-26T05:00:15.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.155+0000 7fd3a1ffb640 1 -- 172.21.15.60:0/3352098422 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd398007d10 con 0x7fd3a4104590 2024-02-26T05:00:15.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.155+0000 7fd3aab99640 1 -- 172.21.15.60:0/3352098422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a41a14e0 con 0x7fd3a4104590 2024-02-26T05:00:15.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.155+0000 7fd3a1ffb640 1 -- 172.21.15.60:0/3352098422 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd398018730 con 0x7fd3a4104590 2024-02-26T05:00:15.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.155+0000 7fd3aab99640 1 -- 172.21.15.60:0/3352098422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a41a19a0 con 0x7fd3a4104590 2024-02-26T05:00:15.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.156+0000 7fd3aab99640 1 -- 172.21.15.60:0/3352098422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a40fd100 con 0x7fd3a4104590 2024-02-26T05:00:15.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.157+0000 7fd3a1ffb640 1 -- 172.21.15.60:0/3352098422 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd39800f450 con 0x7fd3a4104590 2024-02-26T05:00:15.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.158+0000 7fd3a1ffb640 1 --2- 172.21.15.60:0/3352098422 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd38c0a20c0 0x7fd38c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:15.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.158+0000 7fd3a8935640 1 --2- 172.21.15.60:0/3352098422 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd38c0a20c0 0x7fd38c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:15.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.158+0000 7fd3a1ffb640 1 -- 172.21.15.60:0/3352098422 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fd3980ce210 con 0x7fd3a4104590 2024-02-26T05:00:15.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.158+0000 7fd3a8935640 1 --2- 172.21.15.60:0/3352098422 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd38c0a20c0 0x7fd38c0a4580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd39c005fd0 tx=0x7fd39c009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.162+0000 7fd3a1ffb640 1 -- 172.21.15.60:0/3352098422 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd398096d40 con 0x7fd3a4104590 2024-02-26T05:00:15.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.349+0000 7fd3aab99640 1 -- 172.21.15.60:0/3352098422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3a402ce40 con 0x7fd3a4104590 2024-02-26T05:00:15.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.350+0000 7fd3a1ffb640 1 -- 172.21.15.60:0/3352098422 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd39809a400 con 0x7fd3a4104590 2024-02-26T05:00:15.352 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-02-26T05:00:15.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.352+0000 7fd3aab99640 1 -- 172.21.15.60:0/3352098422 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd38c0a20c0 msgr2=0x7fd38c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.352+0000 7fd3aab99640 1 --2- 172.21.15.60:0/3352098422 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd38c0a20c0 0x7fd38c0a4580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd39c005fd0 tx=0x7fd39c009680 comp rx=0 tx=0).stop 2024-02-26T05:00:15.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.352+0000 7fd3aab99640 1 -- 172.21.15.60:0/3352098422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a4104590 msgr2=0x7fd3a40ffc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.352+0000 7fd3aab99640 1 --2- 172.21.15.60:0/3352098422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a4104590 0x7fd3a40ffc10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd39800c9a0 tx=0x7fd39800ce60 comp rx=0 tx=0).stop 2024-02-26T05:00:15.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.352+0000 7fd3aab99640 1 -- 172.21.15.60:0/3352098422 shutdown_connections 2024-02-26T05:00:15.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.352+0000 7fd3aab99640 1 --2- 172.21.15.60:0/3352098422 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd38c0a20c0 0x7fd38c0a4580 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.352+0000 7fd3aab99640 1 --2- 172.21.15.60:0/3352098422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3a4106ec0 0x7fd3a4100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.352+0000 7fd3aab99640 1 --2- 172.21.15.60:0/3352098422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a4104590 0x7fd3a40ffc10 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.352+0000 7fd3aab99640 1 --2- 172.21.15.60:0/3352098422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3a4101c60 0x7fd3a41015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.352+0000 7fd3aab99640 1 -- 172.21.15.60:0/3352098422 >> 172.21.15.60:0/3352098422 conn(0x7fd3a40fb610 msgr2=0x7fd3a4102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:15.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.352+0000 7fd3aab99640 1 -- 172.21.15.60:0/3352098422 shutdown_connections 2024-02-26T05:00:15.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.352+0000 7fd3aab99640 1 -- 172.21.15.60:0/3352098422 wait complete. 2024-02-26T05:00:15.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:15 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3432121961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T05:00:15.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:15 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1511259032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T05:00:15.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:15 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2487881367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T05:00:15.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-26T05:00:15.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.467+0000 7ff7ebede640 1 -- 172.21.15.60:0/3234243519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e4101270 msgr2=0x7ff7e41016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.467+0000 7ff7ebede640 1 --2- 172.21.15.60:0/3234243519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e4101270 0x7ff7e41016f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8009900 tx=0x7ff7d802bc30 comp rx=0 tx=0).stop 2024-02-26T05:00:15.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.467+0000 7ff7ebede640 1 -- 172.21.15.60:0/3234243519 shutdown_connections 2024-02-26T05:00:15.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.467+0000 7ff7ebede640 1 --2- 172.21.15.60:0/3234243519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e4101c30 0x7ff7e41084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.467+0000 7ff7ebede640 1 --2- 172.21.15.60:0/3234243519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e4101270 0x7ff7e41016f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.467+0000 7ff7ebede640 1 --2- 172.21.15.60:0/3234243519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e4100080 0x7ff7e41004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.467+0000 7ff7ebede640 1 -- 172.21.15.60:0/3234243519 >> 172.21.15.60:0/3234243519 conn(0x7ff7e40fb610 msgr2=0x7ff7e40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:15.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.467+0000 7ff7ebede640 1 -- 172.21.15.60:0/3234243519 shutdown_connections 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.468+0000 7ff7ebede640 1 -- 172.21.15.60:0/3234243519 wait complete. 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.468+0000 7ff7ebede640 1 Processor -- start 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.468+0000 7ff7ebede640 1 -- start start 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.468+0000 7ff7ebede640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e4100080 0x7ff7e4197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.468+0000 7ff7ebede640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e4101270 0x7ff7e41977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.468+0000 7ff7ebede640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e4101c30 0x7ff7e419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.468+0000 7ff7ebede640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e419ed70 con 0x7ff7e4100080 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.468+0000 7ff7ebede640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e419eee0 con 0x7ff7e4101270 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.468+0000 7ff7ebede640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e419f0a0 con 0x7ff7e4101c30 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.468+0000 7ff7e9c7a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e4100080 0x7ff7e4197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.468+0000 7ff7e9c7a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e4100080 0x7ff7e4197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58892/0 (socket says 172.21.15.60:58892) 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.468+0000 7ff7e9c7a640 1 -- 172.21.15.60:0/2161382103 learned_addr learned my addr 172.21.15.60:0/2161382103 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.468+0000 7ff7ea47b640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e4101c30 0x7ff7e419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.469+0000 7ff7e9c7a640 1 -- 172.21.15.60:0/2161382103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e4101c30 msgr2=0x7ff7e419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.468+0000 7ff7e9479640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e4101270 0x7ff7e41977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:15.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.469+0000 7ff7e9c7a640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e4101c30 0x7ff7e419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.469+0000 7ff7e9c7a640 1 -- 172.21.15.60:0/2161382103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e4101270 msgr2=0x7ff7e41977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.469+0000 7ff7e9c7a640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e4101270 0x7ff7e41977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.469+0000 7ff7e9c7a640 1 -- 172.21.15.60:0/2161382103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d8009560 con 0x7ff7e4100080 2024-02-26T05:00:15.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.469+0000 7ff7e9479640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e4101270 0x7ff7e41977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T05:00:15.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.469+0000 7ff7e9c7a640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e4100080 0x7ff7e4197280 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00d950 tx=0x7ff7dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:15.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.469+0000 7ff7e2ffd640 1 -- 172.21.15.60:0/2161382103 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc009c30 con 0x7ff7e4100080 2024-02-26T05:00:15.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.469+0000 7ff7e2ffd640 1 -- 172.21.15.60:0/2161382103 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7dc009d90 con 0x7ff7e4100080 2024-02-26T05:00:15.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.469+0000 7ff7ebede640 1 -- 172.21.15.60:0/2161382103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e419f380 con 0x7ff7e4100080 2024-02-26T05:00:15.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.469+0000 7ff7e2ffd640 1 -- 172.21.15.60:0/2161382103 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00f720 con 0x7ff7e4100080 2024-02-26T05:00:15.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.469+0000 7ff7ebede640 1 -- 172.21.15.60:0/2161382103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e419f840 con 0x7ff7e4100080 2024-02-26T05:00:15.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.470+0000 7ff7ebede640 1 -- 172.21.15.60:0/2161382103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e410ba90 con 0x7ff7e4100080 2024-02-26T05:00:15.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.471+0000 7ff7e2ffd640 1 -- 172.21.15.60:0/2161382103 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff7dc010a60 con 0x7ff7e4100080 2024-02-26T05:00:15.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.472+0000 7ff7e2ffd640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff7cc0a20c0 0x7ff7cc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:15.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.472+0000 7ff7e2ffd640 1 -- 172.21.15.60:0/2161382103 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7ff7dc0cde80 con 0x7ff7e4100080 2024-02-26T05:00:15.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.472+0000 7ff7e9479640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff7cc0a20c0 0x7ff7cc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:15.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.472+0000 7ff7e9479640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff7cc0a20c0 0x7ff7cc0a4580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8036040 tx=0x7ff7d8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:15.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.475+0000 7ff7e2ffd640 1 -- 172.21.15.60:0/2161382103 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7dc096a10 con 0x7ff7e4100080 2024-02-26T05:00:15.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:15 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3432121961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T05:00:15.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:15 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1511259032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T05:00:15.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:15 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2487881367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T05:00:15.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.656+0000 7ff7ebede640 1 -- 172.21.15.60:0/2161382103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff7e419fcd0 con 0x7ff7e4100080 2024-02-26T05:00:15.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.657+0000 7ff7e2ffd640 1 -- 172.21.15.60:0/2161382103 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff7dc0170a0 con 0x7ff7e4100080 2024-02-26T05:00:15.659 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-02-26T05:00:15.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.659+0000 7ff7ebede640 1 -- 172.21.15.60:0/2161382103 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff7cc0a20c0 msgr2=0x7ff7cc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.659+0000 7ff7ebede640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff7cc0a20c0 0x7ff7cc0a4580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8036040 tx=0x7ff7d8000bc0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.659+0000 7ff7ebede640 1 -- 172.21.15.60:0/2161382103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e4100080 msgr2=0x7ff7e4197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.659+0000 7ff7ebede640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e4100080 0x7ff7e4197280 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00d950 tx=0x7ff7dc00de10 comp rx=0 tx=0).stop 2024-02-26T05:00:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.659+0000 7ff7ebede640 1 -- 172.21.15.60:0/2161382103 shutdown_connections 2024-02-26T05:00:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.659+0000 7ff7ebede640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7ff7cc0a20c0 0x7ff7cc0a4580 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.659+0000 7ff7ebede640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e4101c30 0x7ff7e419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.659+0000 7ff7ebede640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e4101270 0x7ff7e41977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.659+0000 7ff7ebede640 1 --2- 172.21.15.60:0/2161382103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e4100080 0x7ff7e4197280 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.659+0000 7ff7ebede640 1 -- 172.21.15.60:0/2161382103 >> 172.21.15.60:0/2161382103 conn(0x7ff7e40fb610 msgr2=0x7ff7e40682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.659+0000 7ff7ebede640 1 -- 172.21.15.60:0/2161382103 shutdown_connections 2024-02-26T05:00:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.659+0000 7ff7ebede640 1 -- 172.21.15.60:0/2161382103 wait complete. 2024-02-26T05:00:15.672 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T05:00:15.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:15 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3432121961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T05:00:15.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:15 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1511259032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-26T05:00:15.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:15 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2487881367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-26T05:00:15.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.773+0000 7fd3e261c640 1 -- 172.21.15.60:0/2518100257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3dc102d40 msgr2=0x7fd3dc1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.773+0000 7fd3e261c640 1 --2- 172.21.15.60:0/2518100257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3dc102d40 0x7fd3dc1095d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009ca0 tx=0x7fd3d402c0f0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.774+0000 7fd3e261c640 1 -- 172.21.15.60:0/2518100257 shutdown_connections 2024-02-26T05:00:15.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.774+0000 7fd3e261c640 1 --2- 172.21.15.60:0/2518100257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3dc102d40 0x7fd3dc1095d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.774+0000 7fd3e261c640 1 --2- 172.21.15.60:0/2518100257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3dc102380 0x7fd3dc102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.774+0000 7fd3e261c640 1 --2- 172.21.15.60:0/2518100257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3dc101190 0x7fd3dc1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.774+0000 7fd3e261c640 1 -- 172.21.15.60:0/2518100257 >> 172.21.15.60:0/2518100257 conn(0x7fd3dc0fc720 msgr2=0x7fd3dc0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.774+0000 7fd3e261c640 1 -- 172.21.15.60:0/2518100257 shutdown_connections 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.774+0000 7fd3e261c640 1 -- 172.21.15.60:0/2518100257 wait complete. 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.774+0000 7fd3e261c640 1 Processor -- start 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.774+0000 7fd3e261c640 1 -- start start 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.774+0000 7fd3e261c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3dc101190 0x7fd3dc197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e261c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3dc102380 0x7fd3dc197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e261c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3dc102d40 0x7fd3dc19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e261c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc19ec40 con 0x7fd3dc102d40 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e261c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc19edb0 con 0x7fd3dc101190 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e261c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc19ef70 con 0x7fd3dc102380 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e0bb9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3dc102d40 0x7fd3dc19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e0bb9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3dc102d40 0x7fd3dc19e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58910/0 (socket says 172.21.15.60:58910) 2024-02-26T05:00:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e0bb9640 1 -- 172.21.15.60:0/1547570610 learned_addr learned my addr 172.21.15.60:0/1547570610 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T05:00:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3dbfff640 1 --2- 172.21.15.60:0/1547570610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3dc101190 0x7fd3dc197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3db7fe640 1 --2- 172.21.15.60:0/1547570610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3dc102380 0x7fd3dc197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e0bb9640 1 -- 172.21.15.60:0/1547570610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3dc102380 msgr2=0x7fd3dc197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e0bb9640 1 --2- 172.21.15.60:0/1547570610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3dc102380 0x7fd3dc197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e0bb9640 1 -- 172.21.15.60:0/1547570610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3dc101190 msgr2=0x7fd3dc197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e0bb9640 1 --2- 172.21.15.60:0/1547570610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3dc101190 0x7fd3dc197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e0bb9640 1 -- 172.21.15.60:0/1547570610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d4009900 con 0x7fd3dc102d40 2024-02-26T05:00:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e0bb9640 1 --2- 172.21.15.60:0/1547570610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3dc102d40 0x7fd3dc19e700 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4037040 tx=0x7fd3d4034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3d97fa640 1 -- 172.21.15.60:0/1547570610 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4032eb0 con 0x7fd3dc102d40 2024-02-26T05:00:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3d97fa640 1 -- 172.21.15.60:0/1547570610 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3d4003e50 con 0x7fd3dc102d40 2024-02-26T05:00:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e261c640 1 -- 172.21.15.60:0/1547570610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3dc19f1f0 con 0x7fd3dc102d40 2024-02-26T05:00:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3d97fa640 1 -- 172.21.15.60:0/1547570610 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4040450 con 0x7fd3dc102d40 2024-02-26T05:00:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.775+0000 7fd3e261c640 1 -- 172.21.15.60:0/1547570610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3dc19f6a0 con 0x7fd3dc102d40 2024-02-26T05:00:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.776+0000 7fd3e261c640 1 -- 172.21.15.60:0/1547570610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3dc108450 con 0x7fd3dc102d40 2024-02-26T05:00:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.777+0000 7fd3d97fa640 1 -- 172.21.15.60:0/1547570610 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd3d400a040 con 0x7fd3dc102d40 2024-02-26T05:00:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.778+0000 7fd3d97fa640 1 --2- 172.21.15.60:0/1547570610 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd3c40a20c0 0x7fd3c40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.778+0000 7fd3d97fa640 1 -- 172.21.15.60:0/1547570610 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fd3d40f0f60 con 0x7fd3dc102d40 2024-02-26T05:00:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.778+0000 7fd3dbfff640 1 --2- 172.21.15.60:0/1547570610 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd3c40a20c0 0x7fd3c40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.778+0000 7fd3dbfff640 1 --2- 172.21.15.60:0/1547570610 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd3c40a20c0 0x7fd3c40a4580 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc1021e0 tx=0x7fd3cc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:15.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.782+0000 7fd3d97fa640 1 -- 172.21.15.60:0/1547570610 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd3d40b9af0 con 0x7fd3dc102d40 2024-02-26T05:00:15.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.978+0000 7fd3e261c640 1 -- 172.21.15.60:0/1547570610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd3dc02ce40 con 0x7fd3dc102d40 2024-02-26T05:00:15.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.978+0000 7fd3d97fa640 1 -- 172.21.15.60:0/1547570610 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) v1 ==== 81+0+588 (secure 0 0 0) 0x7fd3d40bd1b0 con 0x7fd3dc102d40 2024-02-26T05:00:15.981 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/556269024 2024-02-27T04:48:18.223906+0000 2024-02-26T05:00:15.981 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1221449590 2024-02-27T04:48:18.223906+0000 2024-02-26T05:00:15.981 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/114766175 2024-02-27T04:47:50.799728+0000 2024-02-26T05:00:15.981 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/2375965956 2024-02-27T04:47:50.799728+0000 2024-02-26T05:00:15.981 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1037269853 2024-02-27T04:48:18.223906+0000 2024-02-26T05:00:15.981 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6800/3694881479 2024-02-27T04:48:18.223906+0000 2024-02-26T05:00:15.981 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/114766175 2024-02-27T04:47:50.799728+0000 2024-02-26T05:00:15.981 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/558398124 2024-02-27T04:47:50.799728+0000 2024-02-26T05:00:15.981 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6801/3694881479 2024-02-27T04:48:18.223906+0000 2024-02-26T05:00:15.981 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:0/1162202856 2024-02-27T04:47:50.799728+0000 2024-02-26T05:00:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:listed 10 entries 2024-02-26T05:00:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.980+0000 7fd3e261c640 1 -- 172.21.15.60:0/1547570610 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd3c40a20c0 msgr2=0x7fd3c40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.980+0000 7fd3e261c640 1 --2- 172.21.15.60:0/1547570610 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd3c40a20c0 0x7fd3c40a4580 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc1021e0 tx=0x7fd3cc00b390 comp rx=0 tx=0).stop 2024-02-26T05:00:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.980+0000 7fd3e261c640 1 -- 172.21.15.60:0/1547570610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3dc102d40 msgr2=0x7fd3dc19e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.980+0000 7fd3e261c640 1 --2- 172.21.15.60:0/1547570610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3dc102d40 0x7fd3dc19e700 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4037040 tx=0x7fd3d4034d40 comp rx=0 tx=0).stop 2024-02-26T05:00:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.980+0000 7fd3e261c640 1 -- 172.21.15.60:0/1547570610 shutdown_connections 2024-02-26T05:00:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.980+0000 7fd3e261c640 1 --2- 172.21.15.60:0/1547570610 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fd3c40a20c0 0x7fd3c40a4580 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.980+0000 7fd3e261c640 1 --2- 172.21.15.60:0/1547570610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3dc102d40 0x7fd3dc19e700 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.980+0000 7fd3e261c640 1 --2- 172.21.15.60:0/1547570610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3dc102380 0x7fd3dc197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.980+0000 7fd3e261c640 1 --2- 172.21.15.60:0/1547570610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3dc101190 0x7fd3dc197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.980+0000 7fd3e261c640 1 -- 172.21.15.60:0/1547570610 >> 172.21.15.60:0/1547570610 conn(0x7fd3dc0fc720 msgr2=0x7fd3dc105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.980+0000 7fd3e261c640 1 -- 172.21.15.60:0/1547570610 shutdown_connections 2024-02-26T05:00:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:15.980+0000 7fd3e261c640 1 -- 172.21.15.60:0/1547570610 wait complete. 2024-02-26T05:00:15.993 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-26T05:00:15.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T05:00:15.993 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-26T05:00:16.031 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T05:00:16.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T05:00:16.031 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-26T05:00:16.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T05:00:16.086 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-02-26T05:00:16.141 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-26T05:00:16.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T05:00:16.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-26T05:00:16.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T05:00:16.209 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-02-26T05:00:16.312 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-26T05:00:16.322 INFO:tasks.cephadm:Teardown begin 2024-02-26T05:00:16.322 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-26T05:00:16.322 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T05:00:16.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.425+0000 7fcbba808640 1 -- 172.21.15.60:0/3962417474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbb41012a0 msgr2=0x7fcbb4101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:16.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.425+0000 7fcbba808640 1 --2- 172.21.15.60:0/3962417474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbb41012a0 0x7fcbb4101720 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009900 tx=0x7fcba802bc30 comp rx=0 tx=0).stop 2024-02-26T05:00:16.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.425+0000 7fcbba808640 1 -- 172.21.15.60:0/3962417474 shutdown_connections 2024-02-26T05:00:16.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.425+0000 7fcbba808640 1 --2- 172.21.15.60:0/3962417474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbb4101c60 0x7fcbb41084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.425+0000 7fcbba808640 1 --2- 172.21.15.60:0/3962417474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbb41012a0 0x7fcbb4101720 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.425+0000 7fcbba808640 1 --2- 172.21.15.60:0/3962417474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbb41000b0 0x7fcbb41004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.425+0000 7fcbba808640 1 -- 172.21.15.60:0/3962417474 >> 172.21.15.60:0/3962417474 conn(0x7fcbb40fb660 msgr2=0x7fcbb40fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:16.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.425+0000 7fcbba808640 1 -- 172.21.15.60:0/3962417474 shutdown_connections 2024-02-26T05:00:16.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.425+0000 7fcbba808640 1 -- 172.21.15.60:0/3962417474 wait complete. 2024-02-26T05:00:16.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.425+0000 7fcbba808640 1 Processor -- start 2024-02-26T05:00:16.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbba808640 1 -- start start 2024-02-26T05:00:16.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbba808640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbb41000b0 0x7fcbb406a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:16.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbba808640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbb41012a0 0x7fcbb419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:16.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbba808640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbb4101c60 0x7fcbb419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:16.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbba808640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb419ed60 con 0x7fcbb41012a0 2024-02-26T05:00:16.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbba808640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb419eed0 con 0x7fcbb41000b0 2024-02-26T05:00:16.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbba808640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb419f090 con 0x7fcbb4101c60 2024-02-26T05:00:16.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbb37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbb41012a0 0x7fcbb419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:16.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbb37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbb41012a0 0x7fcbb419bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58924/0 (socket says 172.21.15.60:58924) 2024-02-26T05:00:16.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbb37fe640 1 -- 172.21.15.60:0/1070271817 learned_addr learned my addr 172.21.15.60:0/1070271817 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T05:00:16.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbb8da5640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbb4101c60 0x7fcbb419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:16.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbb3fff640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbb41000b0 0x7fcbb406a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:16.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbb37fe640 1 -- 172.21.15.60:0/1070271817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbb4101c60 msgr2=0x7fcbb419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:16.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbb37fe640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbb4101c60 0x7fcbb419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbb37fe640 1 -- 172.21.15.60:0/1070271817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbb41000b0 msgr2=0x7fcbb406a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:16.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbb37fe640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbb41000b0 0x7fcbb406a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbb37fe640 1 -- 172.21.15.60:0/1070271817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba8009560 con 0x7fcbb41012a0 2024-02-26T05:00:16.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbb3fff640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbb41000b0 0x7fcbb406a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T05:00:16.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.426+0000 7fcbb37fe640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbb41012a0 0x7fcbb419bf60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fcba802c6a0 tx=0x7fcba80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:16.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.427+0000 7fcbb17fa640 1 -- 172.21.15.60:0/1070271817 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80044e0 con 0x7fcbb41012a0 2024-02-26T05:00:16.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.427+0000 7fcbb17fa640 1 -- 172.21.15.60:0/1070271817 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcba80054e0 con 0x7fcbb41012a0 2024-02-26T05:00:16.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.427+0000 7fcbba808640 1 -- 172.21.15.60:0/1070271817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb419f2b0 con 0x7fcbb41012a0 2024-02-26T05:00:16.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.427+0000 7fcbb17fa640 1 -- 172.21.15.60:0/1070271817 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba803e450 con 0x7fcbb41012a0 2024-02-26T05:00:16.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.427+0000 7fcbba808640 1 -- 172.21.15.60:0/1070271817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb419f810 con 0x7fcbb41012a0 2024-02-26T05:00:16.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.428+0000 7fcbba808640 1 -- 172.21.15.60:0/1070271817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb4107370 con 0x7fcbb41012a0 2024-02-26T05:00:16.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.429+0000 7fcbb17fa640 1 -- 172.21.15.60:0/1070271817 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcba8003710 con 0x7fcbb41012a0 2024-02-26T05:00:16.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.429+0000 7fcbb17fa640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcb9c0a2140 0x7fcb9c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:16.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.429+0000 7fcbb3fff640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcb9c0a2140 0x7fcb9c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:16.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.429+0000 7fcbb17fa640 1 -- 172.21.15.60:0/1070271817 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7fcba80f0b00 con 0x7fcbb41012a0 2024-02-26T05:00:16.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.430+0000 7fcbb3fff640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcb9c0a2140 0x7fcb9c0a4600 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4101100 tx=0x7fcba4005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:16.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.433+0000 7fcbb17fa640 1 -- 172.21.15.60:0/1070271817 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcba80b97b0 con 0x7fcbb41012a0 2024-02-26T05:00:16.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.652+0000 7fcbba808640 1 -- 172.21.15.60:0/1070271817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbb410bb10 con 0x7fcbb41012a0 2024-02-26T05:00:16.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.653+0000 7fcbb17fa640 1 -- 172.21.15.60:0/1070271817 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11347 (secure 0 0 0) 0x7fcba8043070 con 0x7fcbb41012a0 2024-02-26T05:00:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T05:00:16.655 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:57:52.162616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24476,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T05:00:16.656 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-02-26T05:00:16.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.655+0000 7fcbba808640 1 -- 172.21.15.60:0/1070271817 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcb9c0a2140 msgr2=0x7fcb9c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:16.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.655+0000 7fcbba808640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcb9c0a2140 0x7fcb9c0a4600 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4101100 tx=0x7fcba4005e70 comp rx=0 tx=0).stop 2024-02-26T05:00:16.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.655+0000 7fcbba808640 1 -- 172.21.15.60:0/1070271817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbb41012a0 msgr2=0x7fcbb419bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:16.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.655+0000 7fcbba808640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbb41012a0 0x7fcbb419bf60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fcba802c6a0 tx=0x7fcba80042f0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.655+0000 7fcbba808640 1 -- 172.21.15.60:0/1070271817 shutdown_connections 2024-02-26T05:00:16.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.655+0000 7fcbba808640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fcb9c0a2140 0x7fcb9c0a4600 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.655+0000 7fcbba808640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbb4101c60 0x7fcbb419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.655+0000 7fcbba808640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbb41012a0 0x7fcbb419bf60 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.655+0000 7fcbba808640 1 --2- 172.21.15.60:0/1070271817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbb41000b0 0x7fcbb406a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.655+0000 7fcbba808640 1 -- 172.21.15.60:0/1070271817 >> 172.21.15.60:0/1070271817 conn(0x7fcbb40fb660 msgr2=0x7fcbb4104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.655+0000 7fcbba808640 1 -- 172.21.15.60:0/1070271817 shutdown_connections 2024-02-26T05:00:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.655+0000 7fcbba808640 1 -- 172.21.15.60:0/1070271817 wait complete. 2024-02-26T05:00:16.667 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:16 smithi060 ceph-mon[32382]: pgmap v441: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T05:00:16.667 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:16 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3352098422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T05:00:16.667 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:16 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2161382103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T05:00:16.667 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:16 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1547570610' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T05:00:16.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T05:00:16.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:16 smithi156 ceph-mon[35805]: pgmap v441: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T05:00:16.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:16 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3352098422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T05:00:16.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:16 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2161382103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T05:00:16.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:16 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1547570610' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T05:00:16.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:16 smithi191 ceph-mon[36352]: pgmap v441: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-26T05:00:16.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:16 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3352098422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-26T05:00:16.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:16 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2161382103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-26T05:00:16.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:16 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1547570610' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-26T05:00:16.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.768+0000 7f9852688640 1 -- 172.21.15.60:0/100130209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984c1011a0 msgr2=0x7f984c1015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:16.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.768+0000 7f9852688640 1 --2- 172.21.15.60:0/100130209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984c1011a0 0x7f984c1015c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f983c009900 tx=0x7f983c02bc30 comp rx=0 tx=0).stop 2024-02-26T05:00:16.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.768+0000 7f9852688640 1 -- 172.21.15.60:0/100130209 shutdown_connections 2024-02-26T05:00:16.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.768+0000 7f9852688640 1 --2- 172.21.15.60:0/100130209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f984c102d50 0x7f984c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.768+0000 7f9852688640 1 --2- 172.21.15.60:0/100130209 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f984c102390 0x7f984c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.768+0000 7f9852688640 1 --2- 172.21.15.60:0/100130209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984c1011a0 0x7f984c1015c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.768+0000 7f9852688640 1 -- 172.21.15.60:0/100130209 >> 172.21.15.60:0/100130209 conn(0x7f984c0fc730 msgr2=0x7f984c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:16.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.768+0000 7f9852688640 1 -- 172.21.15.60:0/100130209 shutdown_connections 2024-02-26T05:00:16.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.768+0000 7f9852688640 1 -- 172.21.15.60:0/100130209 wait complete. 2024-02-26T05:00:16.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f9852688640 1 Processor -- start 2024-02-26T05:00:16.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f9852688640 1 -- start start 2024-02-26T05:00:16.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f9852688640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f984c1011a0 0x7f984c197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:16.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f9852688640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984c102390 0x7f984c1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:16.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f9852688640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f984c102d50 0x7f984c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f9852688640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c19ed50 con 0x7f984c102390 2024-02-26T05:00:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f9852688640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c19eec0 con 0x7f984c1011a0 2024-02-26T05:00:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f9852688640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c19f080 con 0x7f984c102d50 2024-02-26T05:00:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f984b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984c102390 0x7f984c1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f984b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984c102390 0x7f984c1977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58932/0 (socket says 172.21.15.60:58932) 2024-02-26T05:00:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f984b7fe640 1 -- 172.21.15.60:0/2742009759 learned_addr learned my addr 172.21.15.60:0/2742009759 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T05:00:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f984bfff640 1 --2- 172.21.15.60:0/2742009759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f984c1011a0 0x7f984c197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f984b7fe640 1 -- 172.21.15.60:0/2742009759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f984c102d50 msgr2=0x7f984c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f984b7fe640 1 --2- 172.21.15.60:0/2742009759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f984c102d50 0x7f984c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f984b7fe640 1 -- 172.21.15.60:0/2742009759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f984c1011a0 msgr2=0x7f984c197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f984b7fe640 1 --2- 172.21.15.60:0/2742009759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f984c1011a0 0x7f984c197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.769+0000 7f984b7fe640 1 -- 172.21.15.60:0/2742009759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c009560 con 0x7f984c102390 2024-02-26T05:00:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.770+0000 7f984b7fe640 1 --2- 172.21.15.60:0/2742009759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984c102390 0x7f984c1977a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f984000c970 tx=0x7f984000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.770+0000 7f98497fa640 1 -- 172.21.15.60:0/2742009759 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840007bb0 con 0x7f984c102390 2024-02-26T05:00:16.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.770+0000 7f98497fa640 1 -- 172.21.15.60:0/2742009759 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9840007d10 con 0x7f984c102390 2024-02-26T05:00:16.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.770+0000 7f9852688640 1 -- 172.21.15.60:0/2742009759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984c19f360 con 0x7f984c102390 2024-02-26T05:00:16.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.770+0000 7f98497fa640 1 -- 172.21.15.60:0/2742009759 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984000f450 con 0x7f984c102390 2024-02-26T05:00:16.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.770+0000 7f9852688640 1 -- 172.21.15.60:0/2742009759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984c19f820 con 0x7f984c102390 2024-02-26T05:00:16.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.771+0000 7f9852688640 1 -- 172.21.15.60:0/2742009759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984c1084c0 con 0x7f984c102390 2024-02-26T05:00:16.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.772+0000 7f98497fa640 1 -- 172.21.15.60:0/2742009759 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9840004f70 con 0x7f984c102390 2024-02-26T05:00:16.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.772+0000 7f98497fa640 1 --2- 172.21.15.60:0/2742009759 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f98340a20c0 0x7f98340a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:16.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.773+0000 7f984bfff640 1 --2- 172.21.15.60:0/2742009759 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f98340a20c0 0x7f98340a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:16.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.773+0000 7f98497fa640 1 -- 172.21.15.60:0/2742009759 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f98400cdf30 con 0x7f984c102390 2024-02-26T05:00:16.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.773+0000 7f984bfff640 1 --2- 172.21.15.60:0/2742009759 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f98340a20c0 0x7f98340a4580 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f983c006010 tx=0x7f983c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:16.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.777+0000 7f98497fa640 1 -- 172.21.15.60:0/2742009759 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9840096a60 con 0x7f984c102390 2024-02-26T05:00:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.998+0000 7f9852688640 1 -- 172.21.15.60:0/2742009759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f984c02ce70 con 0x7f984c102390 2024-02-26T05:00:17.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:16.999+0000 7f98497fa640 1 -- 172.21.15.60:0/2742009759 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11347 (secure 0 0 0) 0x7f984009a120 con 0x7f984c102390 2024-02-26T05:00:17.001 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T05:00:17.001 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/894258792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":894258792},{"type":"v1","addr":"172.21.15.60:6835","nonce":894258792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1357036637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":1357036637},{"type":"v1","addr":"172.21.15.60:6837","nonce":1357036637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/2080933038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2080933038},{"type":"v1","addr":"172.21.15.60:6839","nonce":2080933038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6841/1896690404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6840","nonce":1896690404},{"type":"v1","addr":"172.21.15.60:6841","nonce":1896690404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1119041220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1119041220},{"type":"v1","addr":"172.21.15.191:6833","nonce":1119041220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/629272517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":629272517},{"type":"v1","addr":"172.21.15.156:6833","nonce":629272517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-26T04:55:27.553789+0000","modified":"2024-02-26T04:57:52.162616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24476,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1642858086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1642858086},{"type":"v1","addr":"172.21.15.191:6835","nonce":1642858086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2819944290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2819944290},{"type":"v1","addr":"172.21.15.156:6835","nonce":2819944290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/3000497710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3000497710},{"type":"v1","addr":"172.21.15.191:6837","nonce":3000497710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.191:6839/640686517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":640686517},{"type":"v1","addr":"172.21.15.191:6839","nonce":640686517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.156:6837/707575792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":707575792},{"type":"v1","addr":"172.21.15.156:6837","nonce":707575792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/1768629491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1768629491},{"type":"v1","addr":"172.21.15.156:6839","nonce":1768629491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T05:00:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-02-26T05:00:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.001+0000 7f9852688640 1 -- 172.21.15.60:0/2742009759 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f98340a20c0 msgr2=0x7f98340a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.001+0000 7f9852688640 1 --2- 172.21.15.60:0/2742009759 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f98340a20c0 0x7f98340a4580 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f983c006010 tx=0x7f983c000bc0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.001+0000 7f9852688640 1 -- 172.21.15.60:0/2742009759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984c102390 msgr2=0x7f984c1977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.001+0000 7f9852688640 1 --2- 172.21.15.60:0/2742009759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984c102390 0x7f984c1977a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f984000c970 tx=0x7f984000ce30 comp rx=0 tx=0).stop 2024-02-26T05:00:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.001+0000 7f9852688640 1 -- 172.21.15.60:0/2742009759 shutdown_connections 2024-02-26T05:00:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.001+0000 7f9852688640 1 --2- 172.21.15.60:0/2742009759 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f98340a20c0 0x7f98340a4580 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.001+0000 7f9852688640 1 --2- 172.21.15.60:0/2742009759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f984c102d50 0x7f984c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.001+0000 7f9852688640 1 --2- 172.21.15.60:0/2742009759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984c102390 0x7f984c1977a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.001+0000 7f9852688640 1 --2- 172.21.15.60:0/2742009759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f984c1011a0 0x7f984c197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.001+0000 7f9852688640 1 -- 172.21.15.60:0/2742009759 >> 172.21.15.60:0/2742009759 conn(0x7f984c0fc730 msgr2=0x7f984c105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.001+0000 7f9852688640 1 -- 172.21.15.60:0/2742009759 shutdown_connections 2024-02-26T05:00:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.001+0000 7f9852688640 1 -- 172.21.15.60:0/2742009759 wait complete. 2024-02-26T05:00:17.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T05:00:17.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.114+0000 7f943a4a2640 1 -- 172.21.15.60:0/4119005235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9434106e90 msgr2=0x7f94341092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.114+0000 7f943a4a2640 1 --2- 172.21.15.60:0/4119005235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9434106e90 0x7f94341092e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f942c009ca0 tx=0x7f942c02c0f0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.114+0000 7f943a4a2640 1 -- 172.21.15.60:0/4119005235 shutdown_connections 2024-02-26T05:00:17.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.114+0000 7f943a4a2640 1 --2- 172.21.15.60:0/4119005235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9434106e90 0x7f94341092e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.114+0000 7f943a4a2640 1 --2- 172.21.15.60:0/4119005235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9434104560 0x7f9434106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.114+0000 7f943a4a2640 1 --2- 172.21.15.60:0/4119005235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9434101c30 0x7f9434104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.114+0000 7f943a4a2640 1 -- 172.21.15.60:0/4119005235 >> 172.21.15.60:0/4119005235 conn(0x7f94340fb660 msgr2=0x7f94340fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.114+0000 7f943a4a2640 1 -- 172.21.15.60:0/4119005235 shutdown_connections 2024-02-26T05:00:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.114+0000 7f943a4a2640 1 -- 172.21.15.60:0/4119005235 wait complete. 2024-02-26T05:00:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.115+0000 7f943a4a2640 1 Processor -- start 2024-02-26T05:00:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.115+0000 7f943a4a2640 1 -- start start 2024-02-26T05:00:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.115+0000 7f943a4a2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9434101c30 0x7f94341971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.115+0000 7f943a4a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9434104560 0x7f9434197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.115+0000 7f943a4a2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9434106e90 0x7f943419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.115+0000 7f943a4a2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943419ece0 con 0x7f9434104560 2024-02-26T05:00:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.115+0000 7f943a4a2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943419ee50 con 0x7f9434106e90 2024-02-26T05:00:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.115+0000 7f943a4a2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943419f010 con 0x7f9434101c30 2024-02-26T05:00:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.115+0000 7f94337fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9434104560 0x7f9434197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.115+0000 7f94337fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9434104560 0x7f9434197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58948/0 (socket says 172.21.15.60:58948) 2024-02-26T05:00:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.115+0000 7f94337fe640 1 -- 172.21.15.60:0/1343290546 learned_addr learned my addr 172.21.15.60:0/1343290546 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T05:00:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.115+0000 7f9438a3f640 1 --2- 172.21.15.60:0/1343290546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9434106e90 0x7f943419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.115+0000 7f9433fff640 1 --2- 172.21.15.60:0/1343290546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9434101c30 0x7f94341971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.116+0000 7f94337fe640 1 -- 172.21.15.60:0/1343290546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9434101c30 msgr2=0x7f94341971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.116+0000 7f94337fe640 1 --2- 172.21.15.60:0/1343290546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9434101c30 0x7f94341971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.116+0000 7f94337fe640 1 -- 172.21.15.60:0/1343290546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9434106e90 msgr2=0x7f943419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.116+0000 7f94337fe640 1 --2- 172.21.15.60:0/1343290546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9434106e90 0x7f943419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.116+0000 7f94337fe640 1 -- 172.21.15.60:0/1343290546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c009900 con 0x7f9434104560 2024-02-26T05:00:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.116+0000 7f94337fe640 1 --2- 172.21.15.60:0/1343290546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9434104560 0x7f9434197730 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f942400e9c0 tx=0x7f942400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.116+0000 7f94317fa640 1 -- 172.21.15.60:0/1343290546 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424009700 con 0x7f9434104560 2024-02-26T05:00:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.116+0000 7f943a4a2640 1 -- 172.21.15.60:0/1343290546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943419f2f0 con 0x7f9434104560 2024-02-26T05:00:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.116+0000 7f94317fa640 1 -- 172.21.15.60:0/1343290546 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f942400cd80 con 0x7f9434104560 2024-02-26T05:00:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.116+0000 7f94317fa640 1 -- 172.21.15.60:0/1343290546 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424010430 con 0x7f9434104560 2024-02-26T05:00:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.116+0000 7f943a4a2640 1 -- 172.21.15.60:0/1343290546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943419f800 con 0x7f9434104560 2024-02-26T05:00:17.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.117+0000 7f943a4a2640 1 -- 172.21.15.60:0/1343290546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94340fd150 con 0x7f9434104560 2024-02-26T05:00:17.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.118+0000 7f94317fa640 1 -- 172.21.15.60:0/1343290546 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f9424005000 con 0x7f9434104560 2024-02-26T05:00:17.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.119+0000 7f94317fa640 1 --2- 172.21.15.60:0/1343290546 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f94200a20c0 0x7f94200a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:17.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.119+0000 7f9433fff640 1 --2- 172.21.15.60:0/1343290546 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f94200a20c0 0x7f94200a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:17.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.119+0000 7f94317fa640 1 -- 172.21.15.60:0/1343290546 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f9424014070 con 0x7f9434104560 2024-02-26T05:00:17.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.119+0000 7f9433fff640 1 --2- 172.21.15.60:0/1343290546 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f94200a20c0 0x7f94200a4580 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f941c009bd0 tx=0x7f941c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.123+0000 7f94317fa640 1 -- 172.21.15.60:0/1343290546 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9424097040 con 0x7f9434104560 2024-02-26T05:00:17.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.310+0000 7f943a4a2640 1 -- 172.21.15.60:0/1343290546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f943410cd70 con 0x7f9434104560 2024-02-26T05:00:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.311+0000 7f94317fa640 1 -- 172.21.15.60:0/1343290546 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24972 (secure 0 0 0) 0x7f942409a700 con 0x7f9434104560 2024-02-26T05:00:17.314 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-02-26T05:00:17.314 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"fsid":"f12e2206-d461-11ee-95c3-87774f69a715","created":"2024-02-26T04:47:22.521838+0000","modified":"2024-02-26T04:57:21.168776+0000","last_up_change":"2024-02-26T04:54:36.614110+0000","last_in_change":"2024-02-26T04:54:16.328482+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T04:50:44.572921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T04:55:25.530720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T04:55:26.741604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-26T04:55:30.453569+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cd04e2cc-5293-493c-a8e2-bef04f89c6ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6803","nonce":108541016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6805","nonce":108541016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6809","nonce":108541016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":108541016},{"type":"v1","addr":"172.21.15.60:6807","nonce":108541016}]},"public_addr":"172.21.15.60:6803/108541016","cluster_addr":"172.21.15.60:6805/108541016","heartbeat_back_addr":"172.21.15.60:6809/108541016","heartbeat_front_addr":"172.21.15.60:6807/108541016","state":["exists","up"]},{"osd":1,"uuid":"c43bde4b-f31d-4d09-8410-583371866db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6801","nonce":1258051364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6803","nonce":1258051364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6807","nonce":1258051364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1258051364},{"type":"v1","addr":"172.21.15.156:6805","nonce":1258051364}]},"public_addr":"172.21.15.156:6801/1258051364","cluster_addr":"172.21.15.156:6803/1258051364","heartbeat_back_addr":"172.21.15.156:6807/1258051364","heartbeat_front_addr":"172.21.15.156:6805/1258051364","state":["exists","up"]},{"osd":2,"uuid":"5cfe428a-e1f9-46b1-ba71-fe7384aa8b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6801","nonce":1607191954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6803","nonce":1607191954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6807","nonce":1607191954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1607191954},{"type":"v1","addr":"172.21.15.191:6805","nonce":1607191954}]},"public_addr":"172.21.15.191:6801/1607191954","cluster_addr":"172.21.15.191:6803/1607191954","heartbeat_back_addr":"172.21.15.191:6807/1607191954","heartbeat_front_addr":"172.21.15.191:6805/1607191954","state":["exists","up"]},{"osd":3,"uuid":"7e2d7dc0-1002-44e9-9295-2447bf76a03a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6811","nonce":3123201146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6813","nonce":3123201146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6817","nonce":3123201146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3123201146},{"type":"v1","addr":"172.21.15.60:6815","nonce":3123201146}]},"public_addr":"172.21.15.60:6811/3123201146","cluster_addr":"172.21.15.60:6813/3123201146","heartbeat_back_addr":"172.21.15.60:6817/3123201146","heartbeat_front_addr":"172.21.15.60:6815/3123201146","state":["exists","up"]},{"osd":4,"uuid":"d040e4b1-df8e-4f0c-9f84-767081c41939","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6809","nonce":714533576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6811","nonce":714533576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6815","nonce":714533576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":714533576},{"type":"v1","addr":"172.21.15.156:6813","nonce":714533576}]},"public_addr":"172.21.15.156:6809/714533576","cluster_addr":"172.21.15.156:6811/714533576","heartbeat_back_addr":"172.21.15.156:6815/714533576","heartbeat_front_addr":"172.21.15.156:6813/714533576","state":["exists","up"]},{"osd":5,"uuid":"a7807402-620a-40f2-85b4-71f762a1a73b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6809","nonce":1681496417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6811","nonce":1681496417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6815","nonce":1681496417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1681496417},{"type":"v1","addr":"172.21.15.191:6813","nonce":1681496417}]},"public_addr":"172.21.15.191:6809/1681496417","cluster_addr":"172.21.15.191:6811/1681496417","heartbeat_back_addr":"172.21.15.191:6815/1681496417","heartbeat_front_addr":"172.21.15.191:6813/1681496417","state":["exists","up"]},{"osd":6,"uuid":"43d4456c-935e-4e9c-afe8-9a6c72a8a4ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6819","nonce":3054430804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6821","nonce":3054430804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6825","nonce":3054430804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3054430804},{"type":"v1","addr":"172.21.15.60:6823","nonce":3054430804}]},"public_addr":"172.21.15.60:6819/3054430804","cluster_addr":"172.21.15.60:6821/3054430804","heartbeat_back_addr":"172.21.15.60:6825/3054430804","heartbeat_front_addr":"172.21.15.60:6823/3054430804","state":["exists","up"]},{"osd":7,"uuid":"e158aaaf-5538-4711-ae58-2d9acc28c9f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6817","nonce":2499710318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6819","nonce":2499710318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6823","nonce":2499710318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2499710318},{"type":"v1","addr":"172.21.15.156:6821","nonce":2499710318}]},"public_addr":"172.21.15.156:6817/2499710318","cluster_addr":"172.21.15.156:6819/2499710318","heartbeat_back_addr":"172.21.15.156:6823/2499710318","heartbeat_front_addr":"172.21.15.156:6821/2499710318","state":["exists","up"]},{"osd":8,"uuid":"5669eb6b-6750-411a-bd9f-61cf8db220fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6817","nonce":3417694461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6819","nonce":3417694461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6823","nonce":3417694461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3417694461},{"type":"v1","addr":"172.21.15.191:6821","nonce":3417694461}]},"public_addr":"172.21.15.191:6817/3417694461","cluster_addr":"172.21.15.191:6819/3417694461","heartbeat_back_addr":"172.21.15.191:6823/3417694461","heartbeat_front_addr":"172.21.15.191:6821/3417694461","state":["exists","up"]},{"osd":9,"uuid":"c0802892-114a-4875-9914-aa936d399e4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6827","nonce":1562445343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6829","nonce":1562445343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6833","nonce":1562445343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1562445343},{"type":"v1","addr":"172.21.15.60:6831","nonce":1562445343}]},"public_addr":"172.21.15.60:6827/1562445343","cluster_addr":"172.21.15.60:6829/1562445343","heartbeat_back_addr":"172.21.15.60:6833/1562445343","heartbeat_front_addr":"172.21.15.60:6831/1562445343","state":["exists","up"]},{"osd":10,"uuid":"eda157d9-fe5f-46ad-8a82-0d60cf51c438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6825","nonce":3738247798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6827","nonce":3738247798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6831","nonce":3738247798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3738247798},{"type":"v1","addr":"172.21.15.156:6829","nonce":3738247798}]},"public_addr":"172.21.15.156:6825/3738247798","cluster_addr":"172.21.15.156:6827/3738247798","heartbeat_back_addr":"172.21.15.156:6831/3738247798","heartbeat_front_addr":"172.21.15.156:6829/3738247798","state":["exists","up"]},{"osd":11,"uuid":"80ebe4a4-3a19-4074-aa49-bfe2339295b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6825","nonce":277456354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6827","nonce":277456354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6831","nonce":277456354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":277456354},{"type":"v1","addr":"172.21.15.191:6829","nonce":277456354}]},"public_addr":"172.21.15.191:6825/277456354","cluster_addr":"172.21.15.191:6827/277456354","heartbeat_back_addr":"172.21.15.191:6831/277456354","heartbeat_front_addr":"172.21.15.191:6829/277456354","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:49:53.954647+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:17.995335+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:50:41.990981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:08.587069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:33.586069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:51:58.945367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:24.689381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:52:50.601999+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:16.103893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:53:42.495500+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:08.165100+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T04:54:35.224880+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.16","mappings":[{"from":1,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/556269024":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1221449590":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/2375965956":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/1037269853":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6800/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:6801/114766175":"2024-02-27T04:47:50.799728+0000","172.21.15.60:0/558398124":"2024-02-27T04:47:50.799728+0000","172.21.15.60:6801/3694881479":"2024-02-27T04:48:18.223906+0000","172.21.15.60:0/1162202856":"2024-02-27T04:47:50.799728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T05:00:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.313+0000 7f943a4a2640 1 -- 172.21.15.60:0/1343290546 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f94200a20c0 msgr2=0x7f94200a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.313+0000 7f943a4a2640 1 --2- 172.21.15.60:0/1343290546 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f94200a20c0 0x7f94200a4580 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f941c009bd0 tx=0x7f941c005f90 comp rx=0 tx=0).stop 2024-02-26T05:00:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.313+0000 7f943a4a2640 1 -- 172.21.15.60:0/1343290546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9434104560 msgr2=0x7f9434197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.313+0000 7f943a4a2640 1 --2- 172.21.15.60:0/1343290546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9434104560 0x7f9434197730 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f942400e9c0 tx=0x7f942400ee80 comp rx=0 tx=0).stop 2024-02-26T05:00:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.313+0000 7f943a4a2640 1 -- 172.21.15.60:0/1343290546 shutdown_connections 2024-02-26T05:00:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.313+0000 7f943a4a2640 1 --2- 172.21.15.60:0/1343290546 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f94200a20c0 0x7f94200a4580 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.313+0000 7f943a4a2640 1 --2- 172.21.15.60:0/1343290546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9434106e90 0x7f943419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.313+0000 7f943a4a2640 1 --2- 172.21.15.60:0/1343290546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9434104560 0x7f9434197730 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.313+0000 7f943a4a2640 1 --2- 172.21.15.60:0/1343290546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9434101c30 0x7f94341971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.313+0000 7f943a4a2640 1 -- 172.21.15.60:0/1343290546 >> 172.21.15.60:0/1343290546 conn(0x7f94340fb660 msgr2=0x7f9434102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:17.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.313+0000 7f943a4a2640 1 -- 172.21.15.60:0/1343290546 shutdown_connections 2024-02-26T05:00:17.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.313+0000 7f943a4a2640 1 -- 172.21.15.60:0/1343290546 wait complete. 2024-02-26T05:00:17.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-26T05:00:17.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.428+0000 7f1d39c76640 1 -- 172.21.15.60:0/1146531545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d34102380 msgr2=0x7f1d34102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.428+0000 7f1d39c76640 1 --2- 172.21.15.60:0/1146531545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d34102380 0x7f1d34102800 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009900 tx=0x7f1d2802bc30 comp rx=0 tx=0).stop 2024-02-26T05:00:17.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.428+0000 7f1d39c76640 1 -- 172.21.15.60:0/1146531545 shutdown_connections 2024-02-26T05:00:17.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.428+0000 7f1d39c76640 1 --2- 172.21.15.60:0/1146531545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d34102d40 0x7f1d341095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.428+0000 7f1d39c76640 1 --2- 172.21.15.60:0/1146531545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d34102380 0x7f1d34102800 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.428+0000 7f1d39c76640 1 --2- 172.21.15.60:0/1146531545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d34101190 0x7f1d341015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.428+0000 7f1d39c76640 1 -- 172.21.15.60:0/1146531545 >> 172.21.15.60:0/1146531545 conn(0x7f1d340fc720 msgr2=0x7f1d340feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:17.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.428+0000 7f1d39c76640 1 -- 172.21.15.60:0/1146531545 shutdown_connections 2024-02-26T05:00:17.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.428+0000 7f1d39c76640 1 -- 172.21.15.60:0/1146531545 wait complete. 2024-02-26T05:00:17.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d39c76640 1 Processor -- start 2024-02-26T05:00:17.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d39c76640 1 -- start start 2024-02-26T05:00:17.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d39c76640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d34101190 0x7f1d341971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:17.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d39c76640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d34102380 0x7f1d341976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:17.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d39c76640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d34102d40 0x7f1d3419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:17.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d39c76640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3419ec90 con 0x7f1d34102d40 2024-02-26T05:00:17.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d39c76640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3419ee00 con 0x7f1d34102380 2024-02-26T05:00:17.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d39c76640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3419efc0 con 0x7f1d34101190 2024-02-26T05:00:17.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d33fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d34102d40 0x7f1d3419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:17.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d33fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d34102d40 0x7f1d3419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58970/0 (socket says 172.21.15.60:58970) 2024-02-26T05:00:17.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d33fff640 1 -- 172.21.15.60:0/4102640855 learned_addr learned my addr 172.21.15.60:0/4102640855 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T05:00:17.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d32ffd640 1 --2- 172.21.15.60:0/4102640855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d34102380 0x7f1d341976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:17.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d337fe640 1 --2- 172.21.15.60:0/4102640855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d34101190 0x7f1d341971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:17.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d33fff640 1 -- 172.21.15.60:0/4102640855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d34101190 msgr2=0x7f1d341971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d33fff640 1 --2- 172.21.15.60:0/4102640855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d34101190 0x7f1d341971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d33fff640 1 -- 172.21.15.60:0/4102640855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d34102380 msgr2=0x7f1d341976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.429+0000 7f1d33fff640 1 --2- 172.21.15.60:0/4102640855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d34102380 0x7f1d341976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.430+0000 7f1d33fff640 1 -- 172.21.15.60:0/4102640855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d28009560 con 0x7f1d34102d40 2024-02-26T05:00:17.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.430+0000 7f1d33fff640 1 --2- 172.21.15.60:0/4102640855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d34102d40 0x7f1d3419e750 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00cce0 tx=0x7f1d2c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:17.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.430+0000 7f1d30ff9640 1 -- 172.21.15.60:0/4102640855 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c00f450 con 0x7f1d34102d40 2024-02-26T05:00:17.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.430+0000 7f1d30ff9640 1 -- 172.21.15.60:0/4102640855 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d2c00fa90 con 0x7f1d34102d40 2024-02-26T05:00:17.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.430+0000 7f1d39c76640 1 -- 172.21.15.60:0/4102640855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3419f2a0 con 0x7f1d34102d40 2024-02-26T05:00:17.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.430+0000 7f1d30ff9640 1 -- 172.21.15.60:0/4102640855 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c018b90 con 0x7f1d34102d40 2024-02-26T05:00:17.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.430+0000 7f1d39c76640 1 -- 172.21.15.60:0/4102640855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3419f730 con 0x7f1d34102d40 2024-02-26T05:00:17.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.431+0000 7f1d39c76640 1 -- 172.21.15.60:0/4102640855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d34108530 con 0x7f1d34102d40 2024-02-26T05:00:17.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.432+0000 7f1d30ff9640 1 -- 172.21.15.60:0/4102640855 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1d2c00fc00 con 0x7f1d34102d40 2024-02-26T05:00:17.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.433+0000 7f1d30ff9640 1 --2- 172.21.15.60:0/4102640855 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1d1c0a20c0 0x7f1d1c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:17.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.433+0000 7f1d337fe640 1 --2- 172.21.15.60:0/4102640855 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1d1c0a20c0 0x7f1d1c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:17.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.433+0000 7f1d30ff9640 1 -- 172.21.15.60:0/4102640855 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9324+0+0 (secure 0 0 0) 0x7f1d2c0ce850 con 0x7f1d34102d40 2024-02-26T05:00:17.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.433+0000 7f1d337fe640 1 --2- 172.21.15.60:0/4102640855 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1d1c0a20c0 0x7f1d1c0a4580 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1d341021e0 tx=0x7f1d2400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.437+0000 7f1d30ff9640 1 -- 172.21.15.60:0/4102640855 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1d2c097380 con 0x7f1d34102d40 2024-02-26T05:00:17.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.655+0000 7f1d39c76640 1 -- 172.21.15.60:0/4102640855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1d3410cba0 con 0x7f1d34102d40 2024-02-26T05:00:17.659 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:17 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1070271817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T05:00:17.659 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:17 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/2742009759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T05:00:17.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.657+0000 7f1d30ff9640 1 -- 172.21.15.60:0/4102640855 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1d2c09aa40 con 0x7f1d34102d40 2024-02-26T05:00:17.659 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-26T05:00:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.660+0000 7f1d39c76640 1 -- 172.21.15.60:0/4102640855 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1d1c0a20c0 msgr2=0x7f1d1c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.660+0000 7f1d39c76640 1 --2- 172.21.15.60:0/4102640855 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1d1c0a20c0 0x7f1d1c0a4580 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1d341021e0 tx=0x7f1d2400a2d0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.660+0000 7f1d39c76640 1 -- 172.21.15.60:0/4102640855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d34102d40 msgr2=0x7f1d3419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.660+0000 7f1d39c76640 1 --2- 172.21.15.60:0/4102640855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d34102d40 0x7f1d3419e750 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00cce0 tx=0x7f1d2c00a620 comp rx=0 tx=0).stop 2024-02-26T05:00:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.660+0000 7f1d39c76640 1 -- 172.21.15.60:0/4102640855 shutdown_connections 2024-02-26T05:00:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.660+0000 7f1d39c76640 1 --2- 172.21.15.60:0/4102640855 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f1d1c0a20c0 0x7f1d1c0a4580 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.660+0000 7f1d39c76640 1 --2- 172.21.15.60:0/4102640855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d34102d40 0x7f1d3419e750 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.660+0000 7f1d39c76640 1 --2- 172.21.15.60:0/4102640855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d34102380 0x7f1d341976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.660+0000 7f1d39c76640 1 --2- 172.21.15.60:0/4102640855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d34101190 0x7f1d341971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.660+0000 7f1d39c76640 1 -- 172.21.15.60:0/4102640855 >> 172.21.15.60:0/4102640855 conn(0x7f1d340fc720 msgr2=0x7f1d34105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.660+0000 7f1d39c76640 1 -- 172.21.15.60:0/4102640855 shutdown_connections 2024-02-26T05:00:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.660+0000 7f1d39c76640 1 -- 172.21.15.60:0/4102640855 wait complete. 2024-02-26T05:00:17.674 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-26T05:00:17.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:17 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1070271817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T05:00:17.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:17 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/2742009759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T05:00:17.704 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:17 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1070271817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T05:00:17.704 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:17 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/2742009759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-26T05:00:17.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.779+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3748201395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c8101c50 msgr2=0x7fe5c81084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.779+0000 7fe5cf99c640 1 --2- 172.21.15.60:0/3748201395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c8101c50 0x7fe5c81084e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009ca0 tx=0x7fe5b802c0f0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.779+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3748201395 shutdown_connections 2024-02-26T05:00:17.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.779+0000 7fe5cf99c640 1 --2- 172.21.15.60:0/3748201395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c8101c50 0x7fe5c81084e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.779+0000 7fe5cf99c640 1 --2- 172.21.15.60:0/3748201395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c8101290 0x7fe5c8101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.779+0000 7fe5cf99c640 1 --2- 172.21.15.60:0/3748201395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5c81000a0 0x7fe5c81004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.779+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3748201395 >> 172.21.15.60:0/3748201395 conn(0x7fe5c80fb630 msgr2=0x7fe5c80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:17.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.779+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3748201395 shutdown_connections 2024-02-26T05:00:17.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.779+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3748201395 wait complete. 2024-02-26T05:00:17.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cf99c640 1 Processor -- start 2024-02-26T05:00:17.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cf99c640 1 -- start start 2024-02-26T05:00:17.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cf99c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c81000a0 0x7fe5c81972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cf99c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5c8101290 0x7fe5c8197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cf99c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c8101c50 0x7fe5c819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cf99c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c819edd0 con 0x7fe5c81000a0 2024-02-26T05:00:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cf99c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c819ef40 con 0x7fe5c8101c50 2024-02-26T05:00:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cf99c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c819f100 con 0x7fe5c8101290 2024-02-26T05:00:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cd738640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c81000a0 0x7fe5c81972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cd738640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c81000a0 0x7fe5c81972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58976/0 (socket says 172.21.15.60:58976) 2024-02-26T05:00:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cd738640 1 -- 172.21.15.60:0/3122510027 learned_addr learned my addr 172.21.15.60:0/3122510027 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T05:00:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5ccf37640 1 --2- 172.21.15.60:0/3122510027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5c8101290 0x7fe5c8197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cdf39640 1 --2- 172.21.15.60:0/3122510027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c8101c50 0x7fe5c819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cd738640 1 -- 172.21.15.60:0/3122510027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5c8101290 msgr2=0x7fe5c8197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cd738640 1 --2- 172.21.15.60:0/3122510027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5c8101290 0x7fe5c8197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cd738640 1 -- 172.21.15.60:0/3122510027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c8101c50 msgr2=0x7fe5c819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cd738640 1 --2- 172.21.15.60:0/3122510027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c8101c50 0x7fe5c819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:17.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.780+0000 7fe5cd738640 1 -- 172.21.15.60:0/3122510027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b8009900 con 0x7fe5c81000a0 2024-02-26T05:00:17.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.781+0000 7fe5cd738640 1 --2- 172.21.15.60:0/3122510027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c81000a0 0x7fe5c81972e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000c970 tx=0x7fe5c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:17.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.781+0000 7fe5c67fc640 1 -- 172.21.15.60:0/3122510027 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0004220 con 0x7fe5c81000a0 2024-02-26T05:00:17.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.781+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3122510027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c819f3e0 con 0x7fe5c81000a0 2024-02-26T05:00:17.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.781+0000 7fe5c67fc640 1 -- 172.21.15.60:0/3122510027 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5c0007e90 con 0x7fe5c81000a0 2024-02-26T05:00:17.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.781+0000 7fe5c67fc640 1 -- 172.21.15.60:0/3122510027 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0004a00 con 0x7fe5c81000a0 2024-02-26T05:00:17.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.781+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3122510027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c819f8a0 con 0x7fe5c81000a0 2024-02-26T05:00:17.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.782+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3122510027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c8107360 con 0x7fe5c81000a0 2024-02-26T05:00:17.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.783+0000 7fe5c67fc640 1 -- 172.21.15.60:0/3122510027 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe5c00079b0 con 0x7fe5c81000a0 2024-02-26T05:00:17.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.783+0000 7fe5c67fc640 1 --2- 172.21.15.60:0/3122510027 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5b00a20c0 0x7fe5b00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:17.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.783+0000 7fe5c67fc640 1 -- 172.21.15.60:0/3122510027 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fe5c00ce350 con 0x7fe5c81000a0 2024-02-26T05:00:17.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.783+0000 7fe5ccf37640 1 --2- 172.21.15.60:0/3122510027 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5b00a20c0 0x7fe5b00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:17.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.784+0000 7fe5ccf37640 1 --2- 172.21.15.60:0/3122510027 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5b00a20c0 0x7fe5b00a4580 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe5c81987f0 tx=0x7fe5bc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:17.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:17.788+0000 7fe5c67fc640 1 -- 172.21.15.60:0/3122510027 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe5c0096de0 con 0x7fe5c81000a0 2024-02-26T05:00:17.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:17 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a[32378]: 2024-02-26T05:00:17.655+0000 7f909fb31700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T05:00:18.009 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 05:00:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-f[66272]: ignoring --setuser ceph since I am not root 2024-02-26T05:00:18.009 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 05:00:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-f[66272]: ignoring --setgroup ceph since I am not root 2024-02-26T05:00:18.009 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 05:00:17 smithi191 ceph-mds[66276]: -- 172.21.15.191:0/223629100 <== mon.0 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5654ede381a0 con 0x5654ede0e800 2024-02-26T05:00:18.009 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 05:00:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-f[66272]: starting mds.f at 2024-02-26T05:00:18.010 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 05:00:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-i[66892]: ignoring --setuser ceph since I am not root 2024-02-26T05:00:18.010 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 05:00:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-i[66892]: ignoring --setgroup ceph since I am not root 2024-02-26T05:00:18.010 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 05:00:17 smithi191 ceph-mds[66896]: -- 172.21.15.191:0/2382354776 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55de2484e1a0 con 0x55de24825000 2024-02-26T05:00:18.010 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 05:00:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-i[66892]: starting mds.i at 2024-02-26T05:00:18.010 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 05:00:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-c[66585]: ignoring --setuser ceph since I am not root 2024-02-26T05:00:18.010 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 05:00:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-c[66585]: ignoring --setgroup ceph since I am not root 2024-02-26T05:00:18.010 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 05:00:17 smithi191 ceph-mds[66589]: -- 172.21.15.191:0/3517213539 <== mon.0 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aa9b7ee1a0 con 0x55aa9b7c4800 2024-02-26T05:00:18.010 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 05:00:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-c[66585]: starting mds.c at 2024-02-26T05:00:18.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.008+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3122510027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe5c810cc10 con 0x7fe5c81000a0 2024-02-26T05:00:18.177 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 05:00:17 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-e[65247]: ignoring --setuser ceph since I am not root 2024-02-26T05:00:18.178 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 05:00:17 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-e[65247]: ignoring --setgroup ceph since I am not root 2024-02-26T05:00:18.178 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 05:00:17 smithi156 ceph-mds[65251]: -- 172.21.15.156:0/2721360863 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ee2bc301a0 con 0x55ee2bc07000 2024-02-26T05:00:18.178 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 05:00:17 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-e[65247]: starting mds.e at 2024-02-26T05:00:18.178 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 05:00:17 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-b[65871]: ignoring --setuser ceph since I am not root 2024-02-26T05:00:18.178 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 05:00:17 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-b[65871]: ignoring --setgroup ceph since I am not root 2024-02-26T05:00:18.178 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 05:00:17 smithi156 ceph-mds[65875]: -- 172.21.15.156:0/4178930395 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5631c78cc1a0 con 0x5631c78a2800 2024-02-26T05:00:18.178 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 05:00:17 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-b[65871]: starting mds.b at 2024-02-26T05:00:18.178 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 05:00:17 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-h[65555]: ignoring --setuser ceph since I am not root 2024-02-26T05:00:18.178 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 05:00:17 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-h[65555]: ignoring --setgroup ceph since I am not root 2024-02-26T05:00:18.179 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 05:00:17 smithi156 ceph-mds[65559]: -- 172.21.15.156:0/3074383924 <== mon.0 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564b3dc261a0 con 0x564b3dbfd000 2024-02-26T05:00:18.179 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 05:00:17 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-h[65555]: starting mds.h at 2024-02-26T05:00:18.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.659+0000 7fe5c67fc640 1 -- 172.21.15.60:0/3122510027 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe5c810cc10 con 0x7fe5c81000a0 2024-02-26T05:00:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.661+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3122510027 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5b00a20c0 msgr2=0x7fe5b00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.661+0000 7fe5cf99c640 1 --2- 172.21.15.60:0/3122510027 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5b00a20c0 0x7fe5b00a4580 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe5c81987f0 tx=0x7fe5bc008040 comp rx=0 tx=0).stop 2024-02-26T05:00:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.661+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3122510027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c81000a0 msgr2=0x7fe5c81972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.661+0000 7fe5cf99c640 1 --2- 172.21.15.60:0/3122510027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c81000a0 0x7fe5c81972e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000c970 tx=0x7fe5c000ce30 comp rx=0 tx=0).stop 2024-02-26T05:00:18.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.662+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3122510027 shutdown_connections 2024-02-26T05:00:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.662+0000 7fe5cf99c640 1 --2- 172.21.15.60:0/3122510027 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5b00a20c0 0x7fe5b00a4580 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.662+0000 7fe5cf99c640 1 --2- 172.21.15.60:0/3122510027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c8101c50 0x7fe5c819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.662+0000 7fe5cf99c640 1 --2- 172.21.15.60:0/3122510027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5c8101290 0x7fe5c8197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.662+0000 7fe5cf99c640 1 --2- 172.21.15.60:0/3122510027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c81000a0 0x7fe5c81972e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.662+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3122510027 >> 172.21.15.60:0/3122510027 conn(0x7fe5c80fb630 msgr2=0x7fe5c8104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.662+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3122510027 shutdown_connections 2024-02-26T05:00:18.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.662+0000 7fe5cf99c640 1 -- 172.21.15.60:0/3122510027 wait complete. 2024-02-26T05:00:18.677 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:18 smithi060 ceph-mon[32382]: pgmap v442: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T05:00:18.677 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:18 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1343290546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T05:00:18.677 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:18 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4102640855' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T05:00:18.677 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:18 smithi060 ceph-mon[32382]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-26T05:00:18.678 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:18 smithi060 ceph-mon[32382]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T05:00:18.678 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:18 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4102640855' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-26T05:00:18.678 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:18 smithi060 ceph-mon[32382]: osdmap e94: 12 total, 12 up, 12 in 2024-02-26T05:00:18.678 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:18 smithi060 ceph-mon[32382]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-26T05:00:18.678 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:18 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3122510027' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T05:00:18.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:18 smithi156 ceph-mon[35805]: pgmap v442: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T05:00:18.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:18 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1343290546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T05:00:18.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:18 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4102640855' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T05:00:18.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:18 smithi156 ceph-mon[35805]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-26T05:00:18.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:18 smithi156 ceph-mon[35805]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T05:00:18.679 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:18 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4102640855' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-26T05:00:18.679 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:18 smithi156 ceph-mon[35805]: osdmap e94: 12 total, 12 up, 12 in 2024-02-26T05:00:18.679 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:18 smithi156 ceph-mon[35805]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-26T05:00:18.679 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:18 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3122510027' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T05:00:18.680 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-26T05:00:18.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:18 smithi191 ceph-mon[36352]: pgmap v442: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-26T05:00:18.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:18 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1343290546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-26T05:00:18.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:18 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4102640855' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-26T05:00:18.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:18 smithi191 ceph-mon[36352]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-26T05:00:18.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:18 smithi191 ceph-mon[36352]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T05:00:18.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:18 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4102640855' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-26T05:00:18.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:18 smithi191 ceph-mon[36352]: osdmap e94: 12 total, 12 up, 12 in 2024-02-26T05:00:18.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:18 smithi191 ceph-mon[36352]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-26T05:00:18.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:18 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3122510027' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-26T05:00:18.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.790+0000 7f229c91e640 1 -- 172.21.15.60:0/707035451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2298102380 msgr2=0x7f2298102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:18.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.790+0000 7f229c91e640 1 --2- 172.21.15.60:0/707035451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2298102380 0x7f2298102800 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2280009900 tx=0x7f228002bc30 comp rx=0 tx=0).stop 2024-02-26T05:00:18.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.790+0000 7f229c91e640 1 -- 172.21.15.60:0/707035451 shutdown_connections 2024-02-26T05:00:18.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.790+0000 7f229c91e640 1 --2- 172.21.15.60:0/707035451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2298102d40 0x7f22981095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:18.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.790+0000 7f229c91e640 1 --2- 172.21.15.60:0/707035451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2298102380 0x7f2298102800 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:18.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.790+0000 7f229c91e640 1 --2- 172.21.15.60:0/707035451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2298101190 0x7f22981015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:18.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.790+0000 7f229c91e640 1 -- 172.21.15.60:0/707035451 >> 172.21.15.60:0/707035451 conn(0x7f22980fc720 msgr2=0x7f22980feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:18.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.790+0000 7f229c91e640 1 -- 172.21.15.60:0/707035451 shutdown_connections 2024-02-26T05:00:18.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.790+0000 7f229c91e640 1 -- 172.21.15.60:0/707035451 wait complete. 2024-02-26T05:00:18.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f229c91e640 1 Processor -- start 2024-02-26T05:00:18.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f229c91e640 1 -- start start 2024-02-26T05:00:18.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f229c91e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2298101190 0x7f2298197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:18.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f229c91e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2298102380 0x7f2298197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:18.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f229c91e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2298102d40 0x7f229819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:18.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f229c91e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229819ed00 con 0x7f2298102d40 2024-02-26T05:00:18.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f229c91e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229819ee70 con 0x7f2298101190 2024-02-26T05:00:18.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f229c91e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229819f030 con 0x7f2298102380 2024-02-26T05:00:18.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f2296d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2298102d40 0x7f229819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:18.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f2296d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2298102d40 0x7f229819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58998/0 (socket says 172.21.15.60:58998) 2024-02-26T05:00:18.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f2296d9d640 1 -- 172.21.15.60:0/1002721046 learned_addr learned my addr 172.21.15.60:0/1002721046 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T05:00:18.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f229659c640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2298101190 0x7f2298197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:18.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f2295d9b640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2298102380 0x7f2298197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:18.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f2296d9d640 1 -- 172.21.15.60:0/1002721046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2298102380 msgr2=0x7f2298197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:18.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f2296d9d640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2298102380 0x7f2298197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:18.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f2296d9d640 1 -- 172.21.15.60:0/1002721046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2298101190 msgr2=0x7f2298197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:18.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f2296d9d640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2298101190 0x7f2298197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:18.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.791+0000 7f2296d9d640 1 -- 172.21.15.60:0/1002721046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2280009560 con 0x7f2298102d40 2024-02-26T05:00:18.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.792+0000 7f2295d9b640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2298102380 0x7f2298197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T05:00:18.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.792+0000 7f2296d9d640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2298102d40 0x7f229819e7c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f228800eaa0 tx=0x7f228800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:18.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.792+0000 7f228f7fe640 1 -- 172.21.15.60:0/1002721046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228800cdb0 con 0x7f2298102d40 2024-02-26T05:00:18.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.792+0000 7f229c91e640 1 -- 172.21.15.60:0/1002721046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f229819f310 con 0x7f2298102d40 2024-02-26T05:00:18.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.792+0000 7f228f7fe640 1 -- 172.21.15.60:0/1002721046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2288005020 con 0x7f2298102d40 2024-02-26T05:00:18.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.792+0000 7f228f7fe640 1 -- 172.21.15.60:0/1002721046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22880116b0 con 0x7f2298102d40 2024-02-26T05:00:18.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.792+0000 7f229c91e640 1 -- 172.21.15.60:0/1002721046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f229819f820 con 0x7f2298102d40 2024-02-26T05:00:18.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.793+0000 7f229c91e640 1 -- 172.21.15.60:0/1002721046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2298108530 con 0x7f2298102d40 2024-02-26T05:00:18.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.794+0000 7f228f7fe640 1 -- 172.21.15.60:0/1002721046 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2288005520 con 0x7f2298102d40 2024-02-26T05:00:18.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.794+0000 7f228f7fe640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f22840a2140 0x7f22840a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:18.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.795+0000 7f229659c640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f22840a2140 0x7f22840a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:18.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.795+0000 7f228f7fe640 1 -- 172.21.15.60:0/1002721046 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9582+0+0 (secure 0 0 0) 0x7f22880ce400 con 0x7f2298102d40 2024-02-26T05:00:18.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.795+0000 7f229659c640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f22840a2140 0x7f22840a4600 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2290005fd0 tx=0x7f229000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:18.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.799+0000 7f228f7fe640 1 -- 172.21.15.60:0/1002721046 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2288096e30 con 0x7f2298102d40 2024-02-26T05:00:18.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:18.981+0000 7f229c91e640 1 -- 172.21.15.60:0/1002721046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f229810cba0 con 0x7f2298102d40 2024-02-26T05:00:19.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.660+0000 7f228f7fe640 1 -- 172.21.15.60:0/1002721046 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) v1 ==== 181+0+0 (secure 0 0 0) 0x7f228801f0d0 con 0x7f2298102d40 2024-02-26T05:00:19.664 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-02-26T05:00:19.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.664+0000 7f229c91e640 1 -- 172.21.15.60:0/1002721046 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f22840a2140 msgr2=0x7f22840a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:19.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.664+0000 7f229c91e640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f22840a2140 0x7f22840a4600 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2290005fd0 tx=0x7f229000b410 comp rx=0 tx=0).stop 2024-02-26T05:00:19.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.664+0000 7f229c91e640 1 -- 172.21.15.60:0/1002721046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2298102d40 msgr2=0x7f229819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:19.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.664+0000 7f229c91e640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2298102d40 0x7f229819e7c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f228800eaa0 tx=0x7f228800ef60 comp rx=0 tx=0).stop 2024-02-26T05:00:19.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.664+0000 7f229c91e640 1 -- 172.21.15.60:0/1002721046 shutdown_connections 2024-02-26T05:00:19.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.664+0000 7f229c91e640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f22840a2140 0x7f22840a4600 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:19.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.664+0000 7f229c91e640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2298102d40 0x7f229819e7c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:19.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.664+0000 7f229c91e640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2298102380 0x7f2298197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:19.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.664+0000 7f229c91e640 1 --2- 172.21.15.60:0/1002721046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2298101190 0x7f2298197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:19.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.664+0000 7f229c91e640 1 -- 172.21.15.60:0/1002721046 >> 172.21.15.60:0/1002721046 conn(0x7f22980fc720 msgr2=0x7f2298105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:19.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.665+0000 7f229c91e640 1 -- 172.21.15.60:0/1002721046 shutdown_connections 2024-02-26T05:00:19.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.665+0000 7f229c91e640 1 -- 172.21.15.60:0/1002721046 wait complete. 2024-02-26T05:00:19.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-26T05:00:19.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: Cluster is now healthy 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/3122510027' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.156:6835/2744004159,v1:172.21.15.156:6839/2744004159] up:boot 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.191:6835/114180690,v1:172.21.15.191:6838/114180690] up:boot 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.191:6836/3845493975,v1:172.21.15.191:6839/3845493975] up:boot 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.191:6834/2465598014,v1:172.21.15.191:6837/2465598014] up:boot 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.156:6834/2400015199,v1:172.21.15.156:6838/2400015199] up:boot 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: mds.? [v2:172.21.15.156:6836/1909536719,v1:172.21.15.156:6837/1909536719] up:boot 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: fsmap 12 up:standby 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T05:00:19.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T05:00:19.679 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:19 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1002721046' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T05:00:19.679 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-26T05:00:19.679 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-26T05:00:19.679 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: Cluster is now healthy 2024-02-26T05:00:19.680 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/3122510027' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-26T05:00:19.680 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.156:6835/2744004159,v1:172.21.15.156:6839/2744004159] up:boot 2024-02-26T05:00:19.680 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.191:6835/114180690,v1:172.21.15.191:6838/114180690] up:boot 2024-02-26T05:00:19.680 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.191:6836/3845493975,v1:172.21.15.191:6839/3845493975] up:boot 2024-02-26T05:00:19.680 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.191:6834/2465598014,v1:172.21.15.191:6837/2465598014] up:boot 2024-02-26T05:00:19.680 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.156:6834/2400015199,v1:172.21.15.156:6838/2400015199] up:boot 2024-02-26T05:00:19.680 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: mds.? [v2:172.21.15.156:6836/1909536719,v1:172.21.15.156:6837/1909536719] up:boot 2024-02-26T05:00:19.680 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: fsmap 12 up:standby 2024-02-26T05:00:19.680 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T05:00:19.680 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T05:00:19.680 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T05:00:19.680 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T05:00:19.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T05:00:19.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T05:00:19.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:19 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1002721046' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T05:00:19.682 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-26T05:00:19.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-26T05:00:19.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-26T05:00:19.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: Cluster is now healthy 2024-02-26T05:00:19.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/3122510027' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-26T05:00:19.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.156:6835/2744004159,v1:172.21.15.156:6839/2744004159] up:boot 2024-02-26T05:00:19.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.191:6835/114180690,v1:172.21.15.191:6838/114180690] up:boot 2024-02-26T05:00:19.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.191:6836/3845493975,v1:172.21.15.191:6839/3845493975] up:boot 2024-02-26T05:00:19.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.191:6834/2465598014,v1:172.21.15.191:6837/2465598014] up:boot 2024-02-26T05:00:19.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.156:6834/2400015199,v1:172.21.15.156:6838/2400015199] up:boot 2024-02-26T05:00:19.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: mds.? [v2:172.21.15.156:6836/1909536719,v1:172.21.15.156:6837/1909536719] up:boot 2024-02-26T05:00:19.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: fsmap 12 up:standby 2024-02-26T05:00:19.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-26T05:00:19.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-26T05:00:19.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-26T05:00:19.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-26T05:00:19.760 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-26T05:00:19.761 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: from='mgr.14152 172.21.15.60:0/3135965668' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-26T05:00:19.761 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:19 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1002721046' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T05:00:19.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.788+0000 7f5bc10d5640 1 -- 172.21.15.60:0/1907297374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc101c20 msgr2=0x7f5bbc104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:19.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.788+0000 7f5bc10d5640 1 --2- 172.21.15.60:0/1907297374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc101c20 0x7f5bbc104010 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009900 tx=0x7f5bac02bc30 comp rx=0 tx=0).stop 2024-02-26T05:00:19.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.788+0000 7f5bc10d5640 1 -- 172.21.15.60:0/1907297374 shutdown_connections 2024-02-26T05:00:19.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.788+0000 7f5bc10d5640 1 --2- 172.21.15.60:0/1907297374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bbc106e80 0x7f5bbc1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:19.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.788+0000 7f5bc10d5640 1 --2- 172.21.15.60:0/1907297374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bbc104550 0x7f5bbc106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:19.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.788+0000 7f5bc10d5640 1 --2- 172.21.15.60:0/1907297374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc101c20 0x7f5bbc104010 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:19.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.788+0000 7f5bc10d5640 1 -- 172.21.15.60:0/1907297374 >> 172.21.15.60:0/1907297374 conn(0x7f5bbc0fb630 msgr2=0x7f5bbc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:19.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.788+0000 7f5bc10d5640 1 -- 172.21.15.60:0/1907297374 shutdown_connections 2024-02-26T05:00:19.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.788+0000 7f5bc10d5640 1 -- 172.21.15.60:0/1907297374 wait complete. 2024-02-26T05:00:19.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.788+0000 7f5bc10d5640 1 Processor -- start 2024-02-26T05:00:19.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bc10d5640 1 -- start start 2024-02-26T05:00:19.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bc10d5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bbc101c20 0x7f5bbc197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:19.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bc10d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc104550 0x7f5bbc197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:19.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bc10d5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bbc106e80 0x7f5bbc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:19.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bc10d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc19edb0 con 0x7f5bbc104550 2024-02-26T05:00:19.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bc10d5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc19ef20 con 0x7f5bbc101c20 2024-02-26T05:00:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bc10d5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc19f0e0 con 0x7f5bbc106e80 2024-02-26T05:00:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bba59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc104550 0x7f5bbc197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bba59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc104550 0x7f5bbc197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57492/0 (socket says 172.21.15.60:57492) 2024-02-26T05:00:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bba59c640 1 -- 172.21.15.60:0/53653118 learned_addr learned my addr 172.21.15.60:0/53653118 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T05:00:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bbb59e640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bbc106e80 0x7f5bbc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bbad9d640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bbc101c20 0x7f5bbc197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bba59c640 1 -- 172.21.15.60:0/53653118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bbc106e80 msgr2=0x7f5bbc19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bba59c640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bbc106e80 0x7f5bbc19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bba59c640 1 -- 172.21.15.60:0/53653118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bbc101c20 msgr2=0x7f5bbc197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bba59c640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bbc101c20 0x7f5bbc197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bba59c640 1 -- 172.21.15.60:0/53653118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac009560 con 0x7f5bbc104550 2024-02-26T05:00:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bbad9d640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bbc101c20 0x7f5bbc197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T05:00:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.789+0000 7f5bba59c640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc104550 0x7f5bbc197850 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000c970 tx=0x7f5bb000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.790+0000 7f5ba3fff640 1 -- 172.21.15.60:0/53653118 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0007bb0 con 0x7f5bbc104550 2024-02-26T05:00:19.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.790+0000 7f5bc10d5640 1 -- 172.21.15.60:0/53653118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bbc19f3c0 con 0x7f5bbc104550 2024-02-26T05:00:19.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.790+0000 7f5ba3fff640 1 -- 172.21.15.60:0/53653118 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bb0007d10 con 0x7f5bbc104550 2024-02-26T05:00:19.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.790+0000 7f5bc10d5640 1 -- 172.21.15.60:0/53653118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bbc19f9a0 con 0x7f5bbc104550 2024-02-26T05:00:19.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.790+0000 7f5ba3fff640 1 -- 172.21.15.60:0/53653118 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000f450 con 0x7f5bbc104550 2024-02-26T05:00:19.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.791+0000 7f5bc10d5640 1 -- 172.21.15.60:0/53653118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bbc0fd120 con 0x7f5bbc104550 2024-02-26T05:00:19.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.792+0000 7f5ba3fff640 1 -- 172.21.15.60:0/53653118 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5bb0004f20 con 0x7f5bbc104550 2024-02-26T05:00:19.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.792+0000 7f5ba3fff640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ba40a20c0 0x7f5ba40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:19.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.792+0000 7f5bbad9d640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ba40a20c0 0x7f5ba40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:19.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.792+0000 7f5ba3fff640 1 -- 172.21.15.60:0/53653118 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f5bb00ce060 con 0x7f5bbc104550 2024-02-26T05:00:19.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.793+0000 7f5bbad9d640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ba40a20c0 0x7f5ba40a4580 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5bac005ec0 tx=0x7f5bac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:19.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.797+0000 7f5ba3fff640 1 -- 172.21.15.60:0/53653118 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5bb0096d10 con 0x7f5bbc104550 2024-02-26T05:00:19.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:19.977+0000 7f5bc10d5640 1 -- 172.21.15.60:0/53653118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5bbc0fcb90 con 0x7f5bbc104550 2024-02-26T05:00:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.663+0000 7f5ba3fff640 1 -- 172.21.15.60:0/53653118 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5bb009a3d0 con 0x7f5bbc104550 2024-02-26T05:00:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-02-26T05:00:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.666+0000 7f5bc10d5640 1 -- 172.21.15.60:0/53653118 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ba40a20c0 msgr2=0x7f5ba40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.666+0000 7f5bc10d5640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ba40a20c0 0x7f5ba40a4580 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5bac005ec0 tx=0x7f5bac005fd0 comp rx=0 tx=0).stop 2024-02-26T05:00:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.666+0000 7f5bc10d5640 1 -- 172.21.15.60:0/53653118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc104550 msgr2=0x7f5bbc197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.666+0000 7f5bc10d5640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc104550 0x7f5bbc197850 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000c970 tx=0x7f5bb000ce30 comp rx=0 tx=0).stop 2024-02-26T05:00:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.666+0000 7f5bc10d5640 1 -- 172.21.15.60:0/53653118 shutdown_connections 2024-02-26T05:00:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.666+0000 7f5bc10d5640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7f5ba40a20c0 0x7f5ba40a4580 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.666+0000 7f5bc10d5640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bbc106e80 0x7f5bbc19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.666+0000 7f5bc10d5640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc104550 0x7f5bbc197850 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.666+0000 7f5bc10d5640 1 --2- 172.21.15.60:0/53653118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bbc101c20 0x7f5bbc197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.666+0000 7f5bc10d5640 1 -- 172.21.15.60:0/53653118 >> 172.21.15.60:0/53653118 conn(0x7f5bbc0fb630 msgr2=0x7f5bbc102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.667+0000 7f5bc10d5640 1 -- 172.21.15.60:0/53653118 shutdown_connections 2024-02-26T05:00:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.667+0000 7f5bc10d5640 1 -- 172.21.15.60:0/53653118 wait complete. 2024-02-26T05:00:20.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:20 smithi156 ceph-mon[35805]: pgmap v444: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 7 op/s 2024-02-26T05:00:20.677 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:20 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/1002721046' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T05:00:20.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:20 smithi156 ceph-mon[35805]: osdmap e95: 12 total, 12 up, 12 in 2024-02-26T05:00:20.678 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:20 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/53653118' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T05:00:20.683 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:20 smithi060 ceph-mon[32382]: pgmap v444: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 7 op/s 2024-02-26T05:00:20.683 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:20 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/1002721046' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T05:00:20.683 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:20 smithi060 ceph-mon[32382]: osdmap e95: 12 total, 12 up, 12 in 2024-02-26T05:00:20.683 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:20 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/53653118' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T05:00:20.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-02-26T05:00:20.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:20 smithi191 ceph-mon[36352]: pgmap v444: 129 pgs: 129 active+clean; 13 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 7 op/s 2024-02-26T05:00:20.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:20 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/1002721046' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T05:00:20.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:20 smithi191 ceph-mon[36352]: osdmap e95: 12 total, 12 up, 12 in 2024-02-26T05:00:20.759 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:20 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/53653118' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T05:00:20.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.793+0000 7fe5f9836640 1 -- 172.21.15.60:0/3568093282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4101290 msgr2=0x7fe5f4101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:20.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.793+0000 7fe5f9836640 1 --2- 172.21.15.60:0/3568093282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4101290 0x7fe5f4101710 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009900 tx=0x7fe5dc02bc30 comp rx=0 tx=0).stop 2024-02-26T05:00:20.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 -- 172.21.15.60:0/3568093282 shutdown_connections 2024-02-26T05:00:20.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 --2- 172.21.15.60:0/3568093282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5f4101c50 0x7fe5f41084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:20.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 --2- 172.21.15.60:0/3568093282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4101290 0x7fe5f4101710 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:20.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 --2- 172.21.15.60:0/3568093282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f41000a0 0x7fe5f41004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:20.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 -- 172.21.15.60:0/3568093282 >> 172.21.15.60:0/3568093282 conn(0x7fe5f40fb630 msgr2=0x7fe5f40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:20.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 -- 172.21.15.60:0/3568093282 shutdown_connections 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 -- 172.21.15.60:0/3568093282 wait complete. 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 Processor -- start 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 -- start start 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5f41000a0 0x7fe5f406a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4101290 0x7fe5f419c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f4101c50 0x7fe5f419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f419ef20 con 0x7fe5f4101290 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f419f090 con 0x7fe5f4101c50 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.794+0000 7fe5f9836640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f419f250 con 0x7fe5f41000a0 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4101290 0x7fe5f419c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4101290 0x7fe5f419c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57506/0 (socket says 172.21.15.60:57506) 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f27fc640 1 -- 172.21.15.60:0/4080819553 learned_addr learned my addr 172.21.15.60:0/4080819553 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f37fe640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f4101c50 0x7fe5f419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f27fc640 1 -- 172.21.15.60:0/4080819553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5f41000a0 msgr2=0x7fe5f406a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:20.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f2ffd640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5f41000a0 0x7fe5f406a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:20.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f27fc640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5f41000a0 0x7fe5f406a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:20.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f27fc640 1 -- 172.21.15.60:0/4080819553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f4101c50 msgr2=0x7fe5f419e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:20.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f27fc640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f4101c50 0x7fe5f419e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:20.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f27fc640 1 -- 172.21.15.60:0/4080819553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc009560 con 0x7fe5f4101290 2024-02-26T05:00:20.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f37fe640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f4101c50 0x7fe5f419e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T05:00:20.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f27fc640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4101290 0x7fe5f419c120 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc005ab0 tx=0x7fe5dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:20.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f8834640 1 -- 172.21.15.60:0/4080819553 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0044e0 con 0x7fe5f4101290 2024-02-26T05:00:20.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f9836640 1 -- 172.21.15.60:0/4080819553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f419f470 con 0x7fe5f4101290 2024-02-26T05:00:20.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f8834640 1 -- 172.21.15.60:0/4080819553 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5dc0054e0 con 0x7fe5f4101290 2024-02-26T05:00:20.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f8834640 1 -- 172.21.15.60:0/4080819553 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc03e450 con 0x7fe5f4101290 2024-02-26T05:00:20.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.795+0000 7fe5f9836640 1 -- 172.21.15.60:0/4080819553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f419fa00 con 0x7fe5f4101290 2024-02-26T05:00:20.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.796+0000 7fe5f9836640 1 -- 172.21.15.60:0/4080819553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f4107500 con 0x7fe5f4101290 2024-02-26T05:00:20.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.797+0000 7fe5f8834640 1 -- 172.21.15.60:0/4080819553 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fe5dc0035f0 con 0x7fe5f4101290 2024-02-26T05:00:20.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.798+0000 7fe5f8834640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5e00a20c0 0x7fe5e00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T05:00:20.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.798+0000 7fe5f8834640 1 -- 172.21.15.60:0/4080819553 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8324+0+0 (secure 0 0 0) 0x7fe5dc0f0310 con 0x7fe5f4101290 2024-02-26T05:00:20.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.798+0000 7fe5f2ffd640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5e00a20c0 0x7fe5e00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T05:00:20.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.798+0000 7fe5f2ffd640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5e00a20c0 0x7fe5e00a4580 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe5f41010f0 tx=0x7fe5ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T05:00:20.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.802+0000 7fe5f8834640 1 -- 172.21.15.60:0/4080819553 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe5dc0b9280 con 0x7fe5f4101290 2024-02-26T05:00:20.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:20.982+0000 7fe5f9836640 1 -- 172.21.15.60:0/4080819553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe5f410bb90 con 0x7fe5f4101290 2024-02-26T05:00:21.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:21.668+0000 7fe5f8834640 1 -- 172.21.15.60:0/4080819553 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v97) v1 ==== 178+0+0 (secure 0 0 0) 0x7fe5dc0bc940 con 0x7fe5f4101290 2024-02-26T05:00:21.671 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data_ec' removed 2024-02-26T05:00:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:21.673+0000 7fe5f9836640 1 -- 172.21.15.60:0/4080819553 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5e00a20c0 msgr2=0x7fe5e00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:21.673+0000 7fe5f9836640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5e00a20c0 0x7fe5e00a4580 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe5f41010f0 tx=0x7fe5ec008040 comp rx=0 tx=0).stop 2024-02-26T05:00:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:21.673+0000 7fe5f9836640 1 -- 172.21.15.60:0/4080819553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4101290 msgr2=0x7fe5f419c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T05:00:21.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:21.673+0000 7fe5f9836640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4101290 0x7fe5f419c120 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc005ab0 tx=0x7fe5dc0042f0 comp rx=0 tx=0).stop 2024-02-26T05:00:21.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:21.673+0000 7fe5f9836640 1 -- 172.21.15.60:0/4080819553 shutdown_connections 2024-02-26T05:00:21.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:21.673+0000 7fe5f9836640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.60:6800/3458567419,v1:172.21.15.60:6801/3458567419] conn(0x7fe5e00a20c0 0x7fe5e00a4580 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:21.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:21.673+0000 7fe5f9836640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f4101c50 0x7fe5f419e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:21.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:21.673+0000 7fe5f9836640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5f4101290 0x7fe5f419c120 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:21.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:21.673+0000 7fe5f9836640 1 --2- 172.21.15.60:0/4080819553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5f41000a0 0x7fe5f406a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T05:00:21.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:21.673+0000 7fe5f9836640 1 -- 172.21.15.60:0/4080819553 >> 172.21.15.60:0/4080819553 conn(0x7fe5f40fb630 msgr2=0x7fe5f41046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T05:00:21.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:21.673+0000 7fe5f9836640 1 -- 172.21.15.60:0/4080819553 shutdown_connections 2024-02-26T05:00:21.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-02-26T05:00:21.673+0000 7fe5f9836640 1 -- 172.21.15.60:0/4080819553 wait complete. 2024-02-26T05:00:21.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T05:00:21.726 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T05:00:21.761 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T05:00:21.796 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-26T05:00:21.796 DEBUG:teuthology.orchestra.run.smithi060:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-26T05:00:21.812 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-26T05:00:21.829 DEBUG:teuthology.orchestra.run.smithi191:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-26T05:00:21.853 INFO:tasks.cephadm:Stopping all daemons... 2024-02-26T05:00:21.853 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-26T05:00:21.853 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.a 2024-02-26T05:00:21.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:21 smithi156 ceph-mon[35805]: pgmap v446: 97 pgs: 97 active+clean; 577 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T05:00:21.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:21 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/53653118' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T05:00:21.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:21 smithi156 ceph-mon[35805]: osdmap e96: 12 total, 12 up, 12 in 2024-02-26T05:00:21.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:21 smithi156 ceph-mon[35805]: from='client.? 172.21.15.60:0/4080819553' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T05:00:21.979 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:21 smithi060 ceph-mon[32382]: pgmap v446: 97 pgs: 97 active+clean; 577 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T05:00:21.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:21 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/53653118' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T05:00:21.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:21 smithi060 ceph-mon[32382]: osdmap e96: 12 total, 12 up, 12 in 2024-02-26T05:00:21.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:21 smithi060 ceph-mon[32382]: from='client.? 172.21.15.60:0/4080819553' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T05:00:21.980 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:21 smithi060 systemd[1]: Stopping Ceph mon.a for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:00:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:21 smithi191 ceph-mon[36352]: pgmap v446: 97 pgs: 97 active+clean; 577 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-26T05:00:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:21 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/53653118' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-26T05:00:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:21 smithi191 ceph-mon[36352]: osdmap e96: 12 total, 12 up, 12 in 2024-02-26T05:00:22.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:21 smithi191 ceph-mon[36352]: from='client.? 172.21.15.60:0/4080819553' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-26T05:00:22.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:22 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a[32378]: 2024-02-26T05:00:22.150+0000 7f90a533c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:00:22.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:22 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a[32378]: 2024-02-26T05:00:22.150+0000 7f90a533c700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-26T05:00:22.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:22 smithi060 podman[88664]: 2024-02-26 05:00:22.294145575 +0000 UTC m=+0.254636507 container died 5b57dca8a03365e72efd9db9c4e871bf8c64c8eede3518e5747bac26991619b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-26T05:00:22.955 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:22 smithi060 podman[88664]: 2024-02-26 05:00:22.768981828 +0000 UTC m=+0.729472757 container cleanup 5b57dca8a03365e72efd9db9c4e871bf8c64c8eede3518e5747bac26991619b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-26T05:00:22.955 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:22 smithi060 bash[88664]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a 2024-02-26T05:00:23.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:Feb 26 05:00:23 smithi060 podman[88678]: 2024-02-26 05:00:23.079350432 +0000 UTC m=+0.784422520 container remove 5b57dca8a03365e72efd9db9c4e871bf8c64c8eede3518e5747bac26991619b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2) 2024-02-26T05:00:23.242 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.a.service' 2024-02-26T05:00:23.303 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:00:23.303 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-26T05:00:23.303 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-26T05:00:23.303 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.b 2024-02-26T05:00:23.618 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:23 smithi156 systemd[1]: Stopping Ceph mon.b for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:00:23.927 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:23 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-b[35801]: 2024-02-26T05:00:23.615+0000 7f43f9e7a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:00:23.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:23 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-b[35801]: 2024-02-26T05:00:23.615+0000 7f43f9e7a700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-26T05:00:23.928 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:23 smithi156 podman[68755]: 2024-02-26 05:00:23.73849852 +0000 UTC m=+0.245156525 container died f61d55a72bbf863d3848021d1ec9a98efa19fa3802d592732b1ef08ff9a9ae46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-26T05:00:24.395 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:24 smithi156 podman[68755]: 2024-02-26 05:00:24.158256841 +0000 UTC m=+0.664914847 container cleanup f61d55a72bbf863d3848021d1ec9a98efa19fa3802d592732b1ef08ff9a9ae46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-b, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD) 2024-02-26T05:00:24.395 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:24 smithi156 bash[68755]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-b 2024-02-26T05:00:24.624 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.b.service' 2024-02-26T05:00:24.648 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:24 smithi156 podman[68769]: 2024-02-26 05:00:24.39338731 +0000 UTC m=+0.650344891 container remove f61d55a72bbf863d3848021d1ec9a98efa19fa3802d592732b1ef08ff9a9ae46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-26T05:00:24.648 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:24 smithi156 systemd[1]: ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.b.service: Deactivated successfully. 2024-02-26T05:00:24.648 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:24 smithi156 systemd[1]: Stopped Ceph mon.b for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T05:00:24.648 INFO:journalctl@ceph.mon.b.smithi156.stdout:Feb 26 05:00:24 smithi156 systemd[1]: ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.b.service: Consumed 8.373s CPU time. 2024-02-26T05:00:24.684 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:00:24.684 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-26T05:00:24.684 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-26T05:00:24.684 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.c 2024-02-26T05:00:25.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:24 smithi191 systemd[1]: Stopping Ceph mon.c for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:00:25.364 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:25 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-c[36348]: 2024-02-26T05:00:25.009+0000 7f9e0e57f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:00:25.364 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:25 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-c[36348]: 2024-02-26T05:00:25.009+0000 7f9e0e57f700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-26T05:00:25.364 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:25 smithi191 podman[69814]: 2024-02-26 05:00:25.060824277 +0000 UTC m=+0.185474013 container died 462f964553c96d22fff62bdc55db613ec249122b2b18ba64d9e5142730c8c3e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-02-26T05:00:25.734 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:25 smithi191 podman[69814]: 2024-02-26 05:00:25.461232818 +0000 UTC m=+0.585882545 container cleanup 462f964553c96d22fff62bdc55db613ec249122b2b18ba64d9e5142730c8c3e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-c, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T05:00:25.734 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:25 smithi191 bash[69814]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-c 2024-02-26T05:00:26.007 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 26 05:00:25 smithi191 podman[69828]: 2024-02-26 05:00:25.732547892 +0000 UTC m=+0.664386025 container remove 462f964553c96d22fff62bdc55db613ec249122b2b18ba64d9e5142730c8c3e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mon-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=) 2024-02-26T05:00:26.033 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mon.c.service' 2024-02-26T05:00:26.084 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:00:26.085 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-26T05:00:26.085 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-26T05:00:26.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mgr.x 2024-02-26T05:00:26.401 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 05:00:26 smithi060 systemd[1]: Stopping Ceph mgr.x for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:00:26.727 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 05:00:26 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T05:00:26.398+0000 7fc8a4f07700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:00:26.727 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 05:00:26 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x[32615]: 2024-02-26T05:00:26.398+0000 7fc8a4f07700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-26T05:00:26.727 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 05:00:26 smithi060 podman[88786]: 2024-02-26 05:00:26.430284158 +0000 UTC m=+0.157493802 container died 81ee7493b7f617602fb5de8025f4c478285e0df134ded4ac1513df1b939b3ed4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-26T05:00:26.980 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 05:00:26 smithi060 podman[88786]: 2024-02-26 05:00:26.825102655 +0000 UTC m=+0.552312304 container cleanup 81ee7493b7f617602fb5de8025f4c478285e0df134ded4ac1513df1b939b3ed4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:25296a8f176256aea5ed8b5850b25167dce0ae33, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-26T05:00:26.980 INFO:journalctl@ceph.mgr.x.smithi060.stdout:Feb 26 05:00:26 smithi060 bash[88786]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-x 2024-02-26T05:00:27.243 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mgr.x.service' 2024-02-26T05:00:27.281 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:00:27.281 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-26T05:00:27.281 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-26T05:00:27.281 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mgr.y 2024-02-26T05:00:27.611 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 05:00:27 smithi156 systemd[1]: Stopping Ceph mgr.y for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:00:27.908 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 05:00:27 smithi156 podman[68879]: 2024-02-26 05:00:27.609361173 +0000 UTC m=+0.139765128 container died 5d25ce3df1b78bc61026a60744a8f11b97c53856bf115f0584e41eed0b821259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-26T05:00:28.177 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 05:00:28 smithi156 podman[68879]: 2024-02-26 05:00:28.00576368 +0000 UTC m=+0.536167615 container cleanup 5d25ce3df1b78bc61026a60744a8f11b97c53856bf115f0584e41eed0b821259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T05:00:28.178 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Feb 26 05:00:28 smithi156 bash[68879]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-y 2024-02-26T05:00:28.421 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mgr.y.service' 2024-02-26T05:00:28.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:00:28.458 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-26T05:00:28.458 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-26T05:00:28.458 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mgr.z 2024-02-26T05:00:28.759 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 05:00:28 smithi191 systemd[1]: Stopping Ceph mgr.z for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:00:29.114 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 05:00:28 smithi191 podman[69938]: 2024-02-26 05:00:28.807575316 +0000 UTC m=+0.156789538 container died 4bee1c9c1140ed178f9e322f8be99733673993fd7a1a25c6ee662dc65bbb5c28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-26T05:00:29.464 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 05:00:29 smithi191 podman[69938]: 2024-02-26 05:00:29.219833385 +0000 UTC m=+0.569047594 container cleanup 4bee1c9c1140ed178f9e322f8be99733673993fd7a1a25c6ee662dc65bbb5c28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_CLEAN=True) 2024-02-26T05:00:29.465 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 05:00:29 smithi191 bash[69938]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z 2024-02-26T05:00:29.715 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 05:00:29 smithi191 podman[69952]: 2024-02-26 05:00:29.463144293 +0000 UTC m=+0.653911568 container remove 4bee1c9c1140ed178f9e322f8be99733673993fd7a1a25c6ee662dc65bbb5c28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mgr-z, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-02-26T05:00:29.716 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 26 05:00:29 smithi191 systemd[1]: ceph-f12e2206-d461-11ee-95c3-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-02-26T05:00:29.719 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mgr.z.service' 2024-02-26T05:00:29.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:00:29.759 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-26T05:00:29.759 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-26T05:00:29.760 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.0 2024-02-26T05:00:30.082 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 05:00:29 smithi060 systemd[1]: Stopping Ceph osd.0 for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:00:30.479 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 05:00:30 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-0[44691]: 2024-02-26T05:00:30.079+0000 7fc3c7a3e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:00:30.480 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 05:00:30 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-0[44691]: 2024-02-26T05:00:30.079+0000 7fc3c7a3e700 -1 osd.0 97 *** Got signal Terminated *** 2024-02-26T05:00:30.480 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 05:00:30 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-0[44691]: 2024-02-26T05:00:30.079+0000 7fc3c7a3e700 -1 osd.0 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T05:00:35.730 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 05:00:35 smithi060 podman[88905]: 2024-02-26 05:00:35.422189233 +0000 UTC m=+5.471920515 container died 4164e8fbb76496065dd6c7c558bac4c0f6134b7ba21b568f07a485e9d0fa660c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-26T05:00:36.075 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 05:00:35 smithi060 podman[88905]: 2024-02-26 05:00:35.830455841 +0000 UTC m=+5.880187142 container cleanup 4164e8fbb76496065dd6c7c558bac4c0f6134b7ba21b568f07a485e9d0fa660c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T05:00:36.075 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 05:00:35 smithi060 bash[88905]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-0 2024-02-26T05:00:36.384 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 05:00:36 smithi060 podman[88925]: 2024-02-26 05:00:36.073730332 +0000 UTC m=+0.651842248 container remove 4164e8fbb76496065dd6c7c558bac4c0f6134b7ba21b568f07a485e9d0fa660c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, ceph=True, org.label-schema.schema-version=1.0) 2024-02-26T05:00:36.641 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 05:00:36 smithi060 podman[89011]: 2024-02-26 05:00:36.285255555 +0000 UTC m=+0.018157416 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T05:00:36.642 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 05:00:36 smithi060 podman[89011]: 2024-02-26 05:00:36.391437502 +0000 UTC m=+0.124339397 container create cbaa4da8c09c4c9ba7d13abe1576ccc2c73fe2555b895e471f30a149d1db807d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-02-26T05:00:36.979 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 05:00:36 smithi060 podman[89011]: 2024-02-26 05:00:36.677883375 +0000 UTC m=+0.410785226 container init cbaa4da8c09c4c9ba7d13abe1576ccc2c73fe2555b895e471f30a149d1db807d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T05:00:36.979 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 05:00:36 smithi060 podman[89011]: 2024-02-26 05:00:36.681439761 +0000 UTC m=+0.414341620 container start cbaa4da8c09c4c9ba7d13abe1576ccc2c73fe2555b895e471f30a149d1db807d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-26T05:00:36.980 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 05:00:36 smithi060 podman[89011]: 2024-02-26 05:00:36.786264361 +0000 UTC m=+0.519166229 container attach cbaa4da8c09c4c9ba7d13abe1576ccc2c73fe2555b895e471f30a149d1db807d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-0-deactivate, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T05:00:37.229 INFO:journalctl@ceph.osd.0.smithi060.stdout:Feb 26 05:00:36 smithi060 podman[89011]: 2024-02-26 05:00:36.977839629 +0000 UTC m=+0.710741486 container died cbaa4da8c09c4c9ba7d13abe1576ccc2c73fe2555b895e471f30a149d1db807d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-26T05:00:37.788 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.0.service' 2024-02-26T05:00:37.836 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:00:37.836 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-26T05:00:37.837 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-26T05:00:37.837 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.1 2024-02-26T05:00:38.146 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 05:00:37 smithi156 systemd[1]: Stopping Ceph osd.1 for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:00:38.427 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 05:00:38 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-1[40415]: 2024-02-26T05:00:38.143+0000 7f877c1b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:00:38.428 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 05:00:38 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-1[40415]: 2024-02-26T05:00:38.143+0000 7f877c1b0700 -1 osd.1 97 *** Got signal Terminated *** 2024-02-26T05:00:38.428 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 05:00:38 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-1[40415]: 2024-02-26T05:00:38.143+0000 7f877c1b0700 -1 osd.1 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T05:00:43.996 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 05:00:43 smithi156 podman[69004]: 2024-02-26 05:00:43.690422633 +0000 UTC m=+5.669861213 container died 93efe0a029c6e04193323bf87915430f7b6d02dd52aed655f768eff1910aa751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-1, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T05:00:44.347 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 05:00:44 smithi156 podman[69004]: 2024-02-26 05:00:44.101963859 +0000 UTC m=+6.081402433 container cleanup 93efe0a029c6e04193323bf87915430f7b6d02dd52aed655f768eff1910aa751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-26T05:00:44.347 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 05:00:44 smithi156 bash[69004]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-1 2024-02-26T05:00:44.677 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 05:00:44 smithi156 podman[69021]: 2024-02-26 05:00:44.345126684 +0000 UTC m=+0.652780000 container remove 93efe0a029c6e04193323bf87915430f7b6d02dd52aed655f768eff1910aa751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True) 2024-02-26T05:00:44.983 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 05:00:44 smithi156 podman[69122]: 2024-02-26 05:00:44.606110449 +0000 UTC m=+0.023929802 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T05:00:44.984 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 05:00:44 smithi156 podman[69122]: 2024-02-26 05:00:44.723655038 +0000 UTC m=+0.141474379 container create ccec3961377d7a9f1c56e7a1a509341367392e144c60c70af294860873b7e98e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-02-26T05:00:45.289 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 05:00:44 smithi156 podman[69122]: 2024-02-26 05:00:44.982066332 +0000 UTC m=+0.399885692 container init ccec3961377d7a9f1c56e7a1a509341367392e144c60c70af294860873b7e98e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-26T05:00:45.289 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 05:00:44 smithi156 podman[69122]: 2024-02-26 05:00:44.985489657 +0000 UTC m=+0.403309007 container start ccec3961377d7a9f1c56e7a1a509341367392e144c60c70af294860873b7e98e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-1-deactivate, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-26T05:00:45.289 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 05:00:45 smithi156 podman[69122]: 2024-02-26 05:00:45.082139723 +0000 UTC m=+0.499959084 container attach ccec3961377d7a9f1c56e7a1a509341367392e144c60c70af294860873b7e98e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-26T05:00:45.654 INFO:journalctl@ceph.osd.1.smithi156.stdout:Feb 26 05:00:45 smithi156 podman[69122]: 2024-02-26 05:00:45.287868017 +0000 UTC m=+0.705687359 container died ccec3961377d7a9f1c56e7a1a509341367392e144c60c70af294860873b7e98e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2) 2024-02-26T05:00:45.977 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.1.service' 2024-02-26T05:00:46.019 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:00:46.019 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-26T05:00:46.019 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-26T05:00:46.019 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.2 2024-02-26T05:00:46.335 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 05:00:46 smithi191 systemd[1]: Stopping Ceph osd.2 for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:00:46.759 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 05:00:46 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-2[41516]: 2024-02-26T05:00:46.331+0000 7f9405ae0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:00:46.759 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 05:00:46 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-2[41516]: 2024-02-26T05:00:46.331+0000 7f9405ae0700 -1 osd.2 97 *** Got signal Terminated *** 2024-02-26T05:00:46.759 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 05:00:46 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-2[41516]: 2024-02-26T05:00:46.331+0000 7f9405ae0700 -1 osd.2 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T05:00:52.102 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 05:00:51 smithi191 podman[70073]: 2024-02-26 05:00:51.796057306 +0000 UTC m=+5.586280062 container died c3eb7d483b685959474a85e3f5bbb46457b0d269f985cca7ffbe4c27f84dab18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T05:00:52.469 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 05:00:52 smithi191 podman[70073]: 2024-02-26 05:00:52.207319457 +0000 UTC m=+5.997542207 container cleanup c3eb7d483b685959474a85e3f5bbb46457b0d269f985cca7ffbe4c27f84dab18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-26T05:00:52.469 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 05:00:52 smithi191 bash[70073]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-2 2024-02-26T05:00:52.759 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 05:00:52 smithi191 podman[70090]: 2024-02-26 05:00:52.467028554 +0000 UTC m=+0.668565594 container remove c3eb7d483b685959474a85e3f5bbb46457b0d269f985cca7ffbe4c27f84dab18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-26T05:00:53.198 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 05:00:52 smithi191 podman[70184]: 2024-02-26 05:00:52.778875456 +0000 UTC m=+0.017712972 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T05:00:53.198 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 05:00:52 smithi191 podman[70184]: 2024-02-26 05:00:52.926363584 +0000 UTC m=+0.165201083 container create 56fba5f25525c35b62f5c686faf5f2af332840a4cddbed13330308d4ffd04020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-26T05:00:53.487 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 05:00:53 smithi191 podman[70184]: 2024-02-26 05:00:53.196017851 +0000 UTC m=+0.434855367 container init 56fba5f25525c35b62f5c686faf5f2af332840a4cddbed13330308d4ffd04020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-26T05:00:53.488 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 05:00:53 smithi191 podman[70184]: 2024-02-26 05:00:53.199585933 +0000 UTC m=+0.438423439 container start 56fba5f25525c35b62f5c686faf5f2af332840a4cddbed13330308d4ffd04020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-26T05:00:53.488 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 05:00:53 smithi191 podman[70184]: 2024-02-26 05:00:53.329384989 +0000 UTC m=+0.568222496 container attach 56fba5f25525c35b62f5c686faf5f2af332840a4cddbed13330308d4ffd04020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-2-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux ) 2024-02-26T05:00:53.759 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 26 05:00:53 smithi191 podman[70184]: 2024-02-26 05:00:53.486119013 +0000 UTC m=+0.724956527 container died 56fba5f25525c35b62f5c686faf5f2af332840a4cddbed13330308d4ffd04020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-2-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240214) 2024-02-26T05:00:54.177 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.2.service' 2024-02-26T05:00:54.219 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:00:54.219 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-26T05:00:54.219 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-26T05:00:54.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.3 2024-02-26T05:00:54.538 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:00:54 smithi060 systemd[1]: Stopping Ceph osd.3 for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:00:54.894 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:00:54 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3[51193]: 2024-02-26T05:00:54.535+0000 7f4fcad18700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:00:54.894 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:00:54 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3[51193]: 2024-02-26T05:00:54.535+0000 7f4fcad18700 -1 osd.3 97 *** Got signal Terminated *** 2024-02-26T05:00:54.895 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:00:54 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3[51193]: 2024-02-26T05:00:54.535+0000 7f4fcad18700 -1 osd.3 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T05:00:56.980 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:00:56 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3[51193]: 2024-02-26T05:00:56.594+0000 7f4fc3304700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:32.441325+0000 front 2024-02-26T05:00:32.441299+0000 (oldest deadline 2024-02-26T05:00:56.541519+0000) 2024-02-26T05:00:57.177 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:00:56 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:00:56.727+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:00:57.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:00:56 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:00:56.851+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:00:57.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:00:57 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:00:57.186+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:00:57.979 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:00:57 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3[51193]: 2024-02-26T05:00:57.552+0000 7f4fc3304700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:32.441325+0000 front 2024-02-26T05:00:32.441299+0000 (oldest deadline 2024-02-26T05:00:56.541519+0000) 2024-02-26T05:00:58.160 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:00:57 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:00:57.872+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:00:58.177 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:00:57 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:00:57.734+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:00:58.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:00:58 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:00:58.204+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:00:58.979 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:00:58 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3[51193]: 2024-02-26T05:00:58.541+0000 7f4fc3304700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:32.441325+0000 front 2024-02-26T05:00:32.441299+0000 (oldest deadline 2024-02-26T05:00:56.541519+0000) 2024-02-26T05:00:59.177 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:00:58 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:00:58.691+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:00:59.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:00:58 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:00:58.836+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:00:59.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:00:59 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:00:59.200+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:00:59.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:00:59 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:00:59.398+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:00.177 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:00:59 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:00:59.700+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:00.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:00:59 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:00:59.801+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:00.372 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:01:00 smithi060 podman[89119]: 2024-02-26 05:01:00.05345328 +0000 UTC m=+5.641177049 container died c647fc46e5123ca1e2a4031437336ca5468d59ccfe16ea94a957ff8379c27d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5) 2024-02-26T05:01:00.372 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:00 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:00.184+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:00.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:00 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:00.403+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:00.939 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:01:00 smithi060 podman[89119]: 2024-02-26 05:01:00.627913419 +0000 UTC m=+6.215637187 container cleanup c647fc46e5123ca1e2a4031437336ca5468d59ccfe16ea94a957ff8379c27d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T05:01:00.940 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:01:00 smithi060 bash[89119]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3 2024-02-26T05:01:01.177 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:00 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:00.674+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:01.230 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:01:00 smithi060 podman[89149]: 2024-02-26 05:01:00.938116652 +0000 UTC m=+0.881797479 container remove c647fc46e5123ca1e2a4031437336ca5468d59ccfe16ea94a957ff8379c27d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux ) 2024-02-26T05:01:01.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:01 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:01.145+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:01.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:00 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:00.763+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:01.579 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:01:01 smithi060 podman[89243]: 2024-02-26 05:01:01.200102725 +0000 UTC m=+0.016959628 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T05:01:01.579 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:01:01 smithi060 podman[89243]: 2024-02-26 05:01:01.322611772 +0000 UTC m=+0.139468698 container create 82c682b867852ac3fdf8ab4eb92f2c275b77ed12ef8d8460b4c0fb251b806238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-26T05:01:01.717 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:01 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:01.389+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:01.874 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:01:01 smithi060 podman[89243]: 2024-02-26 05:01:01.576826953 +0000 UTC m=+0.393683840 container init 82c682b867852ac3fdf8ab4eb92f2c275b77ed12ef8d8460b4c0fb251b806238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T05:01:01.875 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:01:01 smithi060 podman[89243]: 2024-02-26 05:01:01.580485971 +0000 UTC m=+0.397342862 container start 82c682b867852ac3fdf8ab4eb92f2c275b77ed12ef8d8460b4c0fb251b806238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-26T05:01:01.875 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:01:01 smithi060 podman[89243]: 2024-02-26 05:01:01.683776212 +0000 UTC m=+0.500633107 container attach 82c682b867852ac3fdf8ab4eb92f2c275b77ed12ef8d8460b4c0fb251b806238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-26T05:01:01.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:01 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:01.662+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:02.009 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:01 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:01.714+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:02.155 INFO:journalctl@ceph.osd.3.smithi060.stdout:Feb 26 05:01:01 smithi060 podman[89243]: 2024-02-26 05:01:01.873230216 +0000 UTC m=+0.690087100 container died 82c682b867852ac3fdf8ab4eb92f2c275b77ed12ef8d8460b4c0fb251b806238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True) 2024-02-26T05:01:02.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:02 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:02.152+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:02.644 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.3.service' 2024-02-26T05:01:02.685 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:02 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:02.406+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:02.689 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:01:02.689 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-26T05:01:02.689 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-26T05:01:02.689 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.4 2024-02-26T05:01:02.927 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:02 smithi156 systemd[1]: Stopping Ceph osd.4 for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:01:02.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:02 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:02.646+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:03.009 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:02 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:02.683+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:03.427 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:03 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4[46970]: 2024-02-26T05:01:03.079+0000 7f8d80d6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:01:03.427 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:03 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4[46970]: 2024-02-26T05:01:03.079+0000 7f8d80d6d700 -1 osd.4 97 *** Got signal Terminated *** 2024-02-26T05:01:03.428 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:03 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4[46970]: 2024-02-26T05:01:03.079+0000 7f8d80d6d700 -1 osd.4 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T05:01:03.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:03 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:03.143+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:03.759 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:03 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:03.647+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:03.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:03 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:03.403+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:03.927 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:03 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:03.652+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:04.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:04 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:04.149+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:04.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:04 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:04.149+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:04.696 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:04 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:04.404+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:04.925 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:04 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:04.535+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:04.925 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:04 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:04.670+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:05.009 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:04 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:04.694+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:05.178 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:04 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4[46970]: 2024-02-26T05:01:04.922+0000 7f8d7d361700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.086502+0000 front 2024-02-26T05:00:41.086532+0000 (oldest deadline 2024-02-26T05:01:03.986689+0000) 2024-02-26T05:01:05.184 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:04 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:04.747+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:05.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:05 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:05.182+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:05.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:05 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:05.182+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:05.664 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:05 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:05.404+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:05.903 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:05 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:05.489+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:05.903 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:05 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:05.673+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:06.009 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:05 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:05.662+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:06.009 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:05 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:05.662+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:39.697327+0000 front 2024-02-26T05:00:39.697394+0000 (oldest deadline 2024-02-26T05:01:05.597316+0000) 2024-02-26T05:01:06.177 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:05 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4[46970]: 2024-02-26T05:01:05.900+0000 7f8d7d361700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.086502+0000 front 2024-02-26T05:00:41.086532+0000 (oldest deadline 2024-02-26T05:01:03.986689+0000) 2024-02-26T05:01:06.220 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:05 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:05.767+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:06.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:06 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:06.217+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:06.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:06 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:06.217+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:06.759 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:06 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:06.698+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:06.759 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:06 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:06.698+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:39.697327+0000 front 2024-02-26T05:00:39.697394+0000 (oldest deadline 2024-02-26T05:01:05.597316+0000) 2024-02-26T05:01:06.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:06 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:06.469+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:06.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:06 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:06.450+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:06.916 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:06 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:06.654+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:06.916 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:06 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:06.463+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:07.177 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:06 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4[46970]: 2024-02-26T05:01:06.913+0000 7f8d7d361700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.086502+0000 front 2024-02-26T05:00:41.086532+0000 (oldest deadline 2024-02-26T05:01:03.986689+0000) 2024-02-26T05:01:07.181 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:06 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:06.776+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:07.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:07 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:07.180+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:07.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:07 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:07.180+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:07.677 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:07 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:07.417+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:07.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:07 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:07.631+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:07.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:07 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:07.631+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:07.687 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:07 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:07.402+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:07.687 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:07 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:07.479+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:08.009 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:07 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:07.684+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:08.009 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:07 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:07.684+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:39.697327+0000 front 2024-02-26T05:00:39.697394+0000 (oldest deadline 2024-02-26T05:01:05.597316+0000) 2024-02-26T05:01:08.132 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:07 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:07.792+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:08.177 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:07 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4[46970]: 2024-02-26T05:01:07.889+0000 7f8d7d361700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.086502+0000 front 2024-02-26T05:00:41.086532+0000 (oldest deadline 2024-02-26T05:01:03.986689+0000) 2024-02-26T05:01:08.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:08 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:08.130+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:08.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:08 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:08.130+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:08.708 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:08 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:08.381+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:08.708 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:08 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:08.477+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:08.849 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:08 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:08.457+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:08.850 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:08 smithi156 podman[69248]: 2024-02-26 05:01:08.575714259 +0000 UTC m=+5.695780435 container died 7638ce1b098b72625e0acb7668187c44f3f0105544d35e104fdd03b4148769b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T05:01:08.850 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:08 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:08.642+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:08.850 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:08 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:08.643+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:09.009 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:08 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:08.705+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:09.009 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:08 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:08.705+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:39.697327+0000 front 2024-02-26T05:00:39.697394+0000 (oldest deadline 2024-02-26T05:01:05.597316+0000) 2024-02-26T05:01:09.102 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:08 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:08.775+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:09.177 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:08 smithi156 podman[69248]: 2024-02-26 05:01:08.94691719 +0000 UTC m=+6.066983457 container cleanup 7638ce1b098b72625e0acb7668187c44f3f0105544d35e104fdd03b4148769b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T05:01:09.177 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:08 smithi156 bash[69248]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4 2024-02-26T05:01:09.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:09 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:09.100+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:09.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:09 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:09.100+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:09.559 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:09 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:09.409+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:09.560 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:09 smithi156 podman[69265]: 2024-02-26 05:01:09.207371833 +0000 UTC m=+0.628803607 container remove 7638ce1b098b72625e0acb7668187c44f3f0105544d35e104fdd03b4148769b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_BRANCH=HEAD) 2024-02-26T05:01:09.719 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:09 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:09.446+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:09.720 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:09 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:09.420+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:09.830 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:09 smithi156 podman[69357]: 2024-02-26 05:01:09.558243088 +0000 UTC m=+0.111184062 container create ce1029d76f68077aa90d672079a184218a34e687c464689024467fd73b7b26be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T05:01:09.830 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:09 smithi156 podman[69357]: 2024-02-26 05:01:09.464811508 +0000 UTC m=+0.017752453 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T05:01:09.830 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:09 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:09.651+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:09.830 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:09 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:09.651+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:10.009 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:09 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:09.716+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:10.009 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:09 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:09.716+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:39.697327+0000 front 2024-02-26T05:00:39.697394+0000 (oldest deadline 2024-02-26T05:01:05.597316+0000) 2024-02-26T05:01:10.111 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:09 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:09.785+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:10.126 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:09 smithi156 podman[69357]: 2024-02-26 05:01:09.827788167 +0000 UTC m=+0.380729112 container init ce1029d76f68077aa90d672079a184218a34e687c464689024467fd73b7b26be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-26T05:01:10.126 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:09 smithi156 podman[69357]: 2024-02-26 05:01:09.831532473 +0000 UTC m=+0.384473405 container start ce1029d76f68077aa90d672079a184218a34e687c464689024467fd73b7b26be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-26T05:01:10.126 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:09 smithi156 podman[69357]: 2024-02-26 05:01:09.935961558 +0000 UTC m=+0.488902499 container attach ce1029d76f68077aa90d672079a184218a34e687c464689024467fd73b7b26be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-26T05:01:10.415 INFO:journalctl@ceph.osd.4.smithi156.stdout:Feb 26 05:01:10 smithi156 podman[69357]: 2024-02-26 05:01:10.124316688 +0000 UTC m=+0.677257619 container died ce1029d76f68077aa90d672079a184218a34e687c464689024467fd73b7b26be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-26T05:01:10.481 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:10 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:10.109+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:10.481 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:10 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:10.109+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:10.677 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:10 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:10.430+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:10.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:10 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:10.614+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:10.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:10 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:10.614+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:10.724 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:10 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:10.429+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:10.724 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:10 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:10.450+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:10.731 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.4.service' 2024-02-26T05:01:10.771 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:01:10.771 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-26T05:01:10.772 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-26T05:01:10.772 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.5 2024-02-26T05:01:11.009 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:10 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:10.721+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:11.009 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:10 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:10.721+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:39.697327+0000 front 2024-02-26T05:00:39.697394+0000 (oldest deadline 2024-02-26T05:01:05.597316+0000) 2024-02-26T05:01:11.009 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:10 smithi191 systemd[1]: Stopping Ceph osd.5 for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:01:11.158 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:10 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:10.765+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:11.417 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:11 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:11.093+0000 7fd9fc258700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:01:11.417 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:11 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:11.093+0000 7fd9fc258700 -1 osd.5 97 *** Got signal Terminated *** 2024-02-26T05:01:11.417 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:11 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:11.093+0000 7fd9fc258700 -1 osd.5 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T05:01:11.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:11 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:11.156+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:11.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:11 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:11.156+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:11.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:11 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:11.414+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:11.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:11 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:11.471+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:11.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:11 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:11.456+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:11.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:11 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:11.589+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:11.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:11 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:11.589+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:12.125 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:11 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:11.757+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:12.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:11 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:11.768+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:12.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:11 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:11.768+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:39.697327+0000 front 2024-02-26T05:00:39.697394+0000 (oldest deadline 2024-02-26T05:01:05.597316+0000) 2024-02-26T05:01:12.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:12 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:12.123+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:12.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:12 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:12.123+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:12.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:12 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:12.436+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:12.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:12 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:12.451+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:12.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:12 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:12.451+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:12.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:12 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:12.617+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:12.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:12 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:12.617+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:12.979 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:12 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:12.742+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:12.980 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:12 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:12.742+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.788237+0000 front 2024-02-26T05:00:50.788372+0000 (oldest deadline 2024-02-26T05:01:11.888109+0000) 2024-02-26T05:01:13.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:12 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:12.793+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:13.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:12 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:12.793+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:39.697327+0000 front 2024-02-26T05:00:39.697394+0000 (oldest deadline 2024-02-26T05:01:05.597316+0000) 2024-02-26T05:01:13.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:13 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:13.150+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:13.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:13 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:13.150+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:13.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:13 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:13.400+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:13.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:13 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:13.457+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:13.829 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:13 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:13.447+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:13.829 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:13 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:13.614+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:13.829 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:13 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:13.614+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:14.105 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:13 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:13.771+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:14.105 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:13 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:13.771+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.788237+0000 front 2024-02-26T05:00:50.788372+0000 (oldest deadline 2024-02-26T05:01:11.888109+0000) 2024-02-26T05:01:14.105 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:14 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:14.102+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:14.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:13 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:13.836+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:14.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:13 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:13.836+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:39.697327+0000 front 2024-02-26T05:00:39.697394+0000 (oldest deadline 2024-02-26T05:01:05.597316+0000) 2024-02-26T05:01:14.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:13 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:13.836+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.699771+0000 front 2024-02-26T05:00:50.699798+0000 (oldest deadline 2024-02-26T05:01:12.999963+0000) 2024-02-26T05:01:14.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:14 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:14.102+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:14.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:14 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:14.419+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:14.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:14 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:14.419+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:14.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:14 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:14.412+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:14.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:14 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:14.412+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:14.842 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:14 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:14.467+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:14.842 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:14 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:14.575+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:14.842 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:14 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:14.575+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:14.842 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:14 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:14.575+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:15.099 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:14 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:14.808+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:15.099 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:14 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:14.808+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.788237+0000 front 2024-02-26T05:00:50.788372+0000 (oldest deadline 2024-02-26T05:01:11.888109+0000) 2024-02-26T05:01:15.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:14 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:14.825+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:15.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:14 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:14.825+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:39.697327+0000 front 2024-02-26T05:00:39.697394+0000 (oldest deadline 2024-02-26T05:01:05.597316+0000) 2024-02-26T05:01:15.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:14 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:14.825+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.699771+0000 front 2024-02-26T05:00:50.699798+0000 (oldest deadline 2024-02-26T05:01:12.999963+0000) 2024-02-26T05:01:15.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:15 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:15.096+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:15.499 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:15 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:15.096+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:15.677 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:15 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:15.422+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:15.677 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:15 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:15.422+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:15.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:15 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:15.543+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:15.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:15 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:15.543+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:15.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:15 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:15.543+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:15.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:15 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:15.387+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:15.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:15 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:15.387+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:15.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:15 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:15.443+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:15.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:15 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:15.443+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:16.056 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:15 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:15.763+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:16.056 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:15 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:15.763+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.788237+0000 front 2024-02-26T05:00:50.788372+0000 (oldest deadline 2024-02-26T05:01:11.888109+0000) 2024-02-26T05:01:16.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:15 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:15.834+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.896283+0000 front 2024-02-26T05:00:33.896395+0000 (oldest deadline 2024-02-26T05:00:56.796386+0000) 2024-02-26T05:01:16.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:15 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:15.835+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:39.697327+0000 front 2024-02-26T05:00:39.697394+0000 (oldest deadline 2024-02-26T05:01:05.597316+0000) 2024-02-26T05:01:16.259 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:15 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5[48049]: 2024-02-26T05:01:15.835+0000 7fd9f884c700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.699771+0000 front 2024-02-26T05:00:50.699798+0000 (oldest deadline 2024-02-26T05:01:12.999963+0000) 2024-02-26T05:01:16.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:16 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:16.053+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:16.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:16 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:16.053+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:16.759 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:16 smithi191 podman[70315]: 2024-02-26 05:01:16.559422026 +0000 UTC m=+5.589482991 container died a7b3fb8367d8ae24c91bd454331bf90b6f7d667456655fdefc348564257eb5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-26T05:01:16.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:16 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:16.353+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:16.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:16 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:16.353+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:16.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:16 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:16.413+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:16.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:16 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:16.413+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:16.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:16 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:16.461+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:16.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:16 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:16.461+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:16.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:16 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:16.513+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:16.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:16 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:16.513+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:16.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:16 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:16.513+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:17.037 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:16 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:16.767+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:17.037 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:16 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:16.767+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.788237+0000 front 2024-02-26T05:00:50.788372+0000 (oldest deadline 2024-02-26T05:01:11.888109+0000) 2024-02-26T05:01:17.214 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:16 smithi191 podman[70315]: 2024-02-26 05:01:16.977323404 +0000 UTC m=+6.007384376 container cleanup a7b3fb8367d8ae24c91bd454331bf90b6f7d667456655fdefc348564257eb5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-02-26T05:01:17.214 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:16 smithi191 bash[70315]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5 2024-02-26T05:01:17.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:17 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:17.035+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:17.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:17 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:17.035+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:17.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:17 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:17.035+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:17.509 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:17 smithi191 podman[70334]: 2024-02-26 05:01:17.212131228 +0000 UTC m=+0.650995413 container remove a7b3fb8367d8ae24c91bd454331bf90b6f7d667456655fdefc348564257eb5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS) 2024-02-26T05:01:17.509 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:17.375+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:17.509 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:17.375+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:17.509 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:17.374+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:17.509 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:17 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:17.374+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:17.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:17 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:17.488+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:17.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:17 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:17.488+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:17.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:17 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:17.527+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:17.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:17 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:17.527+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:17.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:17 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:17.527+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:17.942 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:17 smithi191 podman[70427]: 2024-02-26 05:01:17.552117643 +0000 UTC m=+0.017915001 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T05:01:17.942 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:17 smithi191 podman[70427]: 2024-02-26 05:01:17.687989846 +0000 UTC m=+0.153787205 container create 57b5caedf2c14d6e10c42525da1c474368ce863a7278b862c6bdd83245c7f59d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T05:01:18.229 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:17 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:17.761+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:18.230 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:17 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:17.761+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.788237+0000 front 2024-02-26T05:00:50.788372+0000 (oldest deadline 2024-02-26T05:01:11.888109+0000) 2024-02-26T05:01:18.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:18 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:18.010+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:18.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:18 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:18.010+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:18.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:18 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:18.010+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:18.239 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:17 smithi191 podman[70427]: 2024-02-26 05:01:17.940723724 +0000 UTC m=+0.406521075 container init 57b5caedf2c14d6e10c42525da1c474368ce863a7278b862c6bdd83245c7f59d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T05:01:18.240 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:17 smithi191 podman[70427]: 2024-02-26 05:01:17.944175666 +0000 UTC m=+0.409973014 container start 57b5caedf2c14d6e10c42525da1c474368ce863a7278b862c6bdd83245c7f59d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-26T05:01:18.240 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:18 smithi191 podman[70427]: 2024-02-26 05:01:18.04918797 +0000 UTC m=+0.514985327 container attach 57b5caedf2c14d6e10c42525da1c474368ce863a7278b862c6bdd83245c7f59d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T05:01:18.509 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 26 05:01:18 smithi191 podman[70427]: 2024-02-26 05:01:18.237840146 +0000 UTC m=+0.703637496 container died 57b5caedf2c14d6e10c42525da1c474368ce863a7278b862c6bdd83245c7f59d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T05:01:18.509 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:18 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:18.384+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:18.509 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:18 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:18.384+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:18.510 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:18 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:18.369+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:18.510 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:18 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:18.369+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:18.862 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.5.service' 2024-02-26T05:01:18.904 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:01:18.904 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-26T05:01:18.904 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-26T05:01:18.904 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.6 2024-02-26T05:01:18.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:18 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:18.533+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:18.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:18 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:18.533+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:18.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:18 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:18.477+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:18.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:18 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:18.477+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:18.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:18 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:18.477+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:19.053 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:18 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:18.732+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:19.053 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:18 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:18.732+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.788237+0000 front 2024-02-26T05:00:50.788372+0000 (oldest deadline 2024-02-26T05:01:11.888109+0000) 2024-02-26T05:01:19.053 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:18 smithi060 systemd[1]: Stopping Ceph osd.6 for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:01:19.480 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:19 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:19.216+0000 7f9d32259700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:01:19.480 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:19 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:19.216+0000 7f9d32259700 -1 osd.6 97 *** Got signal Terminated *** 2024-02-26T05:01:19.480 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:19 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:19.216+0000 7f9d32259700 -1 osd.6 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T05:01:19.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:19 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:19.050+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:19.481 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:19 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:19.050+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:19.481 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:19 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:19.050+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:19.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:19 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:19.345+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:19.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:19 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:19.345+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:19.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:19 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:19.393+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:19.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:19 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:19.393+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:19.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:19 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:19.489+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:19.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:19 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:19.489+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:19.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:19 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:19.452+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:19.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:19 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:19.452+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:19.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:19 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:19.452+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:20.230 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:19 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:19.776+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:20.230 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:19 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:19.776+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.788237+0000 front 2024-02-26T05:00:50.788372+0000 (oldest deadline 2024-02-26T05:01:11.888109+0000) 2024-02-26T05:01:20.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:20 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:20.024+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:20.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:20 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:20.024+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:20.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:20 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:20.024+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:20.679 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:20 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:20.465+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:20.680 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:20 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:20.465+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:20.680 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:20 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:20.420+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:20.680 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:20 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:20.420+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:20.680 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:20 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:20.420+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:20.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:20 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:20.393+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:20.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:20 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:20.393+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:20.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:20 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:20.418+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:20.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:20 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:20.418+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:20.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:20 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:20.418+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:21.068 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:20 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:20.774+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:21.068 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:20 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:20.774+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.788237+0000 front 2024-02-26T05:00:50.788372+0000 (oldest deadline 2024-02-26T05:01:11.888109+0000) 2024-02-26T05:01:21.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:21 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:21.066+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:21.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:21 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:21.066+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:21.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:21 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:21.066+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:21.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:21 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:21.066+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:21.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:21 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:21.405+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:21.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:21 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:21.405+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:21.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:21 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:21.463+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:21.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:21 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:21.463+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:21.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:21 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:21.463+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:21.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:21 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:21.459+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:21.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:21 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:21.459+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:21.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:21 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:21.459+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.461925+0000 front 2024-02-26T05:00:57.461997+0000 (oldest deadline 2024-02-26T05:01:20.961989+0000) 2024-02-26T05:01:21.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:21 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:21.441+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:21.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:21 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:21.441+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:21.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:21 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:21.441+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:22.019 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:21 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:21.757+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:22.019 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:21 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:21.757+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.788237+0000 front 2024-02-26T05:00:50.788372+0000 (oldest deadline 2024-02-26T05:01:11.888109+0000) 2024-02-26T05:01:22.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:22 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:22.017+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:22.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:22 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:22.017+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:22.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:22 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:22.017+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:22.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:22 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:22.017+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:22.677 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:22 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:22.504+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:22.677 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:22 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:22.504+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:22.678 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:22 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:22.504+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.461925+0000 front 2024-02-26T05:00:57.461997+0000 (oldest deadline 2024-02-26T05:01:20.961989+0000) 2024-02-26T05:01:22.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:22 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:22.396+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:22.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:22 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:22.396+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:22.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:22 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:22.396+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:22.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:22 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:22.420+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:22.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:22 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:22.420+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:22.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:22 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:22.493+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:22.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:22 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:22.493+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:22.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:22 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:22.493+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:23.038 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:22 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:22.745+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:23.039 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:22 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:22.745+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.788237+0000 front 2024-02-26T05:00:50.788372+0000 (oldest deadline 2024-02-26T05:01:11.888109+0000) 2024-02-26T05:01:23.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:23 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:23.037+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:23.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:23 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:23.037+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:23.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:23 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:23.037+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:23.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:23 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:23.037+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:23.678 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:23 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:23.474+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:23.678 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:23 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:23.474+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:23.678 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:23 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:23.474+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.461925+0000 front 2024-02-26T05:00:57.461997+0000 (oldest deadline 2024-02-26T05:01:20.961989+0000) 2024-02-26T05:01:23.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:23 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:23.418+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:23.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:23 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:23.418+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:23.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:23 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:23.418+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:23.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:23 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:23.418+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:23.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:23 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:23.433+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:23.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:23 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:23.433+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:23.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:23 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:23.499+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:23.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:23 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:23.499+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:23.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:23 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:23.499+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:24.039 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:23 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:23.731+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:41.085089+0000 front 2024-02-26T05:00:41.085209+0000 (oldest deadline 2024-02-26T05:01:04.585234+0000) 2024-02-26T05:01:24.040 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:23 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6[57862]: 2024-02-26T05:01:23.731+0000 7f9d2a845700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.788237+0000 front 2024-02-26T05:00:50.788372+0000 (oldest deadline 2024-02-26T05:01:11.888109+0000) 2024-02-26T05:01:24.327 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:24 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:24.038+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:24.327 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:24 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:24.038+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:24.327 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:24 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:24.038+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:24.327 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:24 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:24.038+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:24.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:24 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:24.434+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:24.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:24 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:24.434+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:24.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:24 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:24.549+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:24.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:24 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:24.549+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:24.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:24 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:24.549+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:24.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:24 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:24.471+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:24.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:24 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:24.471+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:24.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:24 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:24.471+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.461925+0000 front 2024-02-26T05:00:57.461997+0000 (oldest deadline 2024-02-26T05:01:20.961989+0000) 2024-02-26T05:01:24.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:24 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:24.451+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:24.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:24 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:24.451+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:24.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:24 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:24.451+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:24.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:24 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:24.451+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:24.944 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:24 smithi060 podman[89365]: 2024-02-26 05:01:24.671042482 +0000 UTC m=+5.574257815 container died c0e8320004ada4db7833e72d3a1a9566acb0c02df06aaf710d33a9dac7b7e9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T05:01:25.230 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:25 smithi060 podman[89365]: 2024-02-26 05:01:25.116527267 +0000 UTC m=+6.019742590 container cleanup c0e8320004ada4db7833e72d3a1a9566acb0c02df06aaf710d33a9dac7b7e9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-26T05:01:25.230 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:25 smithi060 bash[89365]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6 2024-02-26T05:01:25.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:25 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:25.007+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:25.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:25 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:25.007+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:25.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:25 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:25.007+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:25.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:25 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:25.007+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:25.725 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:25 smithi060 podman[89394]: 2024-02-26 05:01:25.410029918 +0000 UTC m=+0.742357336 container remove c0e8320004ada4db7833e72d3a1a9566acb0c02df06aaf710d33a9dac7b7e9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T05:01:25.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:25 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:25.467+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:25.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:25 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:25.467+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:25.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:25 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:25.467+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:25.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:25 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:25.588+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:25.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:25 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:25.588+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:25.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:25 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:25.588+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:25.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:25 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:25.478+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:25.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:25 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:25.478+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:25.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:25 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:25.478+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.461925+0000 front 2024-02-26T05:00:57.461997+0000 (oldest deadline 2024-02-26T05:01:20.961989+0000) 2024-02-26T05:01:25.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:25 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:25.491+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:25.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:25 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:25.491+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:25.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:25 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:25.491+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:25.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:25 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:25.491+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:25.979 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:25 smithi060 podman[89489]: 2024-02-26 05:01:25.626487595 +0000 UTC m=+0.017896511 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T05:01:25.980 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:25 smithi060 podman[89489]: 2024-02-26 05:01:25.860960244 +0000 UTC m=+0.252369183 container create 537176c5b71f76ffd8a250efe917a7840a8955896329a626eae38998eaf426dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-02-26T05:01:25.980 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:25 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:25.970+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:25.980 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:25 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:25.970+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:25.980 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:25 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:25.970+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:25.980 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:25 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:25.970+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:26.453 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:26 smithi060 podman[89489]: 2024-02-26 05:01:26.147128568 +0000 UTC m=+0.538537485 container init 537176c5b71f76ffd8a250efe917a7840a8955896329a626eae38998eaf426dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T05:01:26.454 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:26 smithi060 podman[89489]: 2024-02-26 05:01:26.150492218 +0000 UTC m=+0.541901126 container start 537176c5b71f76ffd8a250efe917a7840a8955896329a626eae38998eaf426dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-26T05:01:26.454 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:26 smithi060 podman[89489]: 2024-02-26 05:01:26.247004854 +0000 UTC m=+0.638413766 container attach 537176c5b71f76ffd8a250efe917a7840a8955896329a626eae38998eaf426dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-26T05:01:26.730 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:26 smithi060 podman[89489]: 2024-02-26 05:01:26.452254723 +0000 UTC m=+0.843663626 container died 537176c5b71f76ffd8a250efe917a7840a8955896329a626eae38998eaf426dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-26T05:01:26.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:26 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:26.426+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:26.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:26 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:26.426+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:26.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:26 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:26.426+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:26.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:26 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:26.564+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:26.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:26 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:26.564+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:26.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:26 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:26.564+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:26.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:26 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:26.510+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:26.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:26 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:26.510+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:26.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:26 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:26.510+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.461925+0000 front 2024-02-26T05:00:57.461997+0000 (oldest deadline 2024-02-26T05:01:20.961989+0000) 2024-02-26T05:01:26.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:26 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:26.489+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:26.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:26 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:26.489+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:26.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:26 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:26.489+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:26.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:26 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:26.489+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:27.116 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.6.service' 2024-02-26T05:01:27.144 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:27 smithi060 podman[89515]: 2024-02-26 05:01:27.092386491 +0000 UTC m=+0.635126964 container remove 537176c5b71f76ffd8a250efe917a7840a8955896329a626eae38998eaf426dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T05:01:27.144 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:27 smithi060 systemd[1]: ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.6.service: Deactivated successfully. 2024-02-26T05:01:27.144 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:27 smithi060 systemd[1]: Stopped Ceph osd.6 for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T05:01:27.144 INFO:journalctl@ceph.osd.6.smithi060.stdout:Feb 26 05:01:27 smithi060 systemd[1]: ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.6.service: Consumed 12.281s CPU time. 2024-02-26T05:01:27.145 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:26 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:26.950+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:27.145 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:26 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:26.950+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:27.145 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:26 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:26.950+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:27.145 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:26 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:26.950+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:27.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:01:27.162 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-26T05:01:27.162 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-26T05:01:27.162 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.7 2024-02-26T05:01:27.477 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:27 smithi156 systemd[1]: Stopping Ceph osd.7 for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:01:27.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:27 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:27.448+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:27.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:27 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:27.448+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:27.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:27 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:27.448+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:27.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:27 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:27.571+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:27.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:27 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:27.571+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:27.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:27 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:27.571+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:27.927 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:27 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:27.474+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:27.927 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:27 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:27.474+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:27.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:27 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:27.474+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:27.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:27 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:27.474+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:27.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:27 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:27.483+0000 7f4876dfe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:01:27.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:27 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:27.483+0000 7f4876dfe700 -1 osd.7 97 *** Got signal Terminated *** 2024-02-26T05:01:27.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:27 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:27.483+0000 7f4876dfe700 -1 osd.7 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T05:01:27.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:27 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:27.551+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:27.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:27 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:27.551+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:27.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:27 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:27.551+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.461925+0000 front 2024-02-26T05:00:57.461997+0000 (oldest deadline 2024-02-26T05:01:20.961989+0000) 2024-02-26T05:01:28.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:27 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:27.982+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:28.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:27 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:27.982+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:28.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:27 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:27.982+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:28.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:27 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:27.982+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:28.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:28 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:28.466+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:28.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:28 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:28.466+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:28.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:28 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:28.466+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:28.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:28 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:28.565+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:28.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:28 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:28.565+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:28.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:28 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:28.565+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:28.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:28 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:28.519+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:28.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:28 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:28.519+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:28.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:28 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:28.519+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.461925+0000 front 2024-02-26T05:00:57.461997+0000 (oldest deadline 2024-02-26T05:01:20.961989+0000) 2024-02-26T05:01:28.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:28 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:28.465+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:28.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:28 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:28.465+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:28.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:28 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:28.465+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:28.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:28 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:28.465+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:29.229 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:28 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:28.962+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:29.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:28 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:28.962+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:29.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:28 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:28.962+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:29.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:28 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:28.962+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:29.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:29 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:29.462+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:29.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:29 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:29.462+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:29.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:29 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:29.462+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:29.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:29 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:29.462+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.572052+0000 front 2024-02-26T05:01:04.572030+0000 (oldest deadline 2024-02-26T05:01:28.671934+0000) 2024-02-26T05:01:29.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:29 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:29.612+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:29.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:29 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:29.612+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:29.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:29 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:29.612+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:29.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:29 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:29.612+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:29.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:29 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:29.554+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:29.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:29 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:29.554+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:29.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:29 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:29.554+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.461925+0000 front 2024-02-26T05:00:57.461997+0000 (oldest deadline 2024-02-26T05:01:20.961989+0000) 2024-02-26T05:01:29.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:29 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:29.480+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:29.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:29 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:29.480+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:29.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:29 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:29.480+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:29.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:29 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:29.480+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:29.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:29 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:29.480+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:30.479 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:29 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:29.998+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:30.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:29 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:29.998+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:30.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:29 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:29.998+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:30.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:29 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:29.998+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:30.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:30.447+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:30.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:30.447+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:30.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:30.447+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:30.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:30.447+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.572052+0000 front 2024-02-26T05:01:04.572030+0000 (oldest deadline 2024-02-26T05:01:28.671934+0000) 2024-02-26T05:01:30.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:30.606+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:30.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:30.606+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:30.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:30.606+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:30.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:30 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:30.606+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:30.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:30 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:30.508+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:30.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:30 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:30.508+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:30.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:30 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:30.508+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.461925+0000 front 2024-02-26T05:00:57.461997+0000 (oldest deadline 2024-02-26T05:01:20.961989+0000) 2024-02-26T05:01:30.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:30 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:30.498+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:30.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:30 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:30.499+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:30.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:30 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:30.499+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:30.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:30 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:30.499+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:30.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:30 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:30.499+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:31.229 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:30 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:30.971+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:31.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:30 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:30.971+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:31.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:30 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:30.971+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:31.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:30 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:30.971+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:31.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:30 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:30.971+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:31.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:31 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:31.625+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:31.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:31 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:31.625+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:31.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:31 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:31.625+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:31.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:31 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:31.625+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:31.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:31 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:31.431+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:31.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:31 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:31.431+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:31.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:31 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:31.431+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:31.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:31 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:31.431+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.572052+0000 front 2024-02-26T05:01:04.572030+0000 (oldest deadline 2024-02-26T05:01:28.671934+0000) 2024-02-26T05:01:31.927 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:31 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:31.527+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.458670+0000 front 2024-02-26T05:00:42.458715+0000 (oldest deadline 2024-02-26T05:01:04.158851+0000) 2024-02-26T05:01:31.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:31 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:31.527+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:46.459687+0000 front 2024-02-26T05:00:46.459770+0000 (oldest deadline 2024-02-26T05:01:14.660239+0000) 2024-02-26T05:01:31.928 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:31 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7[53539]: 2024-02-26T05:01:31.527+0000 7f486f3ea700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.461925+0000 front 2024-02-26T05:00:57.461997+0000 (oldest deadline 2024-02-26T05:01:20.961989+0000) 2024-02-26T05:01:31.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:31 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:31.471+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:31.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:31 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:31.471+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:31.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:31 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:31.471+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:31.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:31 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:31.471+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:31.928 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:31 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:31.471+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:32.229 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:31 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:31.923+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:32.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:31 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:31.923+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:32.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:31 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:31.923+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:32.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:31 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:31.923+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:32.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:31 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:31.923+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:32.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:32 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:32.603+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:32.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:32 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:32.603+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:32.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:32 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:32.603+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:32.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:32 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:32.603+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:32.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:32 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:32.458+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:32.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:32 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:32.458+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:32.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:32 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:32.458+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:32.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:32 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:32.459+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.572052+0000 front 2024-02-26T05:01:04.572030+0000 (oldest deadline 2024-02-26T05:01:28.671934+0000) 2024-02-26T05:01:32.841 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:32 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:32.496+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:32.841 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:32 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:32.496+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:32.841 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:32 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:32.496+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:32.841 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:32 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:32.496+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:32.841 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:32 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:32.496+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:33.177 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:32 smithi156 podman[69474]: 2024-02-26 05:01:32.839365319 +0000 UTC m=+5.488662453 container died 2b20654fe2ea7db408dbf1d844eb2c741b2e0a249ab3182dcfbcde70a9cdbfd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-26T05:01:33.229 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:32 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:32.899+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:33.229 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:32 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:32.899+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:33.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:32 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:32.899+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:33.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:32 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:32.899+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:33.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:32 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:32.899+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:33.459 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:33 smithi156 podman[69474]: 2024-02-26 05:01:33.283203451 +0000 UTC m=+5.932500627 container cleanup 2b20654fe2ea7db408dbf1d844eb2c741b2e0a249ab3182dcfbcde70a9cdbfd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-26T05:01:33.459 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:33 smithi156 bash[69474]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7 2024-02-26T05:01:33.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:33 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:33.492+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:33.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:33 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:33.492+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:33.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:33 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:33.492+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:33.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:33 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:33.492+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.572052+0000 front 2024-02-26T05:01:04.572030+0000 (oldest deadline 2024-02-26T05:01:28.671934+0000) 2024-02-26T05:01:33.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:33 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:33.574+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:33.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:33 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:33.574+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:33.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:33 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:33.574+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:33.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:33 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:33.574+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:33.894 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:33 smithi156 podman[69491]: 2024-02-26 05:01:33.492421038 +0000 UTC m=+0.649296633 container remove 2b20654fe2ea7db408dbf1d844eb2c741b2e0a249ab3182dcfbcde70a9cdbfd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-26T05:01:33.894 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:33 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:33.457+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:33.895 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:33 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:33.457+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:33.895 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:33 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:33.457+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:33.895 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:33 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:33.457+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:33.895 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:33 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:33.457+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:34.156 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:33 smithi156 podman[69582]: 2024-02-26 05:01:33.795608452 +0000 UTC m=+0.018071599 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T05:01:34.156 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:33 smithi156 podman[69582]: 2024-02-26 05:01:33.893250628 +0000 UTC m=+0.115713783 container create 82f27db84622d62a8cd4505230678015d35a098a2bae77f418a390aefc3cd4c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-02-26T05:01:34.156 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:34 smithi156 podman[69582]: 2024-02-26 05:01:34.154719898 +0000 UTC m=+0.377183048 container init 82f27db84622d62a8cd4505230678015d35a098a2bae77f418a390aefc3cd4c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-26T05:01:34.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:33 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:33.891+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:34.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:33 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:33.891+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:34.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:33 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:33.891+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:34.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:33 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:33.891+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:34.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:33 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:33.891+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:34.428 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:34 smithi156 podman[69582]: 2024-02-26 05:01:34.158108522 +0000 UTC m=+0.380571665 container start 82f27db84622d62a8cd4505230678015d35a098a2bae77f418a390aefc3cd4c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-26T05:01:34.428 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:34 smithi156 podman[69582]: 2024-02-26 05:01:34.220960467 +0000 UTC m=+0.443423619 container attach 82f27db84622d62a8cd4505230678015d35a098a2bae77f418a390aefc3cd4c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T05:01:34.725 INFO:journalctl@ceph.osd.7.smithi156.stdout:Feb 26 05:01:34 smithi156 podman[69582]: 2024-02-26 05:01:34.449358855 +0000 UTC m=+0.671821995 container died 82f27db84622d62a8cd4505230678015d35a098a2bae77f418a390aefc3cd4c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T05:01:34.726 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:34 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:34.430+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:34.726 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:34 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:34.430+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:34.726 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:34 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:34.430+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:34.726 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:34 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:34.430+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:34.726 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:34 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:34.430+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:34.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:34 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:34.489+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:34.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:34 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:34.489+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:34.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:34 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:34.489+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:34.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:34 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:34.489+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.572052+0000 front 2024-02-26T05:01:04.572030+0000 (oldest deadline 2024-02-26T05:01:28.671934+0000) 2024-02-26T05:01:34.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:34 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:34.547+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:34.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:34 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:34.547+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:34.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:34 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:34.547+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:34.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:34 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:34.547+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:35.050 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.7.service' 2024-02-26T05:01:35.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:01:35.095 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-26T05:01:35.095 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-02-26T05:01:35.095 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.8 2024-02-26T05:01:35.229 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:34 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:34.862+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:35.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:34 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:34.862+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:35.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:34 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:34.862+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:35.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:34 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:34.862+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:35.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:34 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:34.862+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:35.415 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:35 smithi191 systemd[1]: Stopping Ceph osd.8 for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:01:35.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:35 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:35.387+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:35.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:35 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:35.387+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:35.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:35 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:35.387+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:35.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:35 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:35.387+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:35.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:35 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:35.387+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:35.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:35.584+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:35.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:35.584+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:35.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:35.584+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:35.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:35.584+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:35.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:35.413+0000 7f6a894d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:01:35.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:35.413+0000 7f6a894d5700 -1 osd.8 97 *** Got signal Terminated *** 2024-02-26T05:01:35.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:35.413+0000 7f6a894d5700 -1 osd.8 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T05:01:35.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:35.475+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:35.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:35.475+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:35.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:35.475+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:35.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:35 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:35.475+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.572052+0000 front 2024-02-26T05:01:04.572030+0000 (oldest deadline 2024-02-26T05:01:28.671934+0000) 2024-02-26T05:01:36.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:35 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:35.903+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:36.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:35 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:35.903+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:36.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:35 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:35.903+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:36.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:35 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:35.903+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:36.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:35 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:35.903+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:36.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:36 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:36.356+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:36.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:36 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:36.356+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:36.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:36 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:36.356+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:36.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:36 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:36.356+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:36.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:36 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:36.356+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:36.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:36 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:36.488+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:36.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:36 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:36.488+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:36.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:36 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:36.488+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:36.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:36 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:36.488+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.572052+0000 front 2024-02-26T05:01:04.572030+0000 (oldest deadline 2024-02-26T05:01:28.671934+0000) 2024-02-26T05:01:36.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:36 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:36.633+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:36.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:36 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:36.633+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:36.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:36 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:36.633+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:36.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:36 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:36.633+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:37.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:36 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:36.922+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:37.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:36 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:36.922+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:37.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:36 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:36.922+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:37.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:36 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:36.922+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:37.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:36 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:36.922+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:37.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:37 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:37.393+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:37.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:37 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:37.393+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:37.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:37 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:37.393+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:37.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:37 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:37.393+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:37.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:37 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:37.393+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:38.009 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:37 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:37.507+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:38.009 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:37 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:37.507+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:38.009 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:37 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:37.507+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:38.009 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:37 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:37.507+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.572052+0000 front 2024-02-26T05:01:04.572030+0000 (oldest deadline 2024-02-26T05:01:28.671934+0000) 2024-02-26T05:01:38.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:37 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:37.642+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:38.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:37 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:37.642+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:38.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:37 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:37.642+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:38.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:37 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:37.642+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:38.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:37 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:37.938+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:38.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:37 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:37.938+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:38.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:37 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:37.938+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:38.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:37 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:37.938+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:38.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:37 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:37.938+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:38.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:38 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:38.390+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:38.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:38 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:38.390+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:38.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:38 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:38.390+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:38.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:38 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:38.390+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:38.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:38 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:38.390+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:38.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:38 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:38.631+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:38.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:38 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:38.631+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:38.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:38 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:38.631+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:38.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:38 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:38.631+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:38.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:38 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:38.467+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:38.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:38 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:38.467+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:38.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:38 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:38.467+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:38.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:38 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:38.467+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.572052+0000 front 2024-02-26T05:01:04.572030+0000 (oldest deadline 2024-02-26T05:01:28.671934+0000) 2024-02-26T05:01:38.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:38 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:38.467+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:15.073698+0000 front 2024-02-26T05:01:15.073718+0000 (oldest deadline 2024-02-26T05:01:37.973777+0000) 2024-02-26T05:01:39.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:38 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:38.936+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:39.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:38 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:38.936+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:39.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:38 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:38.936+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:39.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:38 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:38.936+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:39.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:38 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:38.936+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:39.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:38 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:38.936+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.017348+0000 front 2024-02-26T05:01:13.017257+0000 (oldest deadline 2024-02-26T05:01:38.917156+0000) 2024-02-26T05:01:39.680 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:39 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:39.394+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:39.680 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:39 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:39.394+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:39.681 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:39 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:39.394+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:39.681 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:39 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:39.394+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:39.681 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:39 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:39.394+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:39.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:39 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:39.584+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:39.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:39 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:39.584+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:39.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:39 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:39.584+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:39.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:39 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:39.584+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:39.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:39 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:39.584+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:39.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:39 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:39.476+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:42.468181+0000 front 2024-02-26T05:00:42.468192+0000 (oldest deadline 2024-02-26T05:01:05.968183+0000) 2024-02-26T05:01:39.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:39 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:39.476+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:48.869216+0000 front 2024-02-26T05:00:48.869326+0000 (oldest deadline 2024-02-26T05:01:14.169345+0000) 2024-02-26T05:01:39.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:39 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:39.476+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:59.271447+0000 front 2024-02-26T05:00:59.271321+0000 (oldest deadline 2024-02-26T05:01:24.571355+0000) 2024-02-26T05:01:39.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:39 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:39.476+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.572052+0000 front 2024-02-26T05:01:04.572030+0000 (oldest deadline 2024-02-26T05:01:28.671934+0000) 2024-02-26T05:01:39.760 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:39 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8[54904]: 2024-02-26T05:01:39.476+0000 7f6a81ac1700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:15.073698+0000 front 2024-02-26T05:01:15.073718+0000 (oldest deadline 2024-02-26T05:01:37.973777+0000) 2024-02-26T05:01:40.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:39 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:39.904+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:40.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:39 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:39.904+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:40.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:39 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:39.904+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:40.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:39 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:39.904+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:40.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:39 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:39.904+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:40.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:39 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:39.904+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.017348+0000 front 2024-02-26T05:01:13.017257+0000 (oldest deadline 2024-02-26T05:01:38.917156+0000) 2024-02-26T05:01:40.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:40 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:40.384+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:40.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:40 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:40.384+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:40.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:40 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:40.384+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:40.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:40 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:40.384+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:40.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:40 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:40.384+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:40.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:40 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:40.384+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:40.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:40 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:40.592+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:40.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:40 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:40.592+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:40.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:40 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:40.592+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:40.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:40 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:40.592+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:40.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:40 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:40.592+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:41.167 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:40 smithi191 podman[70543]: 2024-02-26 05:01:40.787279877 +0000 UTC m=+5.500311618 container died 0977986a3ab1ba48ed4fb71569c01b41d9aa44c2fa382424b1c847c7ed897e5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-26T05:01:41.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:40 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:40.881+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:41.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:40 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:40.881+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:41.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:40 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:40.881+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:41.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:40 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:40.881+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:41.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:40 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:40.881+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:41.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:40 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:40.881+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.017348+0000 front 2024-02-26T05:01:13.017257+0000 (oldest deadline 2024-02-26T05:01:38.917156+0000) 2024-02-26T05:01:41.509 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:41 smithi191 podman[70543]: 2024-02-26 05:01:41.264685541 +0000 UTC m=+5.977717282 container cleanup 0977986a3ab1ba48ed4fb71569c01b41d9aa44c2fa382424b1c847c7ed897e5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T05:01:41.509 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:41 smithi191 bash[70543]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8 2024-02-26T05:01:41.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:41 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:41.393+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:41.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:41 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:41.393+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:41.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:41 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:41.393+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:41.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:41 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:41.393+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:41.679 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:41 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:41.393+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:41.679 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:41 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:41.393+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:41.876 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:41 smithi191 podman[70560]: 2024-02-26 05:01:41.558447519 +0000 UTC m=+0.767569754 container remove 0977986a3ab1ba48ed4fb71569c01b41d9aa44c2fa382424b1c847c7ed897e5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T05:01:41.877 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:41 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:41.609+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:41.877 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:41 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:41.609+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:41.877 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:41 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:41.609+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:41.877 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:41 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:41.609+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:41.877 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:41 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:41.609+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:42.172 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:41 smithi191 podman[70655]: 2024-02-26 05:01:41.777237527 +0000 UTC m=+0.017586878 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T05:01:42.172 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:41 smithi191 podman[70655]: 2024-02-26 05:01:41.92789973 +0000 UTC m=+0.168249067 container create 267801d64ea8455b8dc448106e20f397a6c2ad12ef52826017a2bee6f1e412ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-26T05:01:42.173 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:42 smithi191 podman[70655]: 2024-02-26 05:01:42.170880856 +0000 UTC m=+0.411230194 container init 267801d64ea8455b8dc448106e20f397a6c2ad12ef52826017a2bee6f1e412ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214) 2024-02-26T05:01:42.229 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:41 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:41.889+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:42.229 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:41 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:41.889+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:42.229 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:41 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:41.889+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:42.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:41 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:41.889+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:42.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:41 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:41.889+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:42.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:41 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:41.889+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.017348+0000 front 2024-02-26T05:01:13.017257+0000 (oldest deadline 2024-02-26T05:01:38.917156+0000) 2024-02-26T05:01:42.465 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:42 smithi191 podman[70655]: 2024-02-26 05:01:42.174219811 +0000 UTC m=+0.414569142 container start 267801d64ea8455b8dc448106e20f397a6c2ad12ef52826017a2bee6f1e412ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-26T05:01:42.465 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:42 smithi191 podman[70655]: 2024-02-26 05:01:42.270876011 +0000 UTC m=+0.511225357 container attach 267801d64ea8455b8dc448106e20f397a6c2ad12ef52826017a2bee6f1e412ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214) 2024-02-26T05:01:42.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:42 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:42.399+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:42.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:42 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:42.399+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:42.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:42 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:42.399+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:42.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:42 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:42.399+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:42.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:42 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:42.399+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:42.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:42 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:42.399+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:42.759 INFO:journalctl@ceph.osd.8.smithi191.stdout:Feb 26 05:01:42 smithi191 podman[70655]: 2024-02-26 05:01:42.46346061 +0000 UTC m=+0.703809976 container died 267801d64ea8455b8dc448106e20f397a6c2ad12ef52826017a2bee6f1e412ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-8-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-02-26T05:01:42.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:42 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:42.604+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:42.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:42 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:42.604+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:42.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:42 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:42.604+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:42.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:42 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:42.604+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:42.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:42 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:42.604+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:43.078 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.8.service' 2024-02-26T05:01:43.123 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:01:43.123 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-02-26T05:01:43.123 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-02-26T05:01:43.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.9 2024-02-26T05:01:43.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:42 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:42.937+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:43.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:42 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:42.937+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:43.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:42 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:42.937+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:43.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:42 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:42.937+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:43.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:42 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:42.937+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:43.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:42 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:42.937+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.017348+0000 front 2024-02-26T05:01:13.017257+0000 (oldest deadline 2024-02-26T05:01:38.917156+0000) 2024-02-26T05:01:43.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:43 smithi060 systemd[1]: Stopping Ceph osd.9 for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:01:43.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:43 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:43.362+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:43.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:43 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:43.362+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:43.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:43 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:43.362+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:43.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:43 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:43.362+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:43.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:43 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:43.362+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:43.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:43 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:43.362+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:43.729 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:43 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:43.434+0000 7f50e7d0b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:01:43.729 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:43 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:43.435+0000 7f50e7d0b700 -1 osd.9 97 *** Got signal Terminated *** 2024-02-26T05:01:43.729 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:43 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:43.435+0000 7f50e7d0b700 -1 osd.9 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T05:01:43.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:43 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:43.576+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:43.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:43 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:43.576+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:43.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:43 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:43.576+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:43.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:43 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:43.576+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:43.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:43 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:43.576+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:44.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:43 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:43.893+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:44.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:43 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:43.893+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:44.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:43 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:43.893+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:44.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:43 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:43.893+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:44.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:43 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:43.893+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:44.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:43 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:43.893+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.017348+0000 front 2024-02-26T05:01:13.017257+0000 (oldest deadline 2024-02-26T05:01:38.917156+0000) 2024-02-26T05:01:44.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:44 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:44.397+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:44.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:44 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:44.397+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:44.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:44 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:44.397+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:44.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:44 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:44.397+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:44.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:44 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:44.397+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:44.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:44 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:44.397+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:45.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:44 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:44.614+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:45.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:44 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:44.614+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:45.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:44 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:44.614+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:45.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:44 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:44.614+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:45.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:44 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:44.614+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:45.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:44 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:44.867+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:45.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:44 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:44.867+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:45.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:44 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:44.867+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:45.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:44 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:44.867+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:45.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:44 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:44.867+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:45.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:44 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:44.867+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.017348+0000 front 2024-02-26T05:01:13.017257+0000 (oldest deadline 2024-02-26T05:01:38.917156+0000) 2024-02-26T05:01:45.737 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:45 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:45.427+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:45.738 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:45 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:45.427+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:45.738 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:45 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:45.427+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:45.738 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:45 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:45.427+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:45.738 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:45 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:45.427+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:45.738 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:45 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:45.427+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:46.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:45 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:45.597+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:46.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:45 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:45.597+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:46.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:45 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:45.597+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:46.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:45 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:45.597+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:46.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:45 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:45.597+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:46.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:45 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:45.597+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:46.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:45 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:45.821+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:46.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:45 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:45.821+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:46.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:45 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:45.821+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:46.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:45 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:45.821+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:46.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:45 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:45.821+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:46.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:45 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:45.821+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.017348+0000 front 2024-02-26T05:01:13.017257+0000 (oldest deadline 2024-02-26T05:01:38.917156+0000) 2024-02-26T05:01:46.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:46 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:46.421+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:46.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:46 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:46.421+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:46.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:46 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:46.421+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:46.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:46 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:46.421+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:46.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:46 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:46.421+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:46.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:46 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:46.421+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:46.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:46 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:46.421+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:20.082152+0000 front 2024-02-26T05:01:20.082117+0000 (oldest deadline 2024-02-26T05:01:45.982077+0000) 2024-02-26T05:01:47.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:46 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:46.603+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:47.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:46 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:46.603+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:47.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:46 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:46.603+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:47.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:46 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:46.603+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:47.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:46 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:46.603+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:47.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:46 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:46.603+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:47.229 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:46 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:46.780+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:47.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:46 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:46.780+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:47.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:46 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:46.780+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:47.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:46 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:46.780+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:47.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:46 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:46.780+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:47.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:46 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:46.780+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.017348+0000 front 2024-02-26T05:01:13.017257+0000 (oldest deadline 2024-02-26T05:01:38.917156+0000) 2024-02-26T05:01:47.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:47 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:47.373+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:47.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:47 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:47.373+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:47.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:47 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:47.373+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:47.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:47 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:47.373+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:47.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:47 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:47.373+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:47.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:47 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:47.373+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:47.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:47 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:47.373+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:20.082152+0000 front 2024-02-26T05:01:20.082117+0000 (oldest deadline 2024-02-26T05:01:45.982077+0000) 2024-02-26T05:01:48.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:47 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:47.613+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:48.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:47 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:47.613+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:48.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:47 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:47.613+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:48.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:47 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:47.613+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:48.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:47 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:47.613+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:48.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:47 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:47.613+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:48.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:47 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:47.742+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:34.510925+0000 front 2024-02-26T05:00:34.511005+0000 (oldest deadline 2024-02-26T05:00:56.810997+0000) 2024-02-26T05:01:48.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:47 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:47.742+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:37.912099+0000 front 2024-02-26T05:00:37.912260+0000 (oldest deadline 2024-02-26T05:01:03.812160+0000) 2024-02-26T05:01:48.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:47 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:47.742+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:51.314410+0000 front 2024-02-26T05:00:51.314600+0000 (oldest deadline 2024-02-26T05:01:16.614633+0000) 2024-02-26T05:01:48.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:47 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:47.742+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:56.615009+0000 front 2024-02-26T05:00:56.615037+0000 (oldest deadline 2024-02-26T05:01:20.715083+0000) 2024-02-26T05:01:48.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:47 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:47.742+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:06.016256+0000 front 2024-02-26T05:01:06.016073+0000 (oldest deadline 2024-02-26T05:01:30.116038+0000) 2024-02-26T05:01:48.230 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:47 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9[64904]: 2024-02-26T05:01:47.742+0000 7f50e02f7700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.017348+0000 front 2024-02-26T05:01:13.017257+0000 (oldest deadline 2024-02-26T05:01:38.917156+0000) 2024-02-26T05:01:48.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:48 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:48.359+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:48.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:48 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:48.359+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:48.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:48 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:48.359+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:48.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:48 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:48.359+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:48.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:48 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:48.359+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:48.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:48 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:48.359+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:48.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:48 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:48.359+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:20.082152+0000 front 2024-02-26T05:01:20.082117+0000 (oldest deadline 2024-02-26T05:01:45.982077+0000) 2024-02-26T05:01:49.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:48 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:48.643+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:49.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:48 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:48.643+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:49.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:48 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:48.643+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:49.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:48 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:48.643+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:49.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:48 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:48.643+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:49.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:48 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:48.643+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:49.039 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:48 smithi060 podman[89595]: 2024-02-26 05:01:48.757314529 +0000 UTC m=+5.440519093 container died 9678427e6b8c6e2c44d24ae4ae93521e55ad73b234de8c6f30261a772c3ad2ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T05:01:49.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:49 smithi060 podman[89595]: 2024-02-26 05:01:49.228195665 +0000 UTC m=+5.911400226 container cleanup 9678427e6b8c6e2c44d24ae4ae93521e55ad73b234de8c6f30261a772c3ad2ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T05:01:49.480 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:49 smithi060 bash[89595]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9 2024-02-26T05:01:49.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:49 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:49.363+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:49.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:49 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:49.363+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:49.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:49 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:49.363+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:49.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:49 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:49.363+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:49.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:49 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:49.363+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:49.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:49 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:49.363+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:49.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:49 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:49.363+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:20.082152+0000 front 2024-02-26T05:01:20.082117+0000 (oldest deadline 2024-02-26T05:01:45.982077+0000) 2024-02-26T05:01:49.797 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:49 smithi060 podman[89614]: 2024-02-26 05:01:49.496252243 +0000 UTC m=+0.733437219 container remove 9678427e6b8c6e2c44d24ae4ae93521e55ad73b234de8c6f30261a772c3ad2ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-26T05:01:50.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:49 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:49.619+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:50.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:49 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:49.619+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:50.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:49 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:49.619+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:50.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:49 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:49.619+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:50.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:49 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:49.619+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:50.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:49 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:49.619+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:50.093 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:49 smithi060 podman[89707]: 2024-02-26 05:01:49.698715699 +0000 UTC m=+0.017608364 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T05:01:50.093 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:49 smithi060 podman[89707]: 2024-02-26 05:01:49.796852656 +0000 UTC m=+0.115745308 container create c19c3b569e2d8c2896aba635b2b809cba0a3a3bf6c97798cfe8f2488a6bf3e74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-26T05:01:50.370 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:50 smithi060 podman[89707]: 2024-02-26 05:01:50.091291586 +0000 UTC m=+0.410184249 container init c19c3b569e2d8c2896aba635b2b809cba0a3a3bf6c97798cfe8f2488a6bf3e74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-26T05:01:50.370 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:50 smithi060 podman[89707]: 2024-02-26 05:01:50.095006199 +0000 UTC m=+0.413898854 container start c19c3b569e2d8c2896aba635b2b809cba0a3a3bf6c97798cfe8f2488a6bf3e74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9-deactivate, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-26T05:01:50.371 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:50 smithi060 podman[89707]: 2024-02-26 05:01:50.199401128 +0000 UTC m=+0.518293790 container attach c19c3b569e2d8c2896aba635b2b809cba0a3a3bf6c97798cfe8f2488a6bf3e74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-26T05:01:50.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:50 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:50.383+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:50.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:50 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:50.383+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:50.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:50 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:50.383+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:50.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:50 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:50.383+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:50.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:50 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:50.383+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:50.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:50 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:50.383+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:50.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:50 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:50.383+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:20.082152+0000 front 2024-02-26T05:01:20.082117+0000 (oldest deadline 2024-02-26T05:01:45.982077+0000) 2024-02-26T05:01:50.730 INFO:journalctl@ceph.osd.9.smithi060.stdout:Feb 26 05:01:50 smithi060 podman[89707]: 2024-02-26 05:01:50.402124026 +0000 UTC m=+0.721016679 container died c19c3b569e2d8c2896aba635b2b809cba0a3a3bf6c97798cfe8f2488a6bf3e74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T05:01:51.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:50 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:50.606+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:51.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:50 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:50.606+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:51.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:50 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:50.606+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:51.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:50 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:50.606+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:51.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:50 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:50.606+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:51.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:50 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:50.606+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:51.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.9.service' 2024-02-26T05:01:51.100 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:01:51.101 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-02-26T05:01:51.101 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-02-26T05:01:51.101 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.10 2024-02-26T05:01:51.427 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:51 smithi156 systemd[1]: Stopping Ceph osd.10 for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:01:51.428 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:51 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:51.347+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:51.428 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:51 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:51.347+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:51.428 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:51 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:51.347+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:51.428 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:51 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:51.347+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:51.428 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:51 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:51.347+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:51.428 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:51 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:51.347+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:51.428 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:51 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:51.347+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:20.082152+0000 front 2024-02-26T05:01:20.082117+0000 (oldest deadline 2024-02-26T05:01:45.982077+0000) 2024-02-26T05:01:51.927 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:51 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:51.625+0000 7f8fe3780700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:01:51.927 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:51 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:51.626+0000 7f8fe3780700 -1 osd.10 97 *** Got signal Terminated *** 2024-02-26T05:01:51.927 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:51 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:51.626+0000 7f8fe3780700 -1 osd.10 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T05:01:52.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:51 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:51.620+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:52.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:51 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:51.620+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:52.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:51 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:51.620+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:52.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:51 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:51.620+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:52.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:51 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:51.620+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:52.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:51 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:51.620+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:52.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:52 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:52.355+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:52.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:52 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:52.355+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:52.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:52 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:52.355+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:52.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:52 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:52.355+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:52.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:52 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:52.355+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:52.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:52 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:52.355+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:52.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:52 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:52.355+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:20.082152+0000 front 2024-02-26T05:01:20.082117+0000 (oldest deadline 2024-02-26T05:01:45.982077+0000) 2024-02-26T05:01:53.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:52 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:52.579+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:53.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:52 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:52.579+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:53.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:52 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:52.579+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:53.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:52 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:52.579+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:53.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:52 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:52.579+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:53.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:52 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:52.579+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:53.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:53 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:53.369+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:53.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:53 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:53.369+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:53.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:53 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:53.369+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:53.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:53 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:53.369+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:53.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:53 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:53.369+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:53.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:53 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:53.369+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:53.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:53 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:53.369+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:20.082152+0000 front 2024-02-26T05:01:20.082117+0000 (oldest deadline 2024-02-26T05:01:45.982077+0000) 2024-02-26T05:01:54.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:53 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:53.582+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:54.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:53 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:53.582+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:54.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:53 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:53.582+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:54.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:53 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:53.582+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:54.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:53 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:53.582+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:54.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:53 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:53.582+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:54.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:53 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:53.582+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-26T05:01:28.933480+0000 front 2024-02-26T05:01:28.933362+0000 (oldest deadline 2024-02-26T05:01:53.033283+0000) 2024-02-26T05:01:54.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:54 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:54.376+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:54.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:54 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:54.376+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:54.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:54 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:54.376+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:54.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:54 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:54.376+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:54.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:54 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:54.376+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:54.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:54 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:54.376+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:54.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:54 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:54.376+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:20.082152+0000 front 2024-02-26T05:01:20.082117+0000 (oldest deadline 2024-02-26T05:01:45.982077+0000) 2024-02-26T05:01:55.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:54 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:54.616+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:55.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:54 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:54.616+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:55.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:54 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:54.616+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:55.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:54 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:54.616+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:55.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:54 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:54.616+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:55.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:54 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:54.616+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:55.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:54 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:54.616+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-26T05:01:28.933480+0000 front 2024-02-26T05:01:28.933362+0000 (oldest deadline 2024-02-26T05:01:53.033283+0000) 2024-02-26T05:01:55.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:55 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:55.394+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:55.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:55 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:55.394+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:55.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:55 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:55.394+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:55.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:55 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:55.394+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:55.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:55 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:55.394+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:55.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:55 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:55.394+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:55.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:55 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:55.394+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:20.082152+0000 front 2024-02-26T05:01:20.082117+0000 (oldest deadline 2024-02-26T05:01:45.982077+0000) 2024-02-26T05:01:56.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:55 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:55.571+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:56.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:55 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:55.571+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:56.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:55 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:55.571+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:56.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:55 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:55.571+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:56.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:55 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:55.571+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:56.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:55 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:55.571+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:56.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:55 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:55.571+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-26T05:01:28.933480+0000 front 2024-02-26T05:01:28.933362+0000 (oldest deadline 2024-02-26T05:01:53.033283+0000) 2024-02-26T05:01:56.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:56 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:56.354+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.875313+0000 front 2024-02-26T05:00:33.875381+0000 (oldest deadline 2024-02-26T05:00:56.175481+0000) 2024-02-26T05:01:56.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:56 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:56.354+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-02-26T05:00:40.876636+0000 front 2024-02-26T05:00:40.876696+0000 (oldest deadline 2024-02-26T05:01:06.776797+0000) 2024-02-26T05:01:56.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:56 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:56.354+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:49.677766+0000 front 2024-02-26T05:00:49.677797+0000 (oldest deadline 2024-02-26T05:01:13.777780+0000) 2024-02-26T05:01:56.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:56 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:56.354+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:57.278817+0000 front 2024-02-26T05:00:57.278940+0000 (oldest deadline 2024-02-26T05:01:22.578976+0000) 2024-02-26T05:01:56.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:56 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:56.354+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:07.879981+0000 front 2024-02-26T05:01:07.879961+0000 (oldest deadline 2024-02-26T05:01:28.980042+0000) 2024-02-26T05:01:56.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:56 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:56.354+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:14.181539+0000 front 2024-02-26T05:01:14.181593+0000 (oldest deadline 2024-02-26T05:01:40.081595+0000) 2024-02-26T05:01:56.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:56 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10[60609]: 2024-02-26T05:01:56.354+0000 7f8fdfd74700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:20.082152+0000 front 2024-02-26T05:01:20.082117+0000 (oldest deadline 2024-02-26T05:01:45.982077+0000) 2024-02-26T05:01:57.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:56 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:56.550+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:57.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:56 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:56.550+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:57.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:56 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:56.550+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:57.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:56 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:56.550+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:57.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:56 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:56.550+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:57.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:56 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:56.550+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:57.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:56 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:56.550+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-26T05:01:28.933480+0000 front 2024-02-26T05:01:28.933362+0000 (oldest deadline 2024-02-26T05:01:53.033283+0000) 2024-02-26T05:01:57.311 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:57 smithi156 podman[69698]: 2024-02-26 05:01:57.010045462 +0000 UTC m=+5.709297128 container died 6809ea1368ca3eddd0bf6d3c22591b330357c319882822032b0cfdf660eab784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-26T05:01:57.311 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:57 smithi156 podman[69698]: 2024-02-26 05:01:57.309165498 +0000 UTC m=+6.008417146 container cleanup 6809ea1368ca3eddd0bf6d3c22591b330357c319882822032b0cfdf660eab784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-26T05:01:57.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:57 smithi156 bash[69698]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10 2024-02-26T05:01:57.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:57 smithi156 podman[69716]: 2024-02-26 05:01:57.485407302 +0000 UTC m=+0.478918304 container remove 6809ea1368ca3eddd0bf6d3c22591b330357c319882822032b0cfdf660eab784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-02-26T05:01:58.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:57 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:57.575+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:58.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:57 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:57.575+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:58.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:57 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:57.575+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:58.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:57 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:57.575+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:58.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:57 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:57.575+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:58.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:57 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:57.575+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:58.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:57 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:57.575+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-26T05:01:28.933480+0000 front 2024-02-26T05:01:28.933362+0000 (oldest deadline 2024-02-26T05:01:53.033283+0000) 2024-02-26T05:01:58.177 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:57 smithi156 podman[69807]: 2024-02-26 05:01:57.911043408 +0000 UTC m=+0.108425323 container create b393947c0d860ce0b28e8fd5fb8db808a4946e7b91b4e625dddf5fe733e10e28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-26T05:01:58.178 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:57 smithi156 podman[69807]: 2024-02-26 05:01:57.819963158 +0000 UTC m=+0.017345083 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T05:01:58.178 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:58 smithi156 podman[69807]: 2024-02-26 05:01:58.146979312 +0000 UTC m=+0.344361244 container init b393947c0d860ce0b28e8fd5fb8db808a4946e7b91b4e625dddf5fe733e10e28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-26T05:01:58.178 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:58 smithi156 podman[69807]: 2024-02-26 05:01:58.150725011 +0000 UTC m=+0.348106934 container start b393947c0d860ce0b28e8fd5fb8db808a4946e7b91b4e625dddf5fe733e10e28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-26T05:01:58.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:58 smithi156 podman[69807]: 2024-02-26 05:01:58.23830152 +0000 UTC m=+0.435683445 container attach b393947c0d860ce0b28e8fd5fb8db808a4946e7b91b4e625dddf5fe733e10e28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T05:01:58.677 INFO:journalctl@ceph.osd.10.smithi156.stdout:Feb 26 05:01:58 smithi156 podman[69807]: 2024-02-26 05:01:58.437664127 +0000 UTC m=+0.635046056 container died b393947c0d860ce0b28e8fd5fb8db808a4946e7b91b4e625dddf5fe733e10e28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-10-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T05:01:58.910 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:58 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:58.583+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:58.910 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:58 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:58.583+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:58.910 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:58 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:58.583+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:58.910 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:58 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:58.583+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:58.910 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:58 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:58.583+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:58.910 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:58 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:58.583+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:58.910 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:58 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:58.583+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-26T05:01:28.933480+0000 front 2024-02-26T05:01:28.933362+0000 (oldest deadline 2024-02-26T05:01:53.033283+0000) 2024-02-26T05:01:59.032 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.10.service' 2024-02-26T05:01:59.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:01:59.073 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-02-26T05:01:59.073 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-02-26T05:01:59.073 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.11 2024-02-26T05:01:59.259 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:59 smithi191 systemd[1]: Stopping Ceph osd.11 for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:01:59.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:59 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:59.384+0000 7f8ab67ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:01:59.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:59 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:59.384+0000 7f8ab67ed700 -1 osd.11 97 *** Got signal Terminated *** 2024-02-26T05:01:59.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:59 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:59.384+0000 7f8ab67ed700 -1 osd.11 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-26T05:01:59.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:59 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:59.598+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:01:59.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:59 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:59.598+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:01:59.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:59 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:59.598+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:01:59.759 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:59 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:59.598+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:01:59.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:59 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:59.598+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:01:59.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:59 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:59.598+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:01:59.760 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:01:59 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:01:59.598+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-26T05:01:28.933480+0000 front 2024-02-26T05:01:28.933362+0000 (oldest deadline 2024-02-26T05:01:53.033283+0000) 2024-02-26T05:02:01.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:00 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:00.552+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:02:01.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:00 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:00.552+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:02:01.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:00 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:00.552+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:02:01.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:00 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:00.552+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:02:01.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:00 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:00.552+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:02:01.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:00 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:00.552+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:02:01.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:00 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:00.552+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-26T05:01:28.933480+0000 front 2024-02-26T05:01:28.933362+0000 (oldest deadline 2024-02-26T05:01:53.033283+0000) 2024-02-26T05:02:02.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:01 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:01.564+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:02:02.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:01 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:01.564+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:02:02.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:01 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:01.564+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:02:02.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:01 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:01.564+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:02:02.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:01 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:01.564+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:02:02.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:01 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:01.564+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:02:02.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:01 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:01.564+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-26T05:01:28.933480+0000 front 2024-02-26T05:01:28.933362+0000 (oldest deadline 2024-02-26T05:01:53.033283+0000) 2024-02-26T05:02:02.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:01 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:01.564+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6820 osd.8 since back 2024-02-26T05:01:37.534760+0000 front 2024-02-26T05:01:37.534742+0000 (oldest deadline 2024-02-26T05:02:01.535277+0000) 2024-02-26T05:02:03.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:02 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:02.526+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:02:03.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:02 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:02.526+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:02:03.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:02 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:02.526+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:02:03.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:02 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:02.526+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:02:03.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:02 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:02.526+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:02:03.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:02 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:02.526+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:02:03.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:02 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:02.526+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-26T05:01:28.933480+0000 front 2024-02-26T05:01:28.933362+0000 (oldest deadline 2024-02-26T05:01:53.033283+0000) 2024-02-26T05:02:03.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:02 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:02.526+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6820 osd.8 since back 2024-02-26T05:01:37.534760+0000 front 2024-02-26T05:01:37.534742+0000 (oldest deadline 2024-02-26T05:02:01.535277+0000) 2024-02-26T05:02:04.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:03 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:03.512+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-02-26T05:00:33.226164+0000 front 2024-02-26T05:00:33.226253+0000 (oldest deadline 2024-02-26T05:00:58.526292+0000) 2024-02-26T05:02:04.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:03 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:03.512+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-02-26T05:00:50.828527+0000 front 2024-02-26T05:00:50.828627+0000 (oldest deadline 2024-02-26T05:01:14.328712+0000) 2024-02-26T05:02:04.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:03 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:03.512+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6814 osd.3 since back 2024-02-26T05:00:54.329251+0000 front 2024-02-26T05:00:54.329219+0000 (oldest deadline 2024-02-26T05:01:20.229087+0000) 2024-02-26T05:02:04.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:03 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:03.512+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-02-26T05:01:04.330502+0000 front 2024-02-26T05:01:04.330319+0000 (oldest deadline 2024-02-26T05:01:29.030329+0000) 2024-02-26T05:02:04.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:03 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:03.512+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-02-26T05:01:13.631830+0000 front 2024-02-26T05:01:13.631728+0000 (oldest deadline 2024-02-26T05:01:39.531855+0000) 2024-02-26T05:02:04.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:03 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:03.512+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.60:6822 osd.6 since back 2024-02-26T05:01:19.532639+0000 front 2024-02-26T05:01:19.532546+0000 (oldest deadline 2024-02-26T05:01:44.832378+0000) 2024-02-26T05:02:04.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:03 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:03.512+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-02-26T05:01:28.933480+0000 front 2024-02-26T05:01:28.933362+0000 (oldest deadline 2024-02-26T05:01:53.033283+0000) 2024-02-26T05:02:04.010 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:03 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11[61541]: 2024-02-26T05:02:03.512+0000 7f8ab2de1700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.191:6820 osd.8 since back 2024-02-26T05:01:37.534760+0000 front 2024-02-26T05:01:37.534742+0000 (oldest deadline 2024-02-26T05:02:01.535277+0000) 2024-02-26T05:02:05.009 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:04 smithi191 podman[70772]: 2024-02-26 05:02:04.740808117 +0000 UTC m=+5.476908659 container died ee7fffe405757991a547a5d8e3a20abbd0b74d5e8fdd982a6bb11d8d9e9f43a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-26T05:02:05.370 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:05 smithi191 podman[70772]: 2024-02-26 05:02:05.09236112 +0000 UTC m=+5.828461684 container cleanup ee7fffe405757991a547a5d8e3a20abbd0b74d5e8fdd982a6bb11d8d9e9f43a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=) 2024-02-26T05:02:05.370 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:05 smithi191 bash[70772]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11 2024-02-26T05:02:05.708 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:05 smithi191 podman[70789]: 2024-02-26 05:02:05.36877857 +0000 UTC m=+0.626180842 container remove ee7fffe405757991a547a5d8e3a20abbd0b74d5e8fdd982a6bb11d8d9e9f43a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-26T05:02:05.982 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:05 smithi191 podman[70880]: 2024-02-26 05:02:05.609756184 +0000 UTC m=+0.017577319 image pull a4519cb49beda23c74b76b4147b00c8e16ecf01864b289b39389859e98927b86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a 2024-02-26T05:02:05.983 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:05 smithi191 podman[70880]: 2024-02-26 05:02:05.719766812 +0000 UTC m=+0.127587933 container create 283b46e55742499721f8d2cb89377ccd63a52327a822d0a6b78427ba2bac6692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD) 2024-02-26T05:02:06.259 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:05 smithi191 podman[70880]: 2024-02-26 05:02:05.981225589 +0000 UTC m=+0.389046727 container init 283b46e55742499721f8d2cb89377ccd63a52327a822d0a6b78427ba2bac6692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-26T05:02:06.259 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:05 smithi191 podman[70880]: 2024-02-26 05:02:05.984322567 +0000 UTC m=+0.392143682 container start 283b46e55742499721f8d2cb89377ccd63a52327a822d0a6b78427ba2bac6692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T05:02:06.259 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:06 smithi191 podman[70880]: 2024-02-26 05:02:06.0727292 +0000 UTC m=+0.480550332 container attach 283b46e55742499721f8d2cb89377ccd63a52327a822d0a6b78427ba2bac6692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-26T05:02:06.610 INFO:journalctl@ceph.osd.11.smithi191.stdout:Feb 26 05:02:06 smithi191 podman[70880]: 2024-02-26 05:02:06.269311526 +0000 UTC m=+0.677132651 container died 283b46e55742499721f8d2cb89377ccd63a52327a822d0a6b78427ba2bac6692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-osd-11-deactivate, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-02-26T05:02:06.899 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@osd.11.service' 2024-02-26T05:02:06.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:02:06.941 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-02-26T05:02:06.942 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-02-26T05:02:06.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.a 2024-02-26T05:02:07.224 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 05:02:06 smithi060 systemd[1]: Stopping Ceph mds.a for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:02:07.480 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 05:02:07 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-a[71584]: 2024-02-26T05:02:07.222+0000 7efca58b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:02:07.480 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 05:02:07 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-a[71584]: 2024-02-26T05:02:07.222+0000 7efca58b7700 -1 mds.a *** got signal Terminated *** 2024-02-26T05:02:11.333 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 05:02:11 smithi060 podman[89825]: 2024-02-26 05:02:11.058615514 +0000 UTC m=+3.920110430 container died b8df58cbd1aedaf43522b1b0ae3de334e41738289e5da0207cd2508fb73073a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-02-26T05:02:11.730 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 05:02:11 smithi060 podman[89825]: 2024-02-26 05:02:11.497059314 +0000 UTC m=+4.358554245 container cleanup b8df58cbd1aedaf43522b1b0ae3de334e41738289e5da0207cd2508fb73073a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-26T05:02:11.730 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 05:02:11 smithi060 bash[89825]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-a 2024-02-26T05:02:12.055 INFO:journalctl@ceph.mds.a.smithi060.stdout:Feb 26 05:02:11 smithi060 podman[89839]: 2024-02-26 05:02:11.764907269 +0000 UTC m=+0.700781585 container remove b8df58cbd1aedaf43522b1b0ae3de334e41738289e5da0207cd2508fb73073a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-a, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T05:02:12.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.a.service' 2024-02-26T05:02:12.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:02:12.116 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-02-26T05:02:12.116 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-02-26T05:02:12.116 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.d 2024-02-26T05:02:12.461 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 05:02:12 smithi060 systemd[1]: Stopping Ceph mds.d for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:02:12.730 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 05:02:12 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-d[70625]: 2024-02-26T05:02:12.458+0000 7fbbf91bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:02:12.730 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 05:02:12 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-d[70625]: 2024-02-26T05:02:12.458+0000 7fbbf91bc700 -1 mds.d *** got signal Terminated *** 2024-02-26T05:02:15.279 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 05:02:15 smithi060 podman[89944]: 2024-02-26 05:02:15.00294193 +0000 UTC m=+2.657455745 container died 5caf4150ff031438d340a21ba5078462b7904da8875038e3573341bb4bd383db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T05:02:15.604 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 05:02:15 smithi060 podman[89944]: 2024-02-26 05:02:15.442769941 +0000 UTC m=+3.097283802 container cleanup 5caf4150ff031438d340a21ba5078462b7904da8875038e3573341bb4bd383db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-26T05:02:15.604 INFO:journalctl@ceph.mds.d.smithi060.stdout:Feb 26 05:02:15 smithi060 bash[89944]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-d 2024-02-26T05:02:15.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.d.service' 2024-02-26T05:02:15.849 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:02:15.850 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-02-26T05:02:15.850 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-02-26T05:02:15.850 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.g 2024-02-26T05:02:16.165 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 05:02:15 smithi060 systemd[1]: Stopping Ceph mds.g for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:02:16.480 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 05:02:16 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-g[71235]: 2024-02-26T05:02:16.162+0000 7fcb201d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:02:16.480 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 05:02:16 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-g[71235]: 2024-02-26T05:02:16.162+0000 7fcb201d8700 -1 mds.g *** got signal Terminated *** 2024-02-26T05:02:20.480 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 05:02:20 smithi060 podman[90065]: 2024-02-26 05:02:20.171155163 +0000 UTC m=+4.107132455 container died eebc6ae4d6a2809770e3a228ab7995bb839054e740349fd652aba92c2de99d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-g, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-26T05:02:20.949 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 05:02:20 smithi060 podman[90065]: 2024-02-26 05:02:20.621509531 +0000 UTC m=+4.557486822 container cleanup eebc6ae4d6a2809770e3a228ab7995bb839054e740349fd652aba92c2de99d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-g, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T05:02:20.949 INFO:journalctl@ceph.mds.g.smithi060.stdout:Feb 26 05:02:20 smithi060 bash[90065]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-g 2024-02-26T05:02:21.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.g.service' 2024-02-26T05:02:21.136 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:02:21.136 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-02-26T05:02:21.136 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-02-26T05:02:21.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.j 2024-02-26T05:02:21.460 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 05:02:21 smithi060 systemd[1]: Stopping Ceph mds.j for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:02:21.729 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 05:02:21 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-j[70273]: 2024-02-26T05:02:21.458+0000 7f3921f33700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:02:21.730 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 05:02:21 smithi060 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-j[70273]: 2024-02-26T05:02:21.458+0000 7f3921f33700 -1 mds.j *** got signal Terminated *** 2024-02-26T05:02:24.419 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 05:02:24 smithi060 podman[90185]: 2024-02-26 05:02:24.144080319 +0000 UTC m=+2.793513607 container died e1af7e57f0a7702f94765e4b7455711e0f69eebe2cdd7d51e522feaf80292dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-j, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-26T05:02:24.730 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 05:02:24 smithi060 podman[90185]: 2024-02-26 05:02:24.575645258 +0000 UTC m=+3.225078600 container cleanup e1af7e57f0a7702f94765e4b7455711e0f69eebe2cdd7d51e522feaf80292dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-j, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-26T05:02:24.730 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 05:02:24 smithi060 bash[90185]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-j 2024-02-26T05:02:25.145 INFO:journalctl@ceph.mds.j.smithi060.stdout:Feb 26 05:02:24 smithi060 podman[90199]: 2024-02-26 05:02:24.885120951 +0000 UTC m=+0.733133323 container remove e1af7e57f0a7702f94765e4b7455711e0f69eebe2cdd7d51e522feaf80292dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-j, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T05:02:25.162 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.j.service' 2024-02-26T05:02:25.195 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:02:25.195 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-02-26T05:02:25.195 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-02-26T05:02:25.195 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.b 2024-02-26T05:02:25.476 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 05:02:25 smithi156 systemd[1]: Stopping Ceph mds.b for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:02:25.741 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 05:02:25 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-b[65871]: 2024-02-26T05:02:25.474+0000 7fbbfaa1d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:02:25.741 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 05:02:25 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-b[65871]: 2024-02-26T05:02:25.474+0000 7fbbfaa1d700 -1 mds.b *** got signal Terminated *** 2024-02-26T05:02:29.770 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 05:02:29 smithi156 podman[69912]: 2024-02-26 05:02:29.482408158 +0000 UTC m=+4.101812189 container died 4b36e180c94762421310b7804c0bf476f24505751f536d456b34f933343e6ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-b, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-26T05:02:30.110 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 05:02:29 smithi156 podman[69912]: 2024-02-26 05:02:29.857684043 +0000 UTC m=+4.477088042 container cleanup 4b36e180c94762421310b7804c0bf476f24505751f536d456b34f933343e6ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-26T05:02:30.110 INFO:journalctl@ceph.mds.b.smithi156.stdout:Feb 26 05:02:29 smithi156 bash[69912]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-b 2024-02-26T05:02:30.227 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.b.service' 2024-02-26T05:02:30.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:02:30.265 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-02-26T05:02:30.265 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-02-26T05:02:30.265 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.e 2024-02-26T05:02:30.427 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 05:02:30 smithi156 systemd[1]: Stopping Ceph mds.e for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:02:30.927 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 05:02:30 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-e[65247]: 2024-02-26T05:02:30.577+0000 7f0523791700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:02:30.927 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 05:02:30 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-e[65247]: 2024-02-26T05:02:30.577+0000 7f0523791700 -1 mds.e *** got signal Terminated *** 2024-02-26T05:02:34.064 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 05:02:33 smithi156 podman[70049]: 2024-02-26 05:02:33.724150495 +0000 UTC m=+3.248228635 container died c6fb7835692b26ab6c9611511439b49176d7b4463232740dff9813068e1cecb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-26T05:02:34.322 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 05:02:34 smithi156 podman[70049]: 2024-02-26 05:02:34.144047703 +0000 UTC m=+3.668125820 container cleanup c6fb7835692b26ab6c9611511439b49176d7b4463232740dff9813068e1cecb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=) 2024-02-26T05:02:34.322 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 05:02:34 smithi156 bash[70049]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-e 2024-02-26T05:02:34.551 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.e.service' 2024-02-26T05:02:34.575 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 05:02:34 smithi156 podman[70063]: 2024-02-26 05:02:34.320304865 +0000 UTC m=+0.595625228 container remove c6fb7835692b26ab6c9611511439b49176d7b4463232740dff9813068e1cecb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-e, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-26T05:02:34.575 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 05:02:34 smithi156 systemd[1]: ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.e.service: Deactivated successfully. 2024-02-26T05:02:34.575 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 05:02:34 smithi156 systemd[1]: Stopped Ceph mds.e for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T05:02:34.575 INFO:journalctl@ceph.mds.e.smithi156.stdout:Feb 26 05:02:34 smithi156 systemd[1]: ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.e.service: Consumed 1.307s CPU time. 2024-02-26T05:02:34.587 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:02:34.587 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-02-26T05:02:34.587 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-02-26T05:02:34.587 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.h 2024-02-26T05:02:34.924 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 05:02:34 smithi156 systemd[1]: Stopping Ceph mds.h for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:02:35.177 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 05:02:34 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-h[65555]: 2024-02-26T05:02:34.922+0000 7f167d2da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:02:35.177 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 05:02:34 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-h[65555]: 2024-02-26T05:02:34.922+0000 7f167d2da700 -1 mds.h *** got signal Terminated *** 2024-02-26T05:02:37.984 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 05:02:37 smithi156 podman[70169]: 2024-02-26 05:02:37.723132642 +0000 UTC m=+2.926115233 container died d4a9ec314a770eb5bd26be64f11baec6f6865eea3fd7b6f1b17a5107ba69346a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-h, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-26T05:02:38.350 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 05:02:38 smithi156 podman[70169]: 2024-02-26 05:02:38.080855933 +0000 UTC m=+3.283838518 container cleanup d4a9ec314a770eb5bd26be64f11baec6f6865eea3fd7b6f1b17a5107ba69346a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-h, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-02-26T05:02:38.350 INFO:journalctl@ceph.mds.h.smithi156.stdout:Feb 26 05:02:38 smithi156 bash[70169]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-h 2024-02-26T05:02:38.504 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.h.service' 2024-02-26T05:02:38.538 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:02:38.539 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-02-26T05:02:38.539 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-02-26T05:02:38.539 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.k 2024-02-26T05:02:38.927 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 26 05:02:38 smithi156 systemd[1]: Stopping Ceph mds.k for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:02:38.927 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 26 05:02:38 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-k[64921]: 2024-02-26T05:02:38.842+0000 7f197bead700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:02:38.928 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 26 05:02:38 smithi156 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-k[64921]: 2024-02-26T05:02:38.842+0000 7f197bead700 -1 mds.k *** got signal Terminated *** 2024-02-26T05:02:41.561 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 26 05:02:41 smithi156 podman[70289]: 2024-02-26 05:02:41.235494116 +0000 UTC m=+2.486556570 container died a5348b5648366ae3853859154570e9b6c164764190f5e4be1fb1fcd264bbc293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-26T05:02:41.561 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 26 05:02:41 smithi156 podman[70289]: 2024-02-26 05:02:41.55932676 +0000 UTC m=+2.810389214 container cleanup a5348b5648366ae3853859154570e9b6c164764190f5e4be1fb1fcd264bbc293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-k, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-26T05:02:41.927 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 26 05:02:41 smithi156 bash[70289]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-k 2024-02-26T05:02:41.927 INFO:journalctl@ceph.mds.k.smithi156.stdout:Feb 26 05:02:41 smithi156 podman[70303]: 2024-02-26 05:02:41.726883712 +0000 UTC m=+0.488748557 container remove a5348b5648366ae3853859154570e9b6c164764190f5e4be1fb1fcd264bbc293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-k, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T05:02:41.976 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.k.service' 2024-02-26T05:02:42.009 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:02:42.010 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-02-26T05:02:42.010 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-02-26T05:02:42.010 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.c 2024-02-26T05:02:42.309 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 05:02:42 smithi191 systemd[1]: Stopping Ceph mds.c for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:02:42.759 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 05:02:42 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-c[66585]: 2024-02-26T05:02:42.306+0000 7fbb061e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:02:42.759 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 05:02:42 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-c[66585]: 2024-02-26T05:02:42.306+0000 7fbb061e0700 -1 mds.c *** got signal Terminated *** 2024-02-26T05:02:45.985 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 05:02:45 smithi191 podman[70985]: 2024-02-26 05:02:45.720365361 +0000 UTC m=+3.519691681 container died 6288967d3c308315c5164fd9026c5af04a8076aa91ea73bf6c70bf329923706a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-02-26T05:02:46.259 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 05:02:46 smithi191 podman[70985]: 2024-02-26 05:02:46.074392601 +0000 UTC m=+3.873718918 container cleanup 6288967d3c308315c5164fd9026c5af04a8076aa91ea73bf6c70bf329923706a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-c, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-26T05:02:46.260 INFO:journalctl@ceph.mds.c.smithi191.stdout:Feb 26 05:02:46 smithi191 bash[70985]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-c 2024-02-26T05:02:46.493 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.c.service' 2024-02-26T05:02:46.529 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:02:46.529 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-02-26T05:02:46.530 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-02-26T05:02:46.530 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.f 2024-02-26T05:02:46.759 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 05:02:46 smithi191 systemd[1]: Stopping Ceph mds.f for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:02:47.259 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 05:02:46 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-f[66272]: 2024-02-26T05:02:46.852+0000 7f83eddd1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:02:47.259 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 05:02:46 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-f[66272]: 2024-02-26T05:02:46.852+0000 7f83eddd1700 -1 mds.f *** got signal Terminated *** 2024-02-26T05:02:49.989 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 05:02:49 smithi191 podman[71105]: 2024-02-26 05:02:49.72339518 +0000 UTC m=+2.979059167 container died 6fac6472ec87cc4a0b5b0923132f520557e1f6c2da816ae7a648bb6143dffdcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-f, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-26T05:02:50.256 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 05:02:50 smithi191 podman[71105]: 2024-02-26 05:02:50.06964115 +0000 UTC m=+3.325305083 container cleanup 6fac6472ec87cc4a0b5b0923132f520557e1f6c2da816ae7a648bb6143dffdcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-f, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T05:02:50.256 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 05:02:50 smithi191 bash[71105]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-f 2024-02-26T05:02:50.484 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.f.service' 2024-02-26T05:02:50.508 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 05:02:50 smithi191 podman[71119]: 2024-02-26 05:02:50.254208 +0000 UTC m=+0.524310199 container remove 6fac6472ec87cc4a0b5b0923132f520557e1f6c2da816ae7a648bb6143dffdcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-26T05:02:50.508 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 05:02:50 smithi191 systemd[1]: ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.f.service: Deactivated successfully. 2024-02-26T05:02:50.508 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 05:02:50 smithi191 systemd[1]: Stopped Ceph mds.f for f12e2206-d461-11ee-95c3-87774f69a715. 2024-02-26T05:02:50.508 INFO:journalctl@ceph.mds.f.smithi191.stdout:Feb 26 05:02:50 smithi191 systemd[1]: ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.f.service: Consumed 1.343s CPU time. 2024-02-26T05:02:50.519 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:02:50.520 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-02-26T05:02:50.520 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-02-26T05:02:50.520 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.i 2024-02-26T05:02:50.759 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 05:02:50 smithi191 systemd[1]: Stopping Ceph mds.i for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:02:51.259 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 05:02:50 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-i[66892]: 2024-02-26T05:02:50.872+0000 7f07b41da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:02:51.259 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 05:02:50 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-i[66892]: 2024-02-26T05:02:50.872+0000 7f07b41da700 -1 mds.i *** got signal Terminated *** 2024-02-26T05:02:53.975 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 05:02:53 smithi191 podman[71228]: 2024-02-26 05:02:53.72116851 +0000 UTC m=+2.981893167 container died e344c5b8d29b24b33a4a3c7bcf28c619abc958aeb036f82fd53b00c316d53868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-i, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-26T05:02:54.259 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 05:02:54 smithi191 podman[71228]: 2024-02-26 05:02:54.064575141 +0000 UTC m=+3.325299772 container cleanup e344c5b8d29b24b33a4a3c7bcf28c619abc958aeb036f82fd53b00c316d53868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-i, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-26T05:02:54.259 INFO:journalctl@ceph.mds.i.smithi191.stdout:Feb 26 05:02:54 smithi191 bash[71228]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-i 2024-02-26T05:02:54.481 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.i.service' 2024-02-26T05:02:54.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:02:54.515 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-02-26T05:02:54.515 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-02-26T05:02:54.515 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.l 2024-02-26T05:02:54.836 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 05:02:54 smithi191 systemd[1]: Stopping Ceph mds.l for f12e2206-d461-11ee-95c3-87774f69a715... 2024-02-26T05:02:55.259 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 05:02:54 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-l[65951]: 2024-02-26T05:02:54.834+0000 7f9385466700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-26T05:02:55.259 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 05:02:54 smithi191 ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-l[65951]: 2024-02-26T05:02:54.834+0000 7f9385466700 -1 mds.l *** got signal Terminated *** 2024-02-26T05:02:58.679 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 05:02:58 smithi191 podman[71348]: 2024-02-26 05:02:58.313400472 +0000 UTC m=+3.583371012 container died 1639a982b18c1104303635d396f823ea29082e87307e0c30028912405182f25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240216.060239-25296a8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214) 2024-02-26T05:02:59.009 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 05:02:58 smithi191 podman[71348]: 2024-02-26 05:02:58.767838642 +0000 UTC m=+4.037809159 container cleanup 1639a982b18c1104303635d396f823ea29082e87307e0c30028912405182f25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b75d31dc2b226f1149fda56cab6d379418706a76c50f66e794dca8e5877e1c8a, name=ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-l, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240216.060239-25296a8) 2024-02-26T05:02:59.009 INFO:journalctl@ceph.mds.l.smithi191.stdout:Feb 26 05:02:58 smithi191 bash[71348]: ceph-f12e2206-d461-11ee-95c3-87774f69a715-mds-l 2024-02-26T05:02:59.186 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f12e2206-d461-11ee-95c3-87774f69a715@mds.l.service' 2024-02-26T05:02:59.219 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-26T05:02:59.220 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-02-26T05:02:59.220 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f12e2206-d461-11ee-95c3-87774f69a715 --force --keep-logs 2024-02-26T05:02:59.418 INFO:teuthology.orchestra.run.smithi060.stdout:Deleting cluster with fsid: f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T05:03:01.866 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f12e2206-d461-11ee-95c3-87774f69a715 --force --keep-logs 2024-02-26T05:03:02.059 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T05:03:04.579 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f12e2206-d461-11ee-95c3-87774f69a715 --force --keep-logs 2024-02-26T05:03:04.769 INFO:teuthology.orchestra.run.smithi191.stdout:Deleting cluster with fsid: f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T05:03:07.163 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T05:03:07.197 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T05:03:07.232 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-26T05:03:07.260 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-26T05:03:07.262 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376/remote/smithi060/crash 2024-02-26T05:03:07.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/crash -- . 2024-02-26T05:03:07.307 INFO:teuthology.orchestra.run.smithi060.stderr:tar: /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/crash: Cannot open: No such file or directory 2024-02-26T05:03:07.307 INFO:teuthology.orchestra.run.smithi060.stderr:tar: Error is not recoverable: exiting now 2024-02-26T05:03:07.309 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376/remote/smithi156/crash 2024-02-26T05:03:07.310 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/crash -- . 2024-02-26T05:03:07.348 INFO:teuthology.orchestra.run.smithi156.stderr:tar: /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/crash: Cannot open: No such file or directory 2024-02-26T05:03:07.348 INFO:teuthology.orchestra.run.smithi156.stderr:tar: Error is not recoverable: exiting now 2024-02-26T05:03:07.350 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376/remote/smithi191/crash 2024-02-26T05:03:07.351 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/crash -- . 2024-02-26T05:03:07.388 INFO:teuthology.orchestra.run.smithi191.stderr:tar: /var/lib/ceph/f12e2206-d461-11ee-95c3-87774f69a715/crash: Cannot open: No such file or directory 2024-02-26T05:03:07.388 INFO:teuthology.orchestra.run.smithi191.stderr:tar: Error is not recoverable: exiting now 2024-02-26T05:03:07.390 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-26T05:03:07.390 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-02-26T05:03:07.544 INFO:teuthology.orchestra.run.smithi060.stdout:2024-02-26T04:49:16.735076+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-26T05:03:07.544 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-26T05:03:07.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[SEC\]' /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-02-26T05:03:07.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]' /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-02-26T05:03:07.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[WRN\]' /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-02-26T05:03:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:2024-02-26T04:49:16.735076+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-26T05:03:07.726 INFO:tasks.cephadm:Compressing logs... 2024-02-26T05:03:07.726 DEBUG:teuthology.orchestra.run.smithi060:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-26T05:03:07.769 DEBUG:teuthology.orchestra.run.smithi156:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-26T05:03:07.771 DEBUG:teuthology.orchestra.run.smithi191:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-26T05:03:07.793 INFO:teuthology.orchestra.run.smithi060.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-26T05:03:07.793 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-26T05:03:07.793 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-02-26T05:03:07.797 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.6.log 2024-02-26T05:03:07.797 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mon.a.log 2024-02-26T05:03:07.797 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.audit.log 2024-02-26T05:03:07.797 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.g.log 2024-02-26T05:03:07.797 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.j.log 2024-02-26T05:03:07.797 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.3.log 2024-02-26T05:03:07.797 INFO:teuthology.orchestra.run.smithi060.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-02-26T05:03:07.798 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mgr.x.log 2024-02-26T05:03:07.798 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.0.log 2024-02-26T05:03:07.816 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mgr.x.log: /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.3.log: 90.0% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.audit.log.gz 2024-02-26T05:03:07.816 INFO:teuthology.orchestra.run.smithi060.stderr: 94.7% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.g.log.gz 2024-02-26T05:03:07.816 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.cephadm.log 2024-02-26T05:03:07.816 INFO:teuthology.orchestra.run.smithi060.stderr: 94.8% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.j.log.gz 2024-02-26T05:03:07.816 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.log 2024-02-26T05:03:07.816 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-volume.log 2024-02-26T05:03:07.817 INFO:teuthology.orchestra.run.smithi060.stderr: 82.8% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.cephadm.log.gz 2024-02-26T05:03:07.817 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.9.log 2024-02-26T05:03:07.817 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-volume.log: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-26T05:03:07.817 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.log.gz 2024-02-26T05:03:07.817 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.d.log 2024-02-26T05:03:07.817 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.a.log 2024-02-26T05:03:07.817 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.d.log.gz 2024-02-26T05:03:07.817 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.a.log: 94.7% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.a.log.gz 2024-02-26T05:03:07.817 INFO:teuthology.orchestra.run.smithi060.stderr: 91.0% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-volume.log.gz 2024-02-26T05:03:07.818 INFO:teuthology.orchestra.run.smithi191.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-26T05:03:07.818 INFO:teuthology.orchestra.run.smithi191.stderr:: No such file or directory 2024-02-26T05:03:07.818 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.l.log 2024-02-26T05:03:07.818 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.audit.log 2024-02-26T05:03:07.819 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.i.log 2024-02-26T05:03:07.819 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.l.log: /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.11.log 2024-02-26T05:03:07.819 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mgr.z.log 2024-02-26T05:03:07.819 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.i.log: /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.8.log 2024-02-26T05:03:07.819 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.5.log 2024-02-26T05:03:07.819 INFO:teuthology.orchestra.run.smithi191.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-26T05:03:07.819 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.8.log: /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.f.log 2024-02-26T05:03:07.819 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.cephadm.log 2024-02-26T05:03:07.819 INFO:teuthology.orchestra.run.smithi191.stderr: 90.2% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.audit.log.gz 2024-02-26T05:03:07.819 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.f.log: 94.9% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.l.log.gz 2024-02-26T05:03:07.819 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.log 2024-02-26T05:03:07.820 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.2.log 2024-02-26T05:03:07.820 INFO:teuthology.orchestra.run.smithi191.stderr: 82.6%/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.log: -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.cephadm.log.gz 2024-02-26T05:03:07.820 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.c.log 2024-02-26T05:03:07.820 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.2.log: 86.0% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.log.gz 2024-02-26T05:03:07.820 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-volume.log 2024-02-26T05:03:07.820 INFO:teuthology.orchestra.run.smithi191.stderr: 92.2%/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.c.log: -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mgr.z.log.gz 2024-02-26T05:03:07.820 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mon.c.log 2024-02-26T05:03:07.824 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-volume.log: /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mon.c.log: 91.0% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-volume.log.gz 2024-02-26T05:03:07.825 INFO:teuthology.orchestra.run.smithi156.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-26T05:03:07.825 INFO:teuthology.orchestra.run.smithi156.stderr:: No such file or directory 2024-02-26T05:03:07.825 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.10.log 2024-02-26T05:03:07.825 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.audit.log 2024-02-26T05:03:07.825 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.b.log 2024-02-26T05:03:07.825 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.10.log: /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.4.log 2024-02-26T05:03:07.825 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.1.log 2024-02-26T05:03:07.826 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.b.log: /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.4.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-26T05:03:07.826 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.k.log 2024-02-26T05:03:07.826 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.h.log 2024-02-26T05:03:07.826 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.7.log 2024-02-26T05:03:07.826 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.e.log 2024-02-26T05:03:07.826 INFO:teuthology.orchestra.run.smithi156.stderr: 90.1% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.audit.log.gz 2024-02-26T05:03:07.826 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mgr.y.log 2024-02-26T05:03:07.826 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.e.log: 94.6% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.k.log.gz 2024-02-26T05:03:07.826 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.cephadm.log 2024-02-26T05:03:07.826 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mgr.y.log: 92.1% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mgr.y.log.gz 2024-02-26T05:03:07.826 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.log 2024-02-26T05:03:07.826 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.cephadm.log.gz 2024-02-26T05:03:07.826 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-volume.log 2024-02-26T05:03:07.827 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.log: 86.0% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph.log.gz 2024-02-26T05:03:07.827 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mon.b.log 2024-02-26T05:03:07.830 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-volume.log: /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-volume.log.gz 2024-02-26T05:03:07.838 INFO:teuthology.orchestra.run.smithi156.stderr: 93.8% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.e.log.gz 2024-02-26T05:03:07.839 INFO:teuthology.orchestra.run.smithi191.stderr: 93.8% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.f.log.gz 2024-02-26T05:03:07.857 INFO:teuthology.orchestra.run.smithi191.stderr: 93.4% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.i.log.gz 2024-02-26T05:03:07.861 INFO:teuthology.orchestra.run.smithi156.stderr: 93.6% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.h.log.gz 2024-02-26T05:03:07.989 INFO:teuthology.orchestra.run.smithi060.stderr: 89.6% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mgr.x.log.gz 2024-02-26T05:03:08.017 INFO:teuthology.orchestra.run.smithi156.stderr: 92.0% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mon.b.log.gz 2024-02-26T05:03:08.020 INFO:teuthology.orchestra.run.smithi191.stderr: 91.9% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mon.c.log.gz 2024-02-26T05:03:08.085 INFO:teuthology.orchestra.run.smithi191.stderr: 95.9% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.c.log.gz 2024-02-26T05:03:08.591 INFO:teuthology.orchestra.run.smithi060.stderr: 90.7% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mon.a.log.gz 2024-02-26T05:03:08.893 INFO:teuthology.orchestra.run.smithi191.stderr: 92.0% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.11.log.gz 2024-02-26T05:03:09.437 INFO:teuthology.orchestra.run.smithi156.stderr: 91.8% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.4.log.gz 2024-02-26T05:03:09.505 INFO:teuthology.orchestra.run.smithi156.stderr: 91.7% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.10.log.gz 2024-02-26T05:03:09.583 INFO:teuthology.orchestra.run.smithi156.stderr: 92.0% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.1.log.gz 2024-02-26T05:03:09.655 INFO:teuthology.orchestra.run.smithi060.stderr: 93.1% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.0.log.gz 2024-02-26T05:03:09.737 INFO:teuthology.orchestra.run.smithi191.stderr: 92.2% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.8.log.gz 2024-02-26T05:03:09.817 INFO:teuthology.orchestra.run.smithi156.stderr: 92.4% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.7.log.gz 2024-02-26T05:03:09.817 INFO:teuthology.orchestra.run.smithi060.stderr: 92.7% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.6.log.gz 2024-02-26T05:03:09.834 INFO:teuthology.orchestra.run.smithi060.stderr: 92.5% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.9.log.gz 2024-02-26T05:03:09.895 INFO:teuthology.orchestra.run.smithi060.stderr: 92.4% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.3.log.gz 2024-02-26T05:03:09.897 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-02-26T05:03:09.897 INFO:teuthology.orchestra.run.smithi060.stderr:real 0m2.116s 2024-02-26T05:03:09.897 INFO:teuthology.orchestra.run.smithi060.stderr:user 0m8.760s 2024-02-26T05:03:09.897 INFO:teuthology.orchestra.run.smithi060.stderr:sys 0m0.298s 2024-02-26T05:03:10.472 INFO:teuthology.orchestra.run.smithi191.stderr: 93.3% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.2.log.gz 2024-02-26T05:03:10.720 INFO:teuthology.orchestra.run.smithi156.stderr: 93.5% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-mds.b.log.gz 2024-02-26T05:03:10.722 INFO:teuthology.orchestra.run.smithi156.stderr: 2024-02-26T05:03:10.722 INFO:teuthology.orchestra.run.smithi156.stderr:real 0m2.936s 2024-02-26T05:03:10.722 INFO:teuthology.orchestra.run.smithi156.stderr:user 0m10.063s 2024-02-26T05:03:10.722 INFO:teuthology.orchestra.run.smithi156.stderr:sys 0m0.318s 2024-02-26T05:03:12.535 INFO:teuthology.orchestra.run.smithi191.stderr: 95.0% -- replaced with /var/log/ceph/f12e2206-d461-11ee-95c3-87774f69a715/ceph-osd.5.log.gz 2024-02-26T05:03:12.537 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-02-26T05:03:12.537 INFO:teuthology.orchestra.run.smithi191.stderr:real 0m4.751s 2024-02-26T05:03:12.537 INFO:teuthology.orchestra.run.smithi191.stderr:user 0m10.674s 2024-02-26T05:03:12.537 INFO:teuthology.orchestra.run.smithi191.stderr:sys 0m0.376s 2024-02-26T05:03:12.537 INFO:tasks.cephadm:Archiving logs... 2024-02-26T05:03:12.537 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376/remote/smithi060/log 2024-02-26T05:03:12.539 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-26T05:03:13.916 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376/remote/smithi156/log 2024-02-26T05:03:13.917 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-26T05:03:15.483 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376/remote/smithi191/log 2024-02-26T05:03:15.484 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-26T05:03:17.043 INFO:tasks.cephadm:Removing cluster... 2024-02-26T05:03:17.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f12e2206-d461-11ee-95c3-87774f69a715 --force 2024-02-26T05:03:17.239 INFO:teuthology.orchestra.run.smithi060.stdout:Deleting cluster with fsid: f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T05:03:17.505 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f12e2206-d461-11ee-95c3-87774f69a715 --force 2024-02-26T05:03:17.697 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T05:03:17.958 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f12e2206-d461-11ee-95c3-87774f69a715 --force 2024-02-26T05:03:18.151 INFO:teuthology.orchestra.run.smithi191.stdout:Deleting cluster with fsid: f12e2206-d461-11ee-95c3-87774f69a715 2024-02-26T05:03:18.414 INFO:tasks.cephadm:Removing cephadm ... 2024-02-26T05:03:18.414 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-26T05:03:18.432 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-26T05:03:18.448 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-26T05:03:18.463 INFO:tasks.cephadm:Teardown complete 2024-02-26T05:03:18.463 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-26T05:03:18.474 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-26T05:03:18.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-26T05:03:18.477 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-26T05:03:18.490 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-26T05:03:18.601 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-26T05:03:18.611 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-26T05:03:18.611 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T05:03:18.614 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T05:03:18.616 DEBUG:teuthology.orchestra.run.smithi191:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T05:03:18.628 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: ntpq: command not found 2024-02-26T05:03:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-02-26T05:03:18.629 INFO:teuthology.orchestra.run.smithi191.stderr:bash: line 1: ntpq: command not found 2024-02-26T05:03:18.632 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T05:03:18.632 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2024-02-26T05:03:18.632 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +2440us[+2440us] +/- 83ms 2024-02-26T05:03:18.632 INFO:teuthology.orchestra.run.smithi156.stdout:^* hv02.front.sepia.ceph.com 3 6 377 16 -1831us[-1735us] +/- 56ms 2024-02-26T05:03:18.632 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +337us[ +337us] +/- 87ms 2024-02-26T05:03:18.632 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T05:03:18.634 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T05:03:18.634 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-02-26T05:03:18.634 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 15 +2467us[+2467us] +/- 82ms 2024-02-26T05:03:18.635 INFO:teuthology.orchestra.run.smithi191.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -1833us[-1753us] +/- 56ms 2024-02-26T05:03:18.635 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +346us[ +346us] +/- 87ms 2024-02-26T05:03:18.635 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T05:03:18.635 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T05:03:18.635 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-02-26T05:03:18.635 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 16 +2439us[+2431us] +/- 82ms 2024-02-26T05:03:18.635 INFO:teuthology.orchestra.run.smithi060.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -1827us[-1835us] +/- 56ms 2024-02-26T05:03:18.635 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +340us[ +340us] +/- 87ms 2024-02-26T05:03:18.635 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T05:03:18.635 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-26T05:03:18.645 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-26T05:03:18.645 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-26T05:03:18.655 DEBUG:teuthology.orchestra.run.smithi060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-26T05:03:18.763 DEBUG:teuthology.orchestra.run.smithi156:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-26T05:03:18.834 DEBUG:teuthology.orchestra.run.smithi191:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-26T05:03:18.903 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T05:03:18.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T05:03:18.936 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T05:03:18.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T05:03:18.966 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T05:03:18.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T05:03:18.996 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-26T05:03:19.007 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-26T05:03:19.036 INFO:teuthology.task.internal:Duration was 1494.990008 seconds 2024-02-26T05:03:19.037 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-26T05:03:19.046 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-26T05:03:19.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-26T05:03:19.049 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-26T05:03:19.051 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-26T05:03:19.092 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T05:03:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T05:03:19.094 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T05:03:19.457 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-26T05:03:19.457 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2024-02-26T05:03:19.458 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-26T05:03:19.479 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2024-02-26T05:03:19.479 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-26T05:03:19.522 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi191.front.sepia.ceph.com 2024-02-26T05:03:19.522 DEBUG:teuthology.orchestra.run.smithi191:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-26T05:03:19.549 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-26T05:03:19.550 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-26T05:03:19.552 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-26T05:03:19.565 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-26T05:03:19.618 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-26T05:03:19.618 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-26T05:03:19.623 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-26T05:03:19.632 DEBUG:teuthology.orchestra.run.smithi191:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-26T05:03:20.029 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-26T05:03:20.043 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-26T05:03:20.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-26T05:03:20.070 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-26T05:03:20.097 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-26T05:03:20.125 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-26T05:03:20.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-26T05:03:20.138 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-26T05:03:20.140 DEBUG:teuthology.orchestra.run.smithi191:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-26T05:03:20.161 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2024-02-26T05:03:20.163 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2024-02-26T05:03:20.190 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = core 2024-02-26T05:03:20.224 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-26T05:03:20.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T05:03:20.238 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-26T05:03:20.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T05:03:20.252 DEBUG:teuthology.orchestra.run.smithi191:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-26T05:03:20.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T05:03:20.278 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-26T05:03:20.291 INFO:teuthology.task.internal:Transferring archived files... 2024-02-26T05:03:20.292 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376/remote/smithi060 2024-02-26T05:03:20.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-26T05:03:20.336 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376/remote/smithi156 2024-02-26T05:03:20.337 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-26T05:03:20.374 DEBUG:teuthology.misc:Transferring archived files from smithi191:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376/remote/smithi191 2024-02-26T05:03:20.374 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-26T05:03:20.413 INFO:teuthology.task.internal:Removing archive directory... 2024-02-26T05:03:20.413 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-26T05:03:20.415 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-26T05:03:20.418 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-26T05:03:20.461 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-26T05:03:20.473 INFO:teuthology.task.internal:Not uploading archives. 2024-02-26T05:03:20.473 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-26T05:03:20.484 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-26T05:03:20.485 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-26T05:03:20.487 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-26T05:03:20.490 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-26T05:03:20.499 INFO:teuthology.orchestra.run.smithi060.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 05:03 /home/ubuntu/cephtest 2024-02-26T05:03:20.501 INFO:teuthology.orchestra.run.smithi156.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 05:03 /home/ubuntu/cephtest 2024-02-26T05:03:20.515 INFO:teuthology.orchestra.run.smithi191.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 05:03 /home/ubuntu/cephtest 2024-02-26T05:03:20.517 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-26T05:03:20.528 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-26T05:03:20.551 INFO:teuthology.nuke:Checking targets against current locks 2024-02-26T05:03:20.577 DEBUG:teuthology.nuke:shortname: smithi060 2024-02-26T05:03:20.577 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T05:03:20.600 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 04:26:44.043620', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBE0248Lkbm9nd8IorZk9W5/tVxMKneLUXTFgFtnIyoDsvbZN+X4FsywsLHE9eMiv/vEsUlUjppN2yM/rWKzMBE='} 2024-02-26T05:03:20.605 DEBUG:teuthology.nuke:shortname: smithi156 2024-02-26T05:03:20.605 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T05:03:20.624 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 04:26:44.045440', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI1FvUUWLV6SY0EJCg0XhSRZuVm3ltLE4Mk/1+rauJq+nj2S/YKmIOQWxo/3vlJWh1MlDKZLLkhZZr0CPBQXVvc='} 2024-02-26T05:03:20.627 DEBUG:teuthology.nuke:shortname: smithi191 2024-02-26T05:03:20.627 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T05:03:20.648 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573376', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 04:26:44.046581', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAzMIylPe644StE4DN8EqmhqurfOL15oHbwkuNjcE0WfcivbJH6NbPUMoNOEQK7pOFksjD1OXmaOZMPSPCRk/6I='} 2024-02-26T05:03:20.657 INFO:teuthology.orchestra.console.smithi060:Power off 2024-02-26T05:03:20.657 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-26T05:03:20.680 DEBUG:teuthology.orchestra.console.smithi060:power off output: Chassis Power Control: Down/Off 2024-02-26T05:03:20.680 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T05:03:20.698 INFO:teuthology.orchestra.console.smithi156:Power off 2024-02-26T05:03:20.698 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-26T05:03:20.708 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2024-02-26T05:03:20.720 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2024-02-26T05:03:20.720 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T05:03:20.736 INFO:teuthology.orchestra.console.smithi191:Power off 2024-02-26T05:03:20.737 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-26T05:03:20.745 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-02-26T05:03:20.761 DEBUG:teuthology.orchestra.console.smithi191:power off output: Chassis Power Control: Down/Off 2024-02-26T05:03:20.762 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T05:03:20.784 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-02-26T05:03:24.710 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T05:03:24.745 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T05:03:24.785 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T05:03:24.821 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2024-02-26T05:03:24.855 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-02-26T05:03:24.896 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-02-26T05:03:28.826 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T05:03:28.855 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T05:03:28.897 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T05:03:28.936 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is off 2024-02-26T05:03:28.964 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-02-26T05:03:29.006 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2024-02-26T05:03:29.037 INFO:teuthology.orchestra.console.smithi060:Power off completed 2024-02-26T05:03:29.107 INFO:teuthology.orchestra.console.smithi191:Power off completed 2024-02-26T05:03:32.969 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T05:03:33.079 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2024-02-26T05:03:33.180 INFO:teuthology.orchestra.console.smithi156:Power off completed 2024-02-26T05:03:33.285 INFO:teuthology.run:Summary data: description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/direct_io}} duration: 1494.9900076389313 failure_reason: '"2024-02-26T04:49:16.735076+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: scheduled_vshankar@teuthology success: false 2024-02-26T05:03:33.285 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-26T05:03:33.366 INFO:teuthology.run:FAIL